Document
1-column report template
ICR 202606-0970-006 · OMB unassigned · Object 169661600.
Document Viewer [docx]
Document Metadata
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
|---|---|
| File Title | 1-column report template |
| Author | Katie Eddins |
| Last Modified By | Writer |
| File Modified | 2026-06-09 |
| File Created | 2026-06-17 |
| Conversion State | complete |
Extracted Text
TANF Pilot Evaluation Pilot staff discussion guide Introduction and Consent Thank you for meeting with us today. My name is [NAME] and my colleague is [NAME] and we are with an independent research organization called [Mathematica/The Adjacent Possible]. We are here to learn about your experience with [TANF PROGRAM NAME/PILOT NAME]. As you may know, your state is among five states that were selected by the U.S. Department of Health and Human Services (HHS) to participate in pilots to try something different in TANF programs, as authorized by the Fiscal Responsibility Act of 2023 (FRA). Mathematica, The Adjacent Possible, and the American Public Human Services Association are conducting the TANF Pilot Evaluation to understand and learn from these pilots under contract with the Administration for Children and Families at HHS. The TANF Pilot Evaluation will help policymakers, program administrators, and researchers learn more about measuring program performance in TANF programs. The goal of this data collection is to talk with program and key partner leadership and staff at all levels to learn about the services you provide to participants, how you measure program performance, your experiences, and lessons you have learned. Our conversation will take about [90 MINUTES if program or partner leadership; 60 MINUTES if program or partner data specialist, supervisor, or frontline staff]. Before we start, I want to let you know that your participation in this study is voluntary. You have the option to skip any question or end the discussion at any time without any consequences. We are interested in your experiences and perspectives; there are no right or wrong answers. We will keep your individual responses completely private and use them only for research purposes. We will not share them with anyone outside our research team. We have obtained a Certificate of Confidentiality from the National Institutes of Health. This helps us protect participants’ privacy. This means no one can force the study team to give out information that identifies them, even in court. The certificate does not prevent us from making disclosures required by law, such as threats of harm or abuse. We will summarize information collected today in future publications. We will combine responses for reporting purposes, and we will never report names or identify any responses with a particular person. I would like to record our conversation so I don’t miss anything. No one will have access or listen to the recording except for people on the study team. All recordings and notes will be stored in a secure network environment maintained by the study team and will be deleted once the study is complete. Is it okay with you if I record this conversation? If you want me to turn the recorder off for any reason or at any time, just say so. [Interviewer: If yes, turn the recorder on. If no, do not record and take very detailed notes instead.] Okay, I have now turned on the recorder. Now that I have the recorder on, do you agree to participate in this interview and have the conversation recorded? [Interviewer: Get verbal consent on the tape recording.] [Interviewer: State your name, the respondent’s first name only, the location, and the date.] PAPERWORK REDUCTION ACT OF 1995 (Public Law 104-13) STATEMENT OF PUBLIC BURDEN: Through this information collection, ACF is gathering information to learn more about measuring program performance in TANF programs. Public reporting burden for this collection of information is estimated to average [60/90] minutes per respondent, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. This is a voluntary collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information subject to the requirements of the Paperwork Reduction Act of 1995, unless it displays a currently valid OMB control number. The OMB # is 0970-0XXX and the expiration date is XX/XX/XXXX. If you have any comments on this collection of information, please contact Quinn Moore at [email protected]. Pilot Staff Discussion Guide Table 1. Discussion questions by respondent type Section Leadership Data specialists Supervisors Frontline staff A. Respondent background X X X X B. Community and economic context X X X C. Pre-pilot program X X D. Intended pilot design X X E. Work environment X X X X F. Data and performance measurement X X X X G. Staffing and staff development X X X X H. Services and service delivery X X I. Reflections on the pilot X X X X J. Wrap-up X X X X A. Respondent background (All respondents) 1. Tell me about your role and background with the organization. (All) • What is your current job title and main responsibilities? • How long have you been in this position? How long have you been with the organization? What other roles have you held here? B. Community and economic context (Leadership, supervisors, and frontline staff) 1. Tell me about the labor market in your area. (Leadership and supervisors) • How would you describe the local labor market in [STATE/LOCAL AREA/REGION]?* • Who are the biggest employers in the area?* Do they partner with the TANF program?* • What kinds of jobs are available for the people you serve?* Which are high-growth jobs?* Which are suitable for TANF participants? • Since we last visited, what positive and negative changes have occurred in the economy in [STATE/LOCAL AREA/REGION]? 2. Tell me about the demographics and geography of your area. (Leadership and supervisors) • What are the community demographics of this [STATE/LOCAL AREA/REGION]?* Have there been any recent shifts or trends? • What are the urban/rural dynamics or population distribution?* Have there been recent changes in where people live? • Are there any geographic factors in the area that affect participants’ access to services? • Since we last visited, how have community demographics or population dynamics changed? 3. How is TANF administered in your state? (Leadership) • What is the administrative structure of the TANF program?* (Probes: county or state administered, lead human services agency, number of offices/local areas, regional support) • What services are typically provided in-house?* What services are typically contracted or referred out?* For contracted services, which organizations provide them?* Are any co-located?* What level of service coordination happens? 4. Tell me about the availability of basic support services in your area. (Leadership, supervisors, and frontline staff) • How would you describe the availability and accessibility of supports in the [STATE/LOCAL AREA] outside of the TANF program?* (Probes: affordable housing, public transportation, child care, employment) • How has the service environment changed since our last visit, if at all? 5. How does the political context of your state affect the TANF program? (Leadership) • How has the current political environment in the state affected [TANF PROGRAM/TANF PILOT]? • How have changes to policy, regulations, or funding affected [TANF PROGRAM/TANF PILOT]? (Probes: federal, state, or local changes) • What role has the state legislature or existing state statutes played in [TANF PROGRAM/TANF PILOT]? • What else is important for us to know when it comes to the local political environment or context for [TANF PROGRAM/TANF PILOT]? C. Pre-pilot program (Leadership and data specialists) 1. Tell me about your state’s experience with the WPR before the pilot. (Leadership and data specialists) • What was your experience with work participation rates (WPR) before the pilot began (any time prior to October 1, 2025)?* Did the state generally meet its targets?* What challenges did you face? What worked well? • What strategies did your state use to meet the WPR before the pilot?* (Probes: Use of small TANF payments to SNAP recipients, strategies to include/exclude certain participants from the rate calculation) • What happened if your [STATE/LOCAL AREA/REGION] did not meet the WPR targets? • How did being held accountable to the WPR affect your program? (Probes: monitoring participation in defined categories of work activities, limiting countable work activities, distinguishing between core and non-core activities, other requirements of Section 407) • How does the WPR align with your program goals? 2. Tell me about your state’s experience with participant outcomes before the pilot. (Leadership and data specialists) • What participant outcomes data did your state collect prior to the pilot?* (Probes: employment, earnings, or family stability) • How were participant outcomes data collected prior to the pilot?* (Probes: NDNH data accessed by ACF, data-sharing with other agencies, direct data collection from participants) • How were participant outcomes data used prior to the pilot?* D. Intended pilot design (Leadership and data specialists) 1. What do you know about the pilot? (Leadership and data specialists) • What are your goals for the pilot?* What outcomes do you hope to achieve for the program?* For participants?* What do you hope to see change in the TANF pilot? • How have the goals for the pilot changed over time, if at all?* • What is your role in the pilot? 2. What progress has your state made in selecting additional performance measures and negotiating benchmarks for the pilot? (Leadership) • Where is your state in the process of finalizing state-selected performance measures?* Where are you in the process of negotiating benchmarks for required and state-selected measures?* Please share the current measures and benchmarks.* • Thinking back to when you applied to the pilot, how did you decide what additional performance measures to propose? Who was involved in deciding which additional performance measures to propose? How did it go? How did you think about how the measures relate to the services you wanted to offer? • Who was involved in negotiating the performance benchmarks? What support did you receive during the benchmarking process (e.g., program TA provider, outside consultant, [SITE’S TA COACH FROM TOTAL AND/OR SUPPORTT])? How did it go? • What successes, challenges, or lessons learned have you experienced in the process of identifying the state-selected measures and negotiating benchmarks? 3. Tell me what you think of the planned pilot measures and benchmarks. (Leadership and data specialists) • How do the required measures align with your program goals? • How confident are you that your state-selected measures reflect your [TANF PROGRAM/TANF PILOT] goals? • How confident are you in the ability of your [STATE/LOCAL AREA/REGION] to meet the benchmarks? Why? • Since we last visited, have there been any changes made to the planned pilot measures or benchmarks?* 4. Tell me about the plans for collecting and reporting data for the pilot measures. (Leadership and data specialists) • Where is your state in the design process for data collection and reporting for the pilot?* • How does your state plan to use the new performance measures for program accountability? How do you think about accountability for staff? For participants? • What are your plans for collecting, tracking, and reporting participant outcomes data for your state-selected pilot measures? Who is involved in the planning? What help, if any, did you receive from [SITE’S TA COACH FROM TOTAL AND/OR SUPPORTT]? • How do you plan to monitor the collection and tracking of participant outcome data? • How does your state plan to coordinate information sharing with ACF for the required performance measures? Who is involved in the planning? What help, if any, did you receive from [SITE’S TA COACH FROM TOTAL AND/OR SUPPORTT]? • Since we last visited, have there been any changes made to the plans for collecting and tracking participant outcomes?* • What successes, challenges, or lessons learned have you experienced with planning for tracking participant outcomes data? 5. Tell me about your plans for rolling out the pilot. (Leadership) • What is the intended focus population for the pilot?* (Probes: work-eligible individuals, one- or two-parent families) • Where is the pilot intended to be implemented?* (Probes: statewide or selected areas, staggered or universal roll-out) • How was the population selected? How were the locations selected? Who is involved in selecting the populations and locations? What help, if any, did you receive from [SITE’S TA COACH FROM TOTAL AND/OR SUPPORTT]? • How are any new staff being recruited and hired for the pilot? How are any staff being reassigned within the agency? • What staff knowledge, skills, and abilities are needed for your initial pilot plans and why? How are these different than the knowledge, skills, and abilities historically needed by staff? What plans do you have, if any, to support staff in strengthening their knowledge, skills, and abilities to carry out the pilot plans? • Since we last visited, have there been any changes made to the roll-out plans for the pilot?* If so, what changes were made and why? • What successes, challenges, or lessons learned have you had in rolling out the pilot? 6. Tell me about how TANF services are intended to change under the pilot. (Leadership) • Which program services are you intending to change under the pilot?* (Probes: intake, orientation, case management or coaching, work, education or training services, personal or family services) • Tell me how the services are intended to change for the pilot.* • What is changing, if anything, related to the service availability? To the service content? The service frequency or dosage? To the process for receiving services? To staff training or materials? • What is changing, if anything, related to the overall service flow? • What is changing, if anything, about the partners involved in providing services, their responsibilities, or their contracts? • Why are you changing these services? How were these service changes decided? Who was involved? What support, if any, did you receive from [SITE’S TA COACH FROM TOTAL AND/OR SUPPORTT]? • Where is your state in rolling out these service changes for the pilot? What have you completed, what is in progress, and what remains? (Probes: developing new materials or trainings, purchasing services, setting up new contracts with providers) • Since we last visited, have there been any changes to the intended services for the pilot?* If so, why? • What successes, challenges, or lessons learned have you experienced with designing and implementing service changes? • What creative approaches to these service changes, if any, have you seen? • How have the changes been received by staff? How receptive have they been to the changes? 7. Tell me about how the work requirement policies are intended to change under the pilot. (Leadership) • What work requirement policies are you intending to change under the pilot?* (Probes: participant hours requirements, allowable activities, sanction policies, work requirement exemptions) • Why are you changing these policies? How did you determine which policies to change? Who was involved? What support, if any, did you receive from [SITE’S TA COACH FROM TOTAL AND/OR SUPPORTT]? • Where is your state in rolling out these work requirement changes for the pilot? What have you completed, what is in progress, and what remains? (Probes: setting policy specifics, updating staff manuals, updating MIS platforms) • Are there any other TANF policies changing under the pilot?* (Probes: current cash assistance amount, diversion strategies, non-recurrent short-term benefits, use of solely or separately state-funded programs, earned income disregards, time limits, transitional assistance) • Since we last visited, have there been any changes to the intended work requirement policies for the pilot?* If so, why? • What successes, challenges, or lessons learned have you experienced with designing and implementing work requirement policies for the pilot? • How have the changes been received by staff? How receptive have they been to the changes? E. Work environment (All respondents) 1. Tell me about the mission and vision for [TANF PROGRAM/TANF PILOT/YOUR ORGANIZATION]. (All) • What is the mission and vision – or big picture goals – of the TANF program?* • How do the pilot goals relate to the TANF program mission and vision? • Since we last visited, have there been any changes to the mission and vision?* 2. Tell me about how [TANF PROGRAM/TANF PILOT/YOUR PROGRAM] is organized. (Leadership and supervisors) • What is the organizational structure or hierarchy of [TANF PROGRAM/TANF PILOT]?* (Probes: leadership roles, how teams are organized, lines of authority) • What types of staff currently support service delivery in [TANF PROGRAM/TANF PILOT]? What are their roles and responsibilities?* (Probes: eligibility, direct service, and supervision; new or specialized roles for the pilot) • How does the organizational structure affect how [TANF PROGRAM/TANF PILOT] operates? • Since we last visited, have there been any changes to the organizational structure?* If so, what has changed? • What successes, challenges, or lessons learned have you experienced related to the organizational structure for [TANF PROGRAM/TANF PILOT]? 3. Tell me about the overall size of [TANF PROGRAM/TANF PILOT/YOUR PROGRAM]. (Leadership and supervisors) • What is the current number of staff in [TANF PROGRAM/TANF PILOT]?* • What is the budget for [TANF PROGRAM/TANF PILOT]?* • Since we last visited, have there been any changes to the program size or budget?* If so, what has changed? 4. Tell me about what leadership looks like in [TANF PROGRAM/TANF PILOT/YOUR PROGRAM]. (All) • What are leaders’ top priorities? To what extent are leaders able to balance competing priorities? • How does the TANF Pilot fit in with those priorities? What are the competing priorities outside of the pilot? • What information do leaders use to guide decisions? Can you share an example of when leaders used information to make a decision? • How are the perspectives of others incorporated into leaders’ decision-making? Whose perspectives are considered (Probes: frontline staff, community partners, current or former participants)? Can you share an example? • Who is championing the pilot? What does their role look like, and how long have they been in it? • How do leaders’ skills align with an outcomes-focused approach? How have leaders adapted to or evolved during the pilot? • Since we last visited, have there been any changes to leadership?* • What successes, challenges, or lessons learned related to leadership have come up in [TANF PROGRAM/TANF PILOT]? 5. Tell me about how decisions are made in [TANF PROGRAM/TANF PILOT]. (All) • How are decisions made for [TANF PROGRAM/TANF PILOT]? Which decisions are handled at the state level and which at the local level*? (Probes: policy decisions, services/service delivery decisions, funding/resource allocation decisions) • How much flexibility do local service areas have and in what areas of [TANF PROGRAM/TANF PILOT]?* • How does [TANF PROGRAM/TANF PILOT] approach change and program improvement? How are challenges identified? How are they addressed? (Probes: formal vs informal processes, bottom-up or top-down, how is feedback solicited and acted on) • Since we last visited, have there been any changes to how decisions are made?* • What successes, challenges, or lessons learned have you experienced with decision-making for [TANF PROGRAM/TANF PILOT]? 6. Tell me about how resources are allocated in [TANF PROGRAM/TANF PILOT]. (Leadership) • How are decisions made about allocating resources in [TANF PROGRAM/TANF PILOT]? Who is involved? • What costs has your program incurred since our last visit related to [TANF PROGRAM/TANF PILOT]? (Probes: new data systems, hiring additional staff, contracting with new providers, changes to administrative activities, changes to supportive services) • How has your program funded those costs? What sources of funding have been particularly helpful for start-up costs? (Probes: federal TANF funds, state TANF funds/MOE dollars, other federal or state funds such as WIOA or SNAP E&T, mixing or braiding funding) • What other financial changes has your program experienced since our last visit? (Probes: overall funding, how resources are allocated, shifts across TANF-allowable categories) • What successes, challenges, or lessons learned have you experienced with finances for [TANF PROGRAM/TANF PILOT]? 7. Tell me about how partnerships are structured and monitored in [TANF PROGRAM/TANF PILOT]. (Leadership) • What are the key service partnerships for [TANF PROGRAM/TANF PILOT]?* What services do partners provide?* (Probes: interagency partnerships, contracted providers, community partners, employers) • How are contract agreements structured?* (e.g. reimbursement, performance-based contracts, accountability processes) • How is partner performance monitored? What are the consequences for underperformance? • How have key partnerships changed since our last visit, if at all?* • What successes, challenges, or lessons learned have you experienced with partnerships for [TANF PROGRAM/TANF PILOT]? How have partners adapted to an outcomes-focused approach? 8. Tell me about your relationships with service delivery partnerships in [TANF PROGRAM/TANF PILOT]. (Leadership and supervisors) • Which service delivery partners do you regularly interact with, if any? How do you interact with partners? What do you discuss? • Tell me about your relationships with service delivery partners. What is going well? What could be improved? • What successes, challenges, or lessons learned have you experienced with service delivery partners for [TANF PROGRAM/TANF PILOT]? 9. Tell me about the organizational culture in [TANF PROGRAM/TANF PILOT/YOUR PROGRAM]. (All) • What values, principles, and goals guide your TANF program?* (Probes: work-first, personal responsibility, family stability) • How would you describe your organizational culture?* (Probes: flexible/responsive to participant needs, supportive of staff, compliance-focused, outcomes-focused, for example, would you say staff are more focused on compliance or on meeting participant needs?) • What organizational resources (or lack thereof) affect your ability to do your work? What supports does the program have for staff to deal with stressful or challenging situations? • What changes, if any, have there been to the organizational culture since our last visit?* • How has the organization adapted to an outcomes-focused approach during the pilot? • What successes, challenges, or lessons learned have you experienced with the organizational culture for [TANF PROGRAM/TANF PILOT]? 10. Tell me about your experiences with communication in [TANF PROGRAM/TANF PILOT]. (All) • What channels of communication are used in [TANF PROGRAM/TANF PILOT]?* • How does leadership communicate with staff? (Probes: mode, frequency, and for what types of information, for example, about policy or procedure changes) • How do staff communicate with each other? (Probes: mode, frequency, individual and team communication, for example, about participants or changes in policy) • How has communication changed since our last visit, if at all?* • What successes, challenges, or lessons learned have you experienced with communication for [TANF PROGRAM/TANF PILOT]? F. Data and performance measurement (All respondents) 1. Tell me about the current performance measures and accountability in [TANF PROGRAM/TANF PILOT]. (All) • How do you use data or performance measures for program accountability? How do you think about accountability for staff? For participants? • What is [TANF PROGRAM/TANF PILOT] held accountable to? How is program performance measured? What targets is [TANF PROGRAM/TANF PILOT] trying to meet? (Probes: WPR, caseload meets hours requirements employment, earnings, other pilot measures) • What is your local area held accountable for? How is that measured? Are there local targets? • What are you held accountable for as a staff person? How is that measured? Do you have individual targets? • What happens if the targets are not met? • How does being held accountable to these measures affect your program? (Probes: how you manage the program, how you manage staff, how you manage workload, how you engage with participants) • How do the required pilot measures related to employment and earnings affect your program? • How do the state-selected measures, including [STATE-SELECTED MEASURES], affect your program? • How does no longer being held accountable to the WPR affect your program? (Probes: not having to monitor participation in defined categories of work activities, limit countable work activities, distinguish between core and non-core activities, or adhere to other requirements of Section 407 of the Social Security Act)? 2. To what extent do you think [TANF PROGRAM/TANF PILOT] is meeting its performance benchmarks? How does [meeting/not meeting] those benchmarks affect the program? (All) • What other measures or strategies do you use to assess the performance of [TANF PROGRAM/TANF PILOT]? • How has program accountability changed since our last visit, if at all?* • What successes, challenges, or lessons learned have you experienced with program accountability [TANF PROGRAM/TANF PILOT]? 3. Tell me about collecting participant data in [TANF PROGRAM/TANF PILOT]. (All) • What data are you currently collecting about participants’ work activities or progress in [TANF PROGRAM/TANF PILOT]?* (Probes: attendance, hours, engagement, completed goals, sanctions) • How are the data collected?* • When are the data collected and by whom?* (Probes: time period for data collection, time lapse between data collection and data entry, who collects data, who enters it into the system) • How is the data quality monitored? How would you describe the quality of the data? (Probes: completeness, accuracy, non-response) 4. What participant outcomes data are you currently collecting in [TANF PROGRAM/TANF PILOT]?* (Probes: employment, earnings, or family stability, short- and long-term outcomes) (All) • How are the data collected?* (Probes: NDNH data accessed by ACF, data-sharing with other agencies, direct data collection from participants) • When are the data collected and by whom?* (Probes: time period for data collection, time lapse between data collection and data entry, who collects data, who enters it into the system) • How is the data quality monitored? How would you describe the quality of the data? (Probes: completeness, accuracy, non-response) 5. What other participant data are collected in [TANF PROGRAM/TANF PILOT]?* (Probes: demographic information, enrollment information)? (All) • How are the data collected?* • When are the data collected and by whom?* (Probes: time period for data collection, time lapse between data collection and data entry, who collects data, who enters it into the system) • How is the data quality monitored? How would you describe the quality of the data? (Probes: completeness, accuracy, non-response) • Are other participant data matched to outcomes data? How? For what purpose? • Since we last visited, how has data collection changed, if at all?* • What successes, challenges, or lessons learned have you experienced with participant data collection for [TANF PROGRAM/TANF PILOT]? 6. What does participant data tracking and reporting look like? (Leadership and data specialists) • How are data about participants’ work activities or progress currently tracked and reported? Who is the data shared with? In what format? Are any data visualizations or dashboards available? With what frequency are the data shared? • How are participant outcomes data currently tracked and reported? Who is the data shared with? In what format? Are any data visualizations or dashboards available? With what frequency are the data shared? • How does this work for the pilot required measures? • How does this work for state-selected measures? • How are pilot data reported to ACF? • Have any changes been made to the data tracking and reporting processes since our last visit?* • What successes, challenges, or lessons learned have you experienced in tracking and reporting data? 7. Tell me about the data systems you use. (Data specialists, supervisors, and frontline staff) • What data systems are used in [TANF PROGRAM/TANF PILOT/YOUR ORGANIZATION]?* (Probes: who has access, how long system has been in use) (All) • How do data systems interact?* How are they used?* (Probes: repositories, case management) • What data sharing/use agreements are in place?* How were the agreements developed?* • What data systems do you use?* For what purposes?* G. Staffing and staff development (All respondents) 1. Tell me about the roles and qualifications of staff supporting [TANF PROGRAM/TANF PILOT]. (All) • How is your team organized?* How does your role fit into the team structure?* What are your main responsibilities?* • What knowledge, skills, and abilities are needed for your current role and why?* • What knowledge, skills, and abilities are needed for the roles you supervise and why?* • How has your role or responsibilities changed since our last visit, if at all?* • What successes, challenges, or lessons learned have you experienced with your role, responsibilities, or team structure during the pilot? 2. Tell me about training and professional development in [TANF PROGRAM/TANF PILOT/YOUR ORGANIZATION]. (All) • Tell me about the types of training, professional development, coaching, or other support you currently receive to do your job (not including supervision, we’ll get to that next).* (Probes: training topics, initial pilot training, ongoing or follow-up training, training for different types of staff) • How is that training or support provided? How often do you receive it? How long is each session? In what format is it offered? 3. Are you responsible for training any other staff (outside of supervision)? Tell me about that. (All) • How has staff training changed since our last visit, if at all?* • What successes, challenges, or lessons learned have you experienced with training or professional development for [TANF PROGRAM/TANF PILOT]? 4. Tell me about supervision and staff performance in [TANF PROGRAM/TANF PILOT]. (Leadership, supervisors, and frontline staff) • What kinds of supervision do you receive?* (Probes: individual vs. group supervision, topics discussed) • What does that supervision look like? How often do you receive it? How long is each session? In what format is it offered? • How is staff performance assessed? What happens if staff do not perform their jobs satisfactorily? What happens if they excel? (Probes: staff performance measures, performance improvement plans, staff incentives or rewards)? • Do you provide supervision to other staff? If so, tell me about that. (Probes: individual vs. group supervision, format, frequency, dosage, topics discussed, assessing staff performance)? • How many frontline staff do you supervise? Do you carry a caseload as well? • How has supervision changed since our last visit, if at all?* • What successes, challenges, or lessons learned have you experienced with supervision for [TANF PROGRAM/TANF PILOT]? 5. Tell me about the workload of staff working with [TANF PROGRAM/TANF PILOT]. (Leadership, supervisors, and frontline staff) • What is your current caseload? Is that a typical caseload for frontline staff in [TANF PROGRAM/TANF PILOT]? Given their workload, what is a reasonable caseload? What would be the ideal caseload? • How would you describe your workload of frontline staff? Are there enough staff to complete the work needed? (Probes: levels of stress or overwhelm, competing priorities) • To what extent is staff turnover an issue? • How easy or difficult is it to replace staff when there are vacancies? What makes attracting a qualified workforce easy or difficult? • In a typical week, about how much time do you spend working directly with participants? On administrative duties? Engaging with partners or employers? (Probes: time spent documenting or verifying client participation) • [If leader or supervisor] In a typical week, about how much would you say staff you supervise or oversee spend working directly with participants? On administrative duties? Engaging with partners or employers? (Probes: time spent documenting or verifying client participation) • Does the balance between administrative work and work with participants seem right to you? How does the balance align with program goals? • What, if any, administrative bottlenecks or inefficiencies do you experience? How could those be improved? • How has staff workload changed since our last visit, if at all?* • What successes, challenges, or lessons learned have you experienced with workload and administrative activities for [TANF PROGRAM/TANF PILOT]? H. Services and service delivery (Supervisors and frontline staff) 1. Tell me about who is served by [TANF PROGRAM/TANF PILOT]. (Supervisors and frontline staff) • How would you describe the participants served in [TANF PROGRAM/TANF PILOT]?* ◦ What are their key characteristics? ◦ What are their key needs and challenges to employment? • Is it common for participants to cycle on and off TANF? What are the reasons for cycling? What are the consequences for participants and your program? ◦ Are participants regularly co-enrolled in other programs? • Have there been any changes to who is served since our last visit?* If so, what has changed? • What successes, challenges, or lessons learned have you experienced with reaching the intended focus population? 2. Tell me about the services offered by [TANF PROGRAM/TANF PILOT/YOUR ORGANIZATION]. (Supervisors and frontline staff) • What services or supports are currently offered in [TANF PROGRAM/TANF PILOT]?* (Probes: intake, orientation, assessment, case management or coaching, employment services, education and training, personal or family services, other state-specific services) • What service changes, if any, have been made since our last visit?* Why were they made? • Which services, if any, are over-utilized? Why? • Which services, if any, are under-utilized? Why? • What services do you think are particularly effective (or not effective) for the focus population? Why? • What services do you think are particularly high-quality (or low-quality)? Why? 3. Tell me about how the services are delivered. (Supervisors and frontline staff) • How is [SERVICE/SUPPORT] delivered?* (Interviewer: ask or confirm information about each service offered, using Table 2 to help structure the conversation.) • How has service delivery changed since our last visit, if at all?* • What successes, challenges, or lessons learned have you experienced with service delivery with [TANF PROGRAM/TANF PILOT]? Table 2. Service delivery* [Interviewer: Please tailor and duplicate this table as needed to reflect the state and local area visited, the focus of the pilot in that area, and the role of the respondent you are speaking to. You may prepopulate and ask about what has changed from prior interviews.] Name of service Purpose or objective Content or approach Mode Dosage Duration Frequency Providing staff or partner Allowable service customization Tools used [Other state-specific topic] Intake Orientation Assessment Case management or coaching Employment services Education and training Personal or family services Other state-specific services 4. Tell me about how participants access services. (Supervisors and frontline staff) • What is a typical participant’s service journey?* (Probes: recruitment/eligibility, orientation, assessment, service providers, bottlenecks or inefficiencies, drop-offs in service, efficiencies or innovations) • How has the service flow changed since our last visit, if at all?* 5. Tell me about how staff communicate with and about participants. (Supervisors and frontline staff) • How do you communicate with participants?* (Probes: mode, frequency, duration, topics) • How do you communicate with other staff or service providers about participants? (Probes: mode, frequency, duration, topics) • What is the process for referrals? How do these processes vary by service or partner organization? What follow-up, if any, is done after referrals are made? • How has communication with or about participants changed since our last visit, if at all?* • What successes, challenges, or lessons learned have you experienced with communication with or about participants? 6. Tell me about what program tools you use. (Supervisors and frontline staff) • What tools, resources, or models do you use to support service delivery?* (Probes: program manuals, curricula, templates, implementation guides) • Who developed the tools? To what extent are they tailored to the communities you serve? • To what extent and how are they oriented towards helping participants obtain employment or [other state-selected outcomes]? • How have program tools changed since our last visit, if at all?* • What successes, challenges, or lessons learned have you experienced with program tools? 7. Tell me what participant accountability looks like. (Supervisors and frontline staff) • What are the expectations for participant engagement and progress in [TANF PROGRAM/TANF PILOT]?* • What, if anything, are participants required to submit to document their engagement and/or progress?* How do they submit documentation?* How often do they have to submit it?* • How is participant accountability tracked?* Who is involved?* What is the process for inputting documentation and verifying participation?* What data systems or technology are used?* • What happens if a participant does not meet the accountability expectations?* • What happens if a participant cycles in and out of participation in [TANF PROGRAM/TANF PILOT]?* • What changes, if any, have been made to participant accountability since our last visit?* • What successes, challenges, or lessons learned have you experienced with participant accountability? I. Reflections on the pilot (All respondents) 1. Tell me about any key successes, challenges, and lessons learned related to the pilot. (All) • What have been the key successes or benefits from the pilot? • What were the major challenges of the pilot? How were those challenges addressed? What challenges remain? • What key lessons have you learned about outcomes-based performance measures and accountability? • What key lessons have you learned about service delivery? 2. Tell me about how changes from the pilot could be sustained. (All) • What would it take to sustain the changes made under the TANF Pilot? • What challenges do you anticipate? What steps are you taking to minimize them? 3. Tell me what recommendations and suggested supports you have for scaling outcomes-based performance measurement and accountability. (All) • What recommendations would you give to federal TANF block grant administrators and policymakers interested in scaling the pilot program to other states? • What advice would you give to other TANF programs that want to implement outcomes-based performance measures and accountability? • What support would you recommend be provided to other TANF programs to implement outcomes-based performance measures and accountability? J. Wrap-up (All respondents) 1. Is there anything else you would like to share about your TANF program, the pilot, or the federal evaluation? (All) • Thanks so much for talking with me today. It’s important to gather this information for the study and we really appreciate you walking us through your work. • If you have questions, please contact us at [email protected]. Thank you for your time and input! [Interviewer: share project business card with contact information]