Semi-structured in-person interviews with State UI/RESEA administrators

DRAFT DISCUSSION GUIDE:

INTERVIEWS WITH STATE WORKFORCE AGENCIES –

CHARACTERISTICS AND STATUS OF REEMPLOYMENT SERVICES AND

ELIGIBILITY ASSESSMENT (RESEA) PROGRAMS

Introduction

Abt Associates—along with partners the Urban Institute, Capital Research Corporation, and the National Association of State Workforce Agencies (NASWA)—is under contract to the U.S. Department of Labor (USDOL) to conduct a study to provide Congress and USDOL with an in-depth assessment of RESEA state programs. An important goal of this study is to assess current RESEA program operations and how states plan to meet the requirements of the Bipartisan Budget Act (BBA) of 2018 (Public Law 115-123). The RESEA provisions, contained in Section 30206 of the BBA, substantially increased funding for RESEA and introduced a future requirement that RESEA funding be linked to the use of evidence-based interventions. The RESEA provisions contained in the BBA are to be phased in over several years, beginning in FY 2019. This USDOL-funded study, which began in October 2018 and is scheduled to be completed by 2021, is being conducted for the Chief Evaluation Office (CEO) in close collaboration with the Office of Unemployment Insurance (OUI). As part of an implementation study component of this study, the Abt team is conducting interviews with state workforce agencies, with a particular focus on better understanding current RESEA program components and plans for future RESEA program modification and development.

Privacy Statement: Before beginning the interview, I (we) want to thank you for agreeing to participate in this study and remind you that your participation is voluntary. I (we) know that you are busy and will try to be as brief as possible. We have many questions and are going to talk to many different people, so please do not feel as though we expect you to be able to answer every question. The interview today should last no longer than 120 minutes. This interview is not part of an audit or a compliance review. We are interested in learning about your ideas, experiences, and opinions about RESEA and workforce services provided under this program to assist unemployment insurance claimants in your state. There are no right or wrong answers. We want to know what you think. In addition, before we start, I want to let you know that although we will take notes during these interviews, we will, wherever possible, maintain privacy in any written reports that might include the information we learn and in discussions with DOL. To make sure that our notes are complete and accurate, we wanted to ask your permission to record this interview. We will erase this interview recording as soon as we have cleaned up our notes. Do we have your permission to record this interview?

Do you have any questions before we begin?

Do I have your permission to begin the interview?

1. Background on the Respondent(s)

Collect the following information on each respondent involved in the interview:

• Respondent’s name

• Respondent’s state agency, division/unit

• Respondent’s contact information (address, telephone, e-mail)

• Respondent’s title and how long the respondent has been in his/her position

• Specific role of respondent in UI/RESEA program and number of years the respondent has been involved with UI and RESEA (and before that, REA)?

• Other roles (other than related to UI/RESEA), if appropriate.

2. Description of the Current RESEA State Program

[Note to Interviewer: Pre-fill the section below based on information in the state plan and any other available documentation, such as what is available about UI/RESEA from the state’s website and the recent NASWA RESEA survey; where possible, confirm with respondent what is already known as a basis for the discussion.]

1. Participant Selection Process

[UIPL 07-19 Guidance: Before FY 2019, RESEA-funded activities targeted two required populations: (1) UI claimants determined to be most likely to exhaust benefits under the methods established for a state's Worker Profiling and Reemployment Services (WPRS) program; and (2) to the greatest extent feasible, transitioning veterans receiving Unemployment Compensation for Ex­Servicemembers (UCX). New Section 306 of the SSA requires RESEA-funded activities to be targeted to claimants determined most likely to exhaust UI benefits as determined under a profiling system required under Section 303(j), SSA, but does not require that states target transitioning veterans receiving UCX benefits.] However , under the Department's appropriation act for the FY 2019, states are no longer required to target UI claimants determined most likely to exhaust UI benefits under Section 303(j) and have discretion in identifying appropriate claimant populations among recipients of UCX and regular UI benefits.

1. Please provide a brief description of the method by which claimants are selected to participate in RESEA. [Note to Interviewer – request written documentation describing the selection process.]

   1. During which week after initial payment are RESEA participants selected?

   2. Prior to application of the WPRS profiling model (or other selection model, tool, or method), what claimants are excluded from the selection process (e.g., such as those claimants that are part of union hiring halls, those who have a return to work date, etc.)?

   3. Beside high exhaustion risk (as determined by the WPRS model), are there any other factors/criteria used in selecting claimants for their initial RESEA session?

   4. Are claimants throughout the state selected or does RESEA serve only a portion of the state?

1. If only a portion of the state is served, what geographic areas are served and what is excluded?

2. Why is this area of the state served and others not included?

3. Are there any plans to expand the RESEA program to other areas of the state in the next three years? If yes, please describe.

2. Is there any variation in the method in which RESEA participants are selected within the state? If so, please discuss.

   1. Do any substate areas select and seek to engage all claimants (who are not excluded b/c they have a return to work date, etc.) in RESEA?

3. As a result of the BBA provisions (as described in the UIPL 07-19) has your state made changes in how participants are selected for RESEA services? Please discuss any changes.

   1. If changes have been made, has there been any impact on types/characteristics of claimants selected for RESEA? If yes, please discuss.

4. Does your state have plans to make changes to how RESEA participants will be selected in the future (i.e., in the next 3 years)?

   1. If yes, please discuss planned changes and the schedule for implementing these changes.

5. Has the BBA provision eliminating the requirement to target transitioning veterans receiving UCX benefits had an impact on number or types of Ex­Servicemembers participating in your state’s RESEA program? Are you continuing to target UCX? Please discuss.

6. [The UIPL 07-19 Guidance: The Department requires that the selection of targeted UI claimant populations be supported by local labor market information, economic trends, and other available data.]

   1. Has the state made any changes in the way in which it collects data in response to this requirement? If yes, please discuss.

   2. More generally, what role (if any), has your LMI shop or a related research unit played in informing your targeting strategy (e.g. provision of data, special analyses, training)?

7. [The UIPL 07-19 Guidance: The programmatic change with regard to selection of UI claimants was aimed at increasing states ' flexibility to target claimants from a variety of backgrounds and lengths of time receiving benefits based on local needs.]

   1. What, if any, populations do you currently target for selection for RESEA services? Why were these claimants targeted?

   2. To what extent have the RESEA provisions (of the BBA) increased your state’s flexibility with regard to targeting of claimants for RESEA? How?

   3. Has greater flexibility resulted in any change in the characteristics of claimants served by the state’s RESEA program?

   4. If you have not yet taken advantage of greater flexibility to target claimants, do you have any plans to make changes in the future (i.e., in the next three years)?

8. Are there technologies, tools, or other practices you’ve documented that you would recommend to other states, regarding the selection of RESEA participants? If so, can you share the documentation today?

9. Are there technologies, tools, or other practices you would recommend to other states, which facilitate communications and the “hand off” between the UI and workforce development agency staff? If so, can you share the documentation today?

2. RESEA Participation Levels

We have several questions on the numbers of claimants served by your RESEA program. [Note to Interviewer: Prior to the interview, collect data from BLS on number of initial claims for which payment was made and the ETA-9128 (RESEA Workload Report) for the state; pre-fill the table, identify trends/changes, and discuss significant trends/changes with the interviewee.]

Table B1 – Trends in UI Claimants and RESEA Participation, FY 2016 - FY 2018

Data Element	Source	FY 2016	FY 2017	FY 2018	FY 2016-18 Change
a. Number of Initial UI Claims for which a Payment Was Made	BLS
b. Number of Claimants Scheduled for Their First RESEA	Item 1, ETA 9128
c. Percentage Scheduled for Their First RESEA (of Initial UI Claims for Which a Payment was Made)	Calculated (Data Element b/a)
d. Number of RESEAs Scheduled	Item 2, ETA 9128
e. Number of RESEAs Completed	Item 3, ETA 9128
f. Percentage of RESEAs Completed (of those Scheduled)	Calculate (Data Elements e/d)

Note: Calculate percentage change in the final column of the table for FY 2016 to FY 2018 ([FY 2018 – FY 2016]/FY 2016).

1. First, I wanted to briefly discuss any of the data items where there is significant year-to-year variation or where there is substantial percentage change between FY 2016 and FY 2018 (as shown in the final column)

   1. If there is variation in any of the data elements above, what factors might account for significant year-to-year variation?

2. Next, I want to focus on the FY 2018 levels, not the trends over the three years.

   1. Does your state have a goal for the percentage of RESEAs completed (of those scheduled)?

   2. Has your state made it a priority to achieve a higher percentage? If so, why?

3. Do you anticipate that the data elements shown in Table B1 will be similar for the current year (FY 2019)?

   1. If available, please provide estimates (or through the most recent quarter) for the current year (FY 2019).

   2. If figures for data elements in Table B1 are likely to be substantially different from FY 2018, please identify (if known) any factors that might account for these differences. Additionally, if known, have new RESEA provisions of the BBA (or UIPL 07-19) been a factor in explaining differences between FY 2018 and FY 2019? If yes, please discuss.

   3. Looking out beyond FY 2019, do you anticipate further change in any of these date elements (in Table B1)? If so, which ones and why?

3. RESEA Funding

[The UIPL 07-19 Guidance: For FY 2019, RESEA award limits are based on a 25 percent increase to each state's FY 2018 award limit.].

[Note to Interviewer: Prior to the interview, collect data from the UIPL on the state’s FY 2019 funding limit and from the state’s RESEA Grant Proposal on its FY 2019 Proposed RESEA Project Cost.]

1. Based on UIPL 07-19 and your state’s RESEA Grant Proposal, we want to review the following funding limit and grant award for FY 2019 (10/01/2018 – 09/30/2019):

   1. FY 2019 Funding Limit for the state (UIPL 07-19, Attachment I): _____

   2. FY 2019 Proposed RESEA Project Costs for the state (RESEA Grant Proposal): _____

2. If there is a difference between the funding limit and the proposed project cost, what accounts for this difference?

3. What effect has the 25 percent increase in the funding award between FY 2018 and FY 2019 had (or will have) on your state’s program?

   1. In what specific ways has the increase in funding been used (or will be used) (e.g., added staff, purchase of new equipment, changing targeting strategies. expanding RESEA program to other areas of the state, new services, more claimants referred to additional reemployment services, etc.)?

   2. How has the added funding impacted the numbers of UI claimants scheduled for RESEAs?

   3. Overall, has the added funding enhanced your RESEA program? If so, how?

   4. Beyond FY 2019, what effect do you expect this additional funding will have on your state?

4. Initial RESEA Session

[The UIPL 07-19 Guidance (and UIPL 08-18): The following core components must be included in the initial RESEA session:

• UI eligibility assessment, including review of work search activities, and referral to adjudication, as appropriate, if an issue or potential issue(s) is identified;

• Providing support to the claimant to develop and implement an individual reemployment plan;

• Providing labor market and career information that addresses the claimant's specific needs;

• Enrollment in Wagner -Peyser Act-funded Employment Services; and

• Providing information and access to American Job Center (AJC) services and referrals to reemployment services and training, as appropriate, to support the claimant 's return to work.

Please note that the UI eligibility assessment and support with the development of an individual reemployment plan must continue to be provided on a one-on-one basis for the initial RESEA. However, states may provide staff-assisted services remotely using technology such as Skype, Zoom, FaceTime or other similar products. The level and timeliness of remote service must be comparable to assistance the individual would receive if staff were assisting such individual in-person.]

1. Once selected, how are UI claimants notified that they are required to participate in their initial RESEA session (e.g., online notification through UI system, email, telephone, US mail, etc.)?

   1. Please provide an overview of the notification process.

   2. During what payment week (e.g., the week of the claimant’s first payment) are RESEA participants typically notified of their selection for RESEA services? Does this vary? If yes, please explain.

   3. How are these appointments scheduled (e.g., automatically scheduled, self-scheduled)?

      1. What technology platforms, tools, or other methods are used?

      2. Would you use recommend these to other states?

   4. Is the notification and scheduling process the same throughout the state?

   5. Are any changes to the notification or scheduling process for the initial RESEA session planned (e.g., in the next three years)?

2. During which payment week (e.g., the claimant’s 3^rd payment week) are RESEA participants typically scheduled to attend their initial RESEA session?

   1. Is there a time limit for scheduling/re-scheduling of this initial session (i.e., before the 5^th payment week)?

   2. Does timing of this initial RESEA session vary across the state or within local areas?

3. Where is the initial RESEA session conducted?

   1. Are all initial RESEA sessions conducted at AJCs in your state?

      1. If yes, how many total AJCs are there in your state?

• # of comprehensive AJCs____

• # of satellite/affiliated AJCs____

1. Are initial RESEA sessions conducted at all or just some of the AJCs in the state? If not at all AJCs, at how many AJCs are initial RESEA sessions conducted?

• # of comprehensive AJCs____

• # of satellite/affiliated AJCs____

1. Are initial RESEA sessions conducted at any other locations within your state (than AJCs), such as mobile sites or other specialized sites?

   1. If yes, at what other locations are initial RESEA sessions conducted?

   2. For each type of location, how many are there across the state?

2. [If RESEA sessions are conducted at sites other than AJCs – Note: Skip this question if there are no other sites except AJC’s conducting RESEA sessions.] In total, how many locations are there in the state where initial RESEA sessions are conducted? ____

1. Please provide a brief description of the format and content of the initial RESEA session, including any parts of the session some or all claimants may/must complete at home on their own or virtually with staff.. [Note to Interviewer: Check to see if there is a narrative description, handbook, and/or flow chart on the initial RESEA session.]

   1. What are the main activities related to review and enforcement of claimant’s UI eligibility? Please discuss.

   2. What are the main activities related to reemployment services (or career services) specifically provided claimants during their initial RESEA session? Please discuss.

   3. What programming/activities helps people get back to work (e.g., workshop to use LinkedIn, workshop on interviewing skills, soft skills training, basic IT training workshops, other skills training, wraparound services, functionality of state job-search websites, etc.)?

2. On average, how many hours/minutes does it take for a RESEA participant to complete an initial RESEA session? ____hours ____minutes

   1. How much time do each of the following activities typically take:

1. UI eligibility assessment, including review of work search activities, and referral to adjudication, as appropriate, if an issue or potential issue(s) is identified: ____minutes

2. Providing support to the claimant to develop and implement an individual reemployment plan: ____minutes

3. Providing labor market and career information that addresses the claimant's specific needs: ____minutes

4. Enrollment in Wagner -Peyser Act-funded Employment Services: ____minutes

5. Providing information and access to American Job Center (AJC) services and referrals to reemployment services and training, as appropriate, to support the claimant 's return to work: ____minutes

6. Other activities: ____minutes; please specify type of activities:______________

1. With regard to any of the above activities, does the state have any best practices or lessons learned to share?

1. Are claimants encouraged or required to complete any of the core RESEA program components before appearing for initial RESEA session?

   1. If so, what components/activities are completed?

   2. If so, what technologies enable this (agency website, etc.)?

2. Is the initial RESEA session conducted entirely in a one-on-one session with RESEA staff or is some portion conducted in a group setting (e.g., an orientation to AJC services)? If any of the activity is conducted other than one-on-one with staff, which of the following activities are provided in a group versus a one-on-one setting:

• Labor market and career information that addresses the claimant's specific needs

• Enrollment in Wagner -Peyser Act-funded Employment Services

• Information and access to American Job Center (AJC) services and referrals to reemployment services and training, as appropriate, to support the claimant 's return to work

• Other

8. Can any portion of the initial RESEA session(s) be conducted virtually or via the telephone with agency staff? If so, how and under what circumstances?

   1. What activities of the initial RESEA can be conducted virtually or via the telephone?

   2. If virtual, what technology is used (e.g., Skype, Zoom, FaceTime, or other similar products)? Please briefly discuss how this technology is used.

   3. If available, about what proportion of initial RESEA sessions are partially conducted virtually or via the telephone?

9. Is there variation in the way in which the initial RESEA session is conducted/or the content of the session across AJCs in the state (e.g., does the process vary between urban and rural areas)? Please discuss.

10. Is there variation in the way in which the initial RESEA session is conducted/the content of the session by claimant characteristics (e.g., those with high likelihood of exhaustion are provided more intensive or additional services)? Please discuss.

11. Do RESEA participants typically attend an orientation to AJC services with other AJC customers, or is there an orientation for RESEA participants only?

12. Other than initial and subsequent RESEA sessions, are the services available to RESEA participants by the AJCs in your state different from the standard array of services provided to other job seekers using the AJCs? For example, other than the in-person meetings with RESEA staff, are there other types of re-employment services that are specifically targeted on RESEA participants (e.g., on-line services; self-services; job readiness workshops, job clubs)? Please describe.

13. To what extent has development of your RESEA program components, and related technologies, tools or practices, influenced the services or service delivery methods available to other AJC customers? Is there a spill-over benefit?

14. Are there any unique tools or other practices you would recommend to other states, regarding the individual components of the RESEA program? If so, can you share documentation of these with us today?

[Note to Interviewer: Collect documentation of best practices, if available.]

15. Moving forward, does your state have plans to make changes in any of the processes or services provided related to the initial RESEA session? If yes, please discuss.

[Note to Interviewer: Check to see if the state distributes a PowerPoint to local workforce agencies to use for the orientation – if yes, request a copy of the PowerPoint; check to see if there is a narrative description, handbook, and/or flow chart describing the initial RESEA session.]

5. Subsequent RESEA Session

[The UIPL 07-19 Guidance: Requirements for the subsequent RESEA session with a UI claimant, if provided, must include, at a minimum, a UI eligibility assessment and review and/or update of the claimant's individual reemployment plan. The UI eligibility assessment and support, along with the development of an individual reemployment plan, must be provided on a one-on-one basis for the subsequent RESEA. However, states may provide staff-assisted services remotely using technology such as Skype, Zoom, FaceTime, or other similar products. The level and timeliness of remote service must be comparable to assistance the individual would receive if staff were assisting such individual in-person.]

1. Are RESEA participants in the state scheduled for a subsequent RESEA session?

1. If no, are there any plans to add a subsequent RESEA session in the future (e.g., in the next 3 years)? If yes, please discuss briefly. [Note to Interviewer: Skip to next section, Section F, of the discussion guide.]

2. If yes, what is the maximum number of subsequent sessions for which a RESEA participant can be scheduled?

2. Are all RESEA participants attending the initial RESEA session required to attend a subsequent RESEA session? If not, who is/isn’t scheduled for a subsequent RESEA session? When is the RESEA participant scheduled for his/her subsequent session (e.g., at the end of the initial RESEA session)?

1. If applicable (only if a 2^nd subsequent RESEA meeting is scheduled), are all RESEA participants attending the 1^st subsequent RESEA session scheduled for a 2^nd RESEA session? If not, who is/is not scheduled for the 2^nd subsequent RESEA session?

2. If applicable (only if a 3rd subsequent RESEA meeting is scheduled), are all RESEA participants attending the 2^nd subsequent RESEA session scheduled for a 3^rd RESEA session? If not, who is/is not scheduled for the 3^rd subsequent RESEA session?

3. What is the typical timing of the subsequent RESEA session(s) (i.e., during what payment week are RESEA participants typically scheduled)?

1. 1^st subsequent RESEA session: ____ payment week or __ week after the initial RESEA session

2. 2^nd subsequent RESEA session: ____ payment week or __ week after the 1^st subsequent RESEA session

3. 3^rd subsequent RESEA session: ____ payment week or __ week after the 2^nd subsequent RESEA session

4. How are RESEA participants notified of their subsequent RESEA session(s) (e.g., online notification through UI system, email, telephone, US mail, etc.)?

1. When is the RESEA participant scheduled for his/her subsequent session (e.g., at the end of the initial RESEA session)?

2. How are these appointments scheduled (e.g., automatically scheduled, self-scheduled?)

5. Are the locations at which subsequent RESEA sessions are conducted the same as those where initial RESEA sessions are conducted? If not, at what type (e.g., AJCs, libraries) and at how many locations in the state are RESEA subsequent sessions conducted?

6. Are the subsequent RESEA sessions conducted entirely in a one-on-one meeting with RESEA staff or is some portion conducted in a group setting? If any of the activity is conducted other than one-on-one with staff, please discuss those activities.

7. Are claimants encouraged or required to complete any of the core RESEA program components before appearing for their subsequent RESEA session? If so, what components/activities are completed and why?

8. Can any portion of the subsequent RESEA session(s) be conducted virtually or via the telephone with agency staff? If so, how and under what circumstances?

1. If yes, can the entire subsequent RESEA session be conducted virtually or via the telephone?

2. If virtual, what technology is used (e.g., Skype, Zoom, FaceTime, or other similar products)?

3. About what proportion of subsequent RESEA sessions are conducted virtually or via the telephone?

9. Please provide a brief description of the format and content of the subsequent RESEA session. [Note to Interviewer: Check to see if there is a narrative description, handbook, and/or flow chart on the subsequent RESEA session.]

10. On average, how many hours/minutes does it take for a RESEA participant to complete a subsequent RESEA session? ____hours ____minutes

1. How much time do each of the following activities typically take:

1. UI eligibility assessment, including review of work search activities, and referral to adjudication, as appropriate, if an issue or potential issue(s) is identified: ____minutes

2. Providing support to the claimant to develop and implement an individual reemployment plan: ____minutes

3. Other activities: ____minutes; please specify type of activities:______________

11. Is there variation in the way in which the subsequent RESEA session(s) are conducted/the content of the sessions across AJCs in the state (e.g., does the process vary between urban and rural areas)? If yes, please discuss.

12. Is there variation in the way in which the subsequent session(s) are conducted/the content of the sessions by claimant characteristics? If yes, please discuss.

13. Moving forward, does your state have plans to make changes in any of the processes or services provided related to the subsequent RESEA session(s)? If yes, please discuss.

6. Failure to Report and Enforcement of Eligibility Requirements

1. How would you characterize the relative balance between the emphasis on enforcement of eligibility requirements (e.g., work search, able and available) and re-employment assistance in your state? Do you feel that your state has more of an emphasis on enforcement or providing reemployment assistance? Why? Does this balance between enforcement versus reemployment assistance vary by region/locality in the state?

2. Based on ETA 9128 data for the most recent year, we want to briefly discuss the number and proportion of RESEA participants that failed to report (FTR) for FY 2018, as well as some of the reasons RESEA participants were disqualified. [Note to Interviewer: Prior to the interview collect data from the ETA 9128 (RESEA Workload report) for the state; pre-fill the table, identify trends/changes, and discuss significant trends/changes.]

Table F1 – Trends in RESEA Failure to Report/Disqualification, FY 2016 - FY 2018

Data Item	Source	FY 2016	FY 2017	FY 2018	FY 2016-18 Change
Number of Completed RESEAs Resulting in a Disqualification or Overpayment	Item 7, ETA 9128
Number of RESEAs for Which the Claimant Failed to Report (FTR)	Item 15, ETA 9128
Number of RESEAs Rescheduled without Disqualification	Item 16, ETA 9128
Number Disqualified for FTR under Reporting Requirements	Item 17, ETA 9128
Number Disqualified for FTR under Issues Other Than Reporting	Item 18, ETA 9128
Number of Overpayments for FTR	Item 19, ETA 9128
Dollar Amount of Overpayments for FTR	Item 20, ETA 9128
Number Who FTR and Were Not Disqualified or Rescheduled	Item 21, ETA 9128
Number Who FTR Who Returned to Work	Item 22, ETA 9128

3. Discuss significant year-to-year changes in the Table F1.

4. Where in the RESEA process does FTR most often occur?

5. What are the consequences of FTR for the initial RESEA session (e.g., loss of benefits for how long)? What is the specific penalty?

6. What are the consequences of FTR for the subsequent RESEA session(s) (e.g., loss of benefits for how long)? What is the specific penalty?

7. What is the feedback loop with UI on eligibility issues that are identified?

8. Can claimants reschedule a missed meeting? How is that done?

9. What is the timeframe between FTR for a meeting and holds on a claim (e.g., immediate or delayed)?

10. After a FTR, what does a claimant need to do to resume benefits (e.g., how is the penalty cured, such as attending the meeting)?

11. What, if anything, has been done to reduce FTR in the state? Are there any plans to introduce new processes/procedures to reduce FTR in the future (e.g., the next 3 years)?

7. Staffing

1. Typically, what type of staff are involved in administering RESEA at the local level (e.g., within the AJC)?

   1. Is there variation in type of staff involved within the state (i.e., across local workforce areas/AJCs)? If so please discuss.

2. Are there dedicated RESEA staff?

   1. Is there variation across AJCs within the state? If so, please discuss.

3. What role (if any) do UI staff play in RESEA service delivery?

   1. Is there variation across AJCs within the state? If so, please discuss.

4. What role do Wagner-Peyser staff play in RESEA service delivery?

   1. Is there variation across AJCs within the state? If so, please discuss.

5. What role do WIOA staff play in RESEA service delivery?

   1. Is there variation across AJCs within the state? If so, please discuss.

6. Do the same staff conduct eligibility assessments and provide RESEA services? Or are those tasks performed by different staff?

   1. Is there variation across AJCs within the state? If so, please discuss.

7. Are staff cross-trained for any or all RESEA services/components?

   1. Is there variation across AJCs within the state? If so, please discuss.

8. Additional Questions on RESEA Service Delivery and Costs

1. Are there any elements of your RESEA program that you feel are unique or otherwise notable/innovative?

2. Are there any staffing, technology or process innovations or practices you’ve adopted that have enabled significant improvement in RESEA processes or participant outcomes (that you haven’t already mentioned)?

3. What, if any, are your biggest challenges in operating the RESEA program currently?

4. Which program elements do you believe are most critical to helping get UI claimants back to work most quickly?

5. What program elements, if any, do you believe are ineffective?

6. What changes either at the national level or to your state’s RESEA program do you believe would improve the RESEA program?

7. What impact, if any, do you think the expanded RESEA program will have more broadly within AJCs -- on the flow of customers and availability of workforce services, on innovation within the AJCs, on the financial viability of the public workforce infrastructure?

8. Have the RESEA requirements under the BBA of 2018 and/or guidance issued by DOL (UIPL 07-19) led to any changes in the format or content of initial or subsequent RESEA sessions [if not already discussed]? If yes, please briefly describe any changes made.

9. [The UIPL 07-19 Guidance: In FY 2019, administrative costs for the RESEA program are limited to 15 percent of the total grant award. This is a change from FY 2017 and FY 2018.]

1. Has this change limiting administrative costs to 15% had an effect on your program? If yes, please discuss.

2. In your opinion, is this administrative cost limit set at an appropriate level? If not, what should the limit be?

10. [The UIPL 07-19 Guidance: Beginning in FY 2017, ETA implemented a cost per initial RESEA limit of no more than $155 per session and $80 per subsequent session. During FY 2019, states are expected to operate at an average cost per initial RESEA session between $155 and $175 and an average cost per subsequent RESEA session between $80 and $100. States that are unable to operate within these ranges must provide an explanation for the higher costs as part of their grant proposal or submit a grant modification if costs change or are higher than projected during the grant's period of performance.]

[Note to Interviewer: Check state grant proposal to see if the state provided an explanation for higher cost and, if so, the higher cost amount.]

1. How does your state view the change effective in FY 2019 of an average cost per initial RESEA session between $155 and $175 and an average cost per subsequent RESEA session between $80 and $100?

2. What effect do these cost parameters set for FY 2019 have on your state’s provision of initial/subsequent RESEA sessions? If the cost parameters did not exist, are there practices or services your state might be inclined to change in the near term, for some or all claimants?

3. Evidence-Based Strategies and Evaluation Requirements; Technical Assistance and Guidance

[The UIPL 07-19 Guidance: Section 306 of the SSA includes a tiered -evidence approach for the RESEA program to encourage the Department to fund and states to use evidence-based strategies where they exist and to conduct evaluations and build evidence in places where needed. Specifically, the statute requires that interventions or service delivery strategies funded by RESEA grant funds must be demonstrated to reduce the average number of weeks participants receive benefits by improving employment outcomes, including earnings. The Department interprets this to permit states to expend RESEA grant funds on interventions that have not previously been implemented in the field where there is sufficient analysis and reasoning to "demonstrate" that the new intervention will reduce the weeks of UI benefits received through improved employment outcomes. The statute also requires the Secretary of Labor to define and assess whether interventions and service delivery strategies used by states have a "moderate or high causal evidence rating,'' and if an intervention or strategy does not, it must be under evaluation at the time of use. These requirements apply to FY 2019 RESEA grant funds and future year grants funds.] [Note to Interviewer: Prior to the interview, review the state’s RESEA grant proposal with regard to planned evaluation activities; where possible confirm what is included in the plan.]

1. Describe the extent to which your current RESEA program model, or individual elements of it, based on analysis or research evidence developed in your state? Or other states?

   1. Please discuss the type of evidence upon which your current RESEA is based?

2. To what extent are those models, or individual elements of the model, being implemented with fidelity to those described in the evidence?

3. What are the models or individual elements that states are implementing that are not evidence based, and could be studied in the future?

2. How are decisions related to the RESEA program model made in your state, and who is involved within and outside the agency(ies)? Are there controlling legislative provisions (if so, please provide the citations)?

3. Has your state funded any evaluations of its REA/RESEA program in the past 10 years (including current evaluations)?

1. If so, what types of studies were funded (i.e., have studies been RTCs)?

2. What interventions were assessed?

3. What were the key study findings/results, especially with respect to claimant duration and/or employment outcomes? [Note to Interviewer: Request copies of reports.]

4. Has your state been part of any DOL-funded, foundation-funded, or other national evaluations of its REA/RESEA program in the past 10 years (including current evaluations)?

1. If so, in what types of studies did your state participate (i.e., have studies been RTCs)?

2. What interventions were assessed?

3. What were the key study findings/results, especially with respect to claimant duration and/or employment outcomes? [Note to Interviewer: Request copies of reports.]

5. [The UIPL 07-19 Guidance: States may use up to 10 percent of their RESEA grant funds to conduct or cause to conduct evaluations of interventions used in carrying out the program (note: this amount is in addition to the administrative costs discussed above). Because this evaluation funding may not be sufficient to do rigorous evaluations on a state-by-state basis, states are encouraged to pool their funds to support more rigorous evaluations. The Department of Labor may also engage with states to support RESEA evaluations.]

1. In response to this provision, is your state planning to use FY 2019 grant funds to conduct evaluation(s) of your RESEA program? If yes:

1. Will the state use the full 10 percent of its grant to fund evaluation(s) of its RESEA program for FY 2019? If not, what portion of grant funds will be used?

2. What types of studies are planned and what interventions will be assessed?

3. What is the period of performance for the study(ies) (i.e., begin and end date)?

4. Who is involved in the evaluation efforts? Is your LMI shop or related research office helping or leading the efforts? Are you partnering with any other agency units or another agency? Are you partnering with any outside research entities (university or not)?

2. Does your state plan to pool grant funds with other states in FY 2019 to conduct evaluation(s) of your RESEA program? If yes, please discuss (e.g., which states and the nature of the evaluation).

3. In response to this provision, is your state planning to use future grant funds (e.g., FY 2020-2023) to conduct evaluation(s) of your RESEA program? If yes:

1. Will the state use the full 10 percent of its grant to fund evaluation(s) of its RESEA program? If not, what portion of grant funds will be used?

2. What types of studies are planned and what interventions will be assessed?

3. If known, what is the planned period of performance for the future study(ies) (i.e., begin and end date)?

4. In the years ahead (after FY 2019), to what extent is your state interested in and/or planning on being part of larger evaluations across state lines (i.e., to pool grant funds with other states to conduct evaluations with larger sample sizes)? Why or why not?

5. Would your state be willing to participate in future DOL-sponsored studies of RESEA programs? Why or why not?

6. [The UIPL 07-19 Guidance: In FY 2023 and FY 2024, states will be required to use no less than 25 percent of the grant funds for interventions or service delivery strategies with a high or moderate causal evidence rating that show a demonstrated capacity to improve employment and earnings outcomes for program participants. For FY 2025 and 2026, states must use no less than 40 percent of funds for interventions or service delivery strategies with a high or moderate causal evidence rating. For FY 2027 and beyond, states must use no less than 50 percent of RESEA funds for interventions or service delivery strategies with a high or moderate causal evidence rating.]

1. What is the state’s view of the requirement that beginning in FY 2023 and FY 2024 no less than 25 percent of grant funds must be used on service delivery strategies with high or moderate causal evidence ratings? How do you plan to meet this requirement?

2. Does your state expect it will need to make changes to RESEA service delivery in response to this provision?

3. Do you view the percentage of funding (at 25 percent for FY 2023-24; increasing to 40 percent in FY 2025-26; and 50 percent for 2027 and beyond) as set at appropriate levels?

4. Moving forward, how do you think linking funding to high or moderate causal evidence rating might affect your program?

7. [The UIPL 07-19 Guidance: States can use the information in CLEAR to identify studies of strategies with positive findings and a moderate or high causal rating to expand their use of evidence-based interventions for FY 2019. More detailed information and tools to quickly help states identify interventions and strategies to consider will be developed and available for states' use to meet the requirements of FY 2023 and beyond.]

1. In the past, how has the state identified strategies/interventions to use for helping UI claimants to return to work (e.g., looked at impact evaluations, looked at what other states are doing, reviewed academic literature)?

2. What websites or data sources, national conferences, meetings, or other avenues, if any, has the state used to identify potentially effective/innovative practices?

3. Has the state referred to CLEAR to identify potentially effective/innovative strategies/interventions for structuring RESEA interventions? If yes, did the state find CLEAR helpful? Was CLEAR easy to use?

4. To date, has use of CLEAR had any effect on RESEA service delivery interventions?

8. [The UIPL 07-19 Guidance: The Department is committed to supporting state efforts to meet the new RESEA statutory requirements related to evidence and evaluation and is taking the following steps to enhance how interventions and strategies are identified as highly or moderately causal moving forward and to putting in place resources to support technical assistance for states as they navigate these new requirements.]

1. Does the state have any views on the webinars and guidance provided to date to states on the requirements of the BBA with regard to RESEA, especially guidance included in UIPL 07-19? Were they useful? How could they be improved?

2. Moving forward, would technical guidance or assistance in additional areas with regard to the BBA requirements for RESEA be helpful? If yes, what type of guidance is needed?

3. Moving forward, does your state need additional evaluation technical assistance to effectively and efficiently implement the BBA requirements with regard to evidence-based practices? If yes, in what specific areas of evaluation technical assistance are needed and how should USDOL provide this assistance (e.g., webinar, issuance of written guidance, etc.)?

4. Moving forward, are there specific implementation challenges that we have not already discussed that you anticipate encountering in implementing the BBA RESEA provisions? Please discuss.

5. Do you have any final views about the BBA RESEA provisions or the guidance that you have received to date from DOL on these provisions that we haven’t already discussed?

We want to thank you for your time and the thoughtful responses you have provided.

15