Att 3.4 Local Title II and IV Protocol

OMB No. 1290-XXXX
Exp. Date XX/XX/2021

3.4 PROTOCOL FOR LOCAL TITLES II AND IV STAFF
Instructions to site visitors: Bullets below each question represent probes for important details;
depending on how well you are doing on time, try to make sure respondents touch on each of
these issues. Text in brackets [ ] should be tailored based on the state or local area. Sections in
gray are the high priority sections for this respondent.

Introduction
Thank you for taking the time to meet with me today. My name is [NAME] and I work for
[Mathematica Policy Research/Social Policy Research Associates]. I am part of an independent
research team conducting a study of WIOA implementation on behalf of the U.S. Department of
Labor. Thank you very much for agreeing to participate in this conversation.
The purpose of this three-year study is to help DOL understand how states are implementing
changes to the core workforce programs authorized under Titles I and III of WIOA, as well as
how those changes are influenced by integration with other partners such as adult education,
vocational rehabilitation, and TANF. To help us better understand WIOA’s implementation, we
are conducting site visits to 14 states, including two local areas in each state, to learn about their
experiences. The information you share will help us understand the range of experiences that
states have had, including key successes and challenges.
We have about [duration] for our conversation. I want to let you know that all interview data will
be reported in the aggregate and your name will never be mentioned in any report that we write,
though we might use quotes from your interview to illustrate findings, without using your name.
I would also like to record our conversation so that I can listen to it later as I complete my notes,
and I will not share the recording with anyone outside of the research team. Being part of this
discussion is voluntary, and you may choose not to answer a question if you wish, or to pause the
recording at any time. Do you have any objections to being part of this interview or to my
recording our discussion?
Okay, I’m going to turn on the audio recorder now.

Respondent Background
1. [If not already known for each person participating in the interview]
a. What is your official title and role?
b. What are your main responsibilities related to implementation of WIOA?

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection
displays a valid OMB control number. Public reporting burden for this collection of information is estimated to average 1 hour and 30 minutes
per response, including time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and
completing and reviewing the collection of information. The obligation to respond to this collection is voluntary. Send comments regarding the
burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to
[email protected] and reference the 0MB Control Number 1290-XXXX. Comments can also be mailed to: U.S. Department of
Labor, Chief Evaluation Office, 200 Constitution Ave., NW, S-2312, Washington, DC 20210. Note: Please do not return the completed interview
guide to the email or mailing address.

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 3.4

A. Governance and Planning
1. Local Plan Development
1.

Describe your general planning process for developing your local plan. What entity
and staff members led the process?
a. How and to what extent was your agency involved?

2.

Did this process differ from previous local plan development efforts?
a. Was the process longer than with previous plans?
b. Was your program more involved? Less involved?
c. How did any changes to the process from prior planning efforts improve the final
plan?
d. What were you able to achieve that had not been possible before? What did you
lose, if anything?

3.

Do you have any lessons to share on developing your local plan around what worked
well or has been challenging?

4.

Have there been changes in federal or state funding for the workforce system that
have affected your plan or how the plan will be implemented?
a. If there have been significant cuts to funding, how are you adjusting the system in
response to these cuts?
b. What will be the impacts of these cuts on infrastructure, programming, and
service delivery?

2. Regional Designations and Regional Plans
1.

How satisfied are you with your state-designated region under WIOA?
a. Do you think it accurately reflects your regional labor market?

2.

Has being part of a region helped you to better engage with regional employers? With
regional economic development entities?

3.

Were you involved in developing the regional plan?

4.

[If yes]: What challenges did you experience in developing your regional plans? What
worked well?
a.

Is there anything you think should change related to the regional planning
process for the next time?

3. Local Workforce Development Board Role
1.

How, if at all, has your agency’s representation on the local board changed as a result
of WIOA?

2.

How, if at all, have the role and approaches of the board changed under WIOA?

3.40

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 3.4

a. Is it more strategic, more innovative, more focused on continuous improvement,
more focused on improving service integration?
b. Why has the role of the board changed?
i. New board mandates under WIOA?
ii. Required changes in membership?
iii. Some other reason?
a)

Employer Engagement

1.

What policies and strategies has your local board adopted to support employer
engagement? How successful have these been?

2.

How has employer representation on the Board changed, and how have these changes
affected employer engagement in the workforce system?

3.

How have employer engagement efforts and activities changed the workforce system
in your local area?
b)

Sector Strategies

1.

What policies and strategies has your state adopted to increase the use of sector
strategies in the workforce system?

2.

What policies and strategies has your local board adopted to increase the use of sector
strategies in the workforce system? Involving new partners, expanding services and
training options, targeting new populations, etc.?

3.

Has the focus on developing and expanding sector strategies been at the state, local,
or regional levels?

4.

What funding sources are being used to develop and support your sector strategy
programs and initiatives (e.g., local formula funds, WIOA state set-aside funds, Title
II or IV funding)?

B. AJC System
1. AJC Certification
1.

Were you at all involved with the AJC certification process, particularly around
ensuring access for customers with disabilities? If so:
a. What challenges did you face and how did you address them?
b. What worked well in the certification process?
c. How useful do you think the certification process is in ensuring AJC
effectiveness, accessibility, and continuous improvement? How could it be
improved?

3.41

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 3.4

2. Infrastructure Funding and MOUs
1.

How does your MOU (excluding resource sharing components) differ from the last
one you had under WIA?

2.

What was included in your budget for One-Stop infrastructure and other costs?
a. Why are you contributing for those components?
b. How much was cash versus in-kind? Why?

3.

What was the process for working with the LWDB determining proportionate use and
relative benefit for your program?
a. How well did that process work?
b. How could it have been improved?

4.

How has the WIOA RSA requirement affected your partnership (either positively or
negatively)?

5.

How often is the LWDB reviewing your agreement (for issues like value of non-cash
contributions, proportionate use and relative benefit)?
a. Are there any challenges related to this review process?

6.

What challenges did you face in the process of developing your MOU and budget?

7.

Has your state agency taken any steps to facilitate negotiation of One-Stop operating
budgets (e.g. meetings at the state level, with state partners, to facilitate their
participation)?

8.

Did your area achieve consensus on your IFA?
a. [If so]: Why do you think your area succeeded?
b. [If not]: Which required partner(s) did not agree? What were the sticking points?
Where are you in the process of reaching agreement?

9.

How helpful was the guidance provided by your state and USDOL on resource
sharing and MOUs? What could have been more helpful

10.

What TA have you participated in on MOUs and resource sharing?
a. How helpful was this TA?
b. What could have been more helpful?

11.

What additional guidance or technical assistance do you need from your state or
USDOL on MOUs and resource sharing?

3. One-Stop Operator
1.

Operators have different responsibilities in different areas; for your local area, what is
the operator’s role as it relates to:
a. Partner meetings?
b. Cross-training sessions?

3.42

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 3.4

c. Coordinating AJC operations & policies?
d. Managing AJC facilities?
e. Implementation of MOUs/RSAs?
2.

Does the operator have any other significant roles in your local area?

3.

How and why, if at all, has the operator’s role changed with the implementation of
WIOA?

4.

What is the role of the AJC manager?

5.

How and why has the AJC manager’s role changed, if at all, with the implementation
of WIOA?

6.

How, if at all, do you think these changes in what entity serves as the operator will
affect AJC operations?

4. Collocation at AJCs
1.

Across all the AJCs in the local area, how often is your program collocated at
comprehensive centers? At affiliate sites?
a. What do you think are the primary reasons for this?

2.

[If program is not collocated at comprehensive AJC(s)]: Are you providing access to
services via training an onsite staff member or via a “direct linkage”?

3.

Why did you opt for that approach and how is it being operationalized?
a. What challenges and benefits have you experienced with each approach?

4.

Has your collocation at AJCs changed since WIOA implementation? If so, how and
why?

5.

What guidance have you received from the state on collocation of partners at AJCs
under WIOA?
a. Did that guidance change from what was provided under WIA? If so how?
b. Is any additional guidance needed?

C. Overall AJC Streamlining & Accessible Services
1. Accessibility
1.

How integrated and streamlined is service delivery in the local area?

2.

Please describe any new state or local policies and actions that promote a more
seamless, accessible and customer-focused workforce service delivery network?

3.

Has the state or LWDB encouraged or required any changes to your AJC customer
intake process? What changes did this involve?

3.43

WIOA IMPLEMENTATION EVALUATION

4.

OMB SUPPORTING STATEMENT: PROTOCOL 3.4

To what extent has the state or LWDB emphasized customer or human-centered
design as a key method for making your state’s One-Stop career center system more
customer-focused?
a. What problems or goals do you plan to address through this approach?
b. What challenges and benefits have you encountered in operationalizing it?

5.

Please describe any state or local efforts to improve the accessibility of One-Stops for
individuals with disabilities (e.g. policies, additional funding, or training).

2. Referral Process
1.

How frequently do you make referrals to partner programs?
a. Which partners are you most likely to initiate referrals to?
b. Which partners are customers most frequently referred to your program from?

2.

How do you make referrals to and from other partners?
a. Do staff provide contact information for the program to a customer?
b. Do staff connect directly with the program they are referring a customer to?
c. Do staff submit a referral via shared MIS?
d. Or, is there some other referral process?

3.

Do you track your referrals to partners? Is information shared about referred
customers with the program the customer is being referred to?

4.

What do you think is the most successful aspect of your referral process?

5.

Is there anything about the referral process you would like to improve?

3. Co-enrollment
1.

About how frequently are your customers co-enrolled in partner programs? Which
programs and for what reasons?

2.

How, if at all, do co-enrollment processes differ from referral processes?

3.

How, if at all, is information about your co-enrolled customers shared with partner
programs?

4.

What is the most successful aspect(s) of your co-enrollment process?
a. What requirements particularly facilitate co-enrollment?
b. Are there factors that impede co-enrollment?

5.

Is there anything about local co-enrollment processes you would like to improve?

4. AJC Partner Meetings
1.

Do you have regular AJC partner meetings? (If no, skip to cross-training)

2.

Are these meetings for individual AJCs or only across the local area?

3.44

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 3.4

3.

Do you attend regularly?

4.

Who from your program attends these meetings? Program managers? Line staff?
Both?

5.

How helpful have these meetings been?
a. Is there any way they could have been improved?

5. Cross-training
1.

Have you organized or participated in any cross-training sessions across different
programs? (If not, skip to 9)

2.

How helpful have these sessions been?
a. Is there any way they could have been improved?

3.

In what other ways—if any—do you have the opportunity to share program
information with other partners?
a. How well do these other methods work?

6. Use of shared MIS
1.

Which management information system (MIS) do you use? For what purposes?
i. Case management?
ii. Participant tracking?
iii. Performance reporting?
iv. Fiscal reporting?

2.

Can other partners use this system too? If so:
a. Can all partners view data? Are there specific fields that are restricted?
b. Can all partners enter data into the system? Are there specific fields that only
some partners can enter?
c. Can all partners revise data? Are there specific fields than only some partners can
modify?

3.

How do you ensure that the data being entered into these systems by different
partners is accurate?

4.

How are you protecting the security of the data being entered and stored?

5.

How important is it for seamless services that partner programs use the same MIS?

D. Performance Accountability
1. Common Measures
1.

Have you faced any challenges implementing the WIOA common performance
measures?

3.45

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 3.4

2.

Have you faced any challenges in implementing state-specific performance
measures?

3.

Have you faced any challenges in implementing local performance measures?

E. Services for Job Seekers and Employers
1.

Has WIOA changed your approach to providing services to businesses? In what way?

2.

What strategies have you implemented to increase the use of work-based and jobdriven training (such as OJT, apprenticeships, transitional training, internships)?
a. What work-based training models are most commonly used in your local area?
b. How is your local area working to improve the use of registered apprenticeship?
c. Are there regulations around apprenticeship that could or should be relaxed to
improve the use of registered apprenticeship in your state/region?

3.

Please describe your efforts to increase the use of registered apprenticeship programs.
a. Have there been any challenges related to including apprenticeships on the ETPL?

4.

Please describe your efforts to improve access and services to more barriered
customers, including disabled customers and those with low basic skills.

5.

Have you received any guidance or TA from the state on improving services to
customers with barriers? How helpful was it?

6.

What, if any, additional guidance should USDOL or your state should provide on
serving customers with barriers?

G. Overall Questions
1. Partnerships
1.

How would you assess relationships among core partners? Required partners? Other
partners?
a. Has this changed because of WIOA?
b. If not, why do you think WIOA’s focus on enhanced partnerships has not affected
your partnerships?

2.

Are there sufficient systems in place to ensure regular and effective communication
between partners?

3.

What have been the key challenges and successes related to increasing collaboration
across partners?
a. Are there differences in the extent of collaboration between core partners versus
other required partners, such as UI, SCSEP, Veterans programs, and TANF?

4.

Overall, how integrated are workforce services now?

3.46

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 3.4

a. Has this changed because of WIOA?
b. What changes reflect this more integrated service delivery? For example, are case
managers of different programs meeting more often to discuss co-enrolled clients?
5.

[If respondent feels workforce system is not integrated]: Why do you think WIOA’s
focus on enhanced partnerships has not affected the integration of services?

2. Systems Change
1.

Overall, do you think the workforce system here has changed as a result of WIOA?
a. If so, what are the major changes you can attribute to WIOA? E.g. employer
engagement, partnerships, services more streamlined, data sharing, others?
b. If not, do you feel systems had already begun to change prior to WIOA, or that
efforts under WIOA have not yet been effective in changing the system?

2.

Have there been any negative, unintended consequences of implementing the WIOA
regulations?
a. Are there ways in which you feel the new regulations actually make it more
difficult for you to achieve your goals?

3.

[If respondent doesn’t think WIOA has changed the system]: Do you feel systems had
already begun to change prior to WIOA, or that efforts under WIOA have not yet
been effective in changing the system?

4.

What additional changes are planned or needed to transform your workforce system
to meet the goals of WIOA?
b. What changes would you like to see to make the workforce system better overall?

3. Guidance and Technical Assistance
1. Of the TA provided by DOL national and regional offices on WIOA implementation,
which assistance has been most helpful?
a. Which TA formats and types of materials have been most useful for you?
2. Are you satisfied with the level of TA DOL has provided on WIOA implementation?
3. Have you accessed the ION site to obtain TA materials or guidance on specific issues? If
so, which issues?
4. In what areas would you like to receive additional TA?

Wrap-up
1. How would you describe your overall progress to date with WIOA implementation?
a. Where do you still have plans to make significant changes?
2. Beyond what we’ve discussed today, are there other areas you would like to highlight?
3. Are there any other areas of WIOA implementation our study should explore?

3.47

This page has been left blank for double-sided copying.