Att 3.8 Local AJC Manager Protocol

OMB No. 1290-XXXX
Exp. Date XX/XX/2021

3.8 PROTOCOL FOR AJC MANAGER
Instructions to site visitors: Bullets below each question represent probes for important details;
depending on how well you are doing on time, try to make sure respondents touch on each of
these issues. Text in brackets [ ] should be tailored based on the state or local area. Sections in
gray are the high priority sections for this respondent.

Introduction
Thank you for taking the time to meet with me today. My name is [NAME] and I work for
[Mathematica Policy Research/Social Policy Research Associates]. I am part of an independent
research team conducting a study of WIOA implementation on behalf of the U.S. Department of
Labor. Thank you very much for agreeing to participate in this conversation.
The purpose of this three-year study is to help DOL understand how states are implementing
changes to the core workforce programs authorized under Titles I and III of WIOA, as well as
how those changes are influenced by integration with other partners such as adult education,
vocational rehabilitation, and TANF. To help us better understand WIOA’s implementation, we
are conducting site visits to 14 states, including two local areas in each state, to learn about their
experiences. The information you share will help us understand the range of experiences that
states have had, including key successes and challenges.
We have about [duration] for our conversation. I want to let you know that all interview data will
be reported in the aggregate and your name will never be mentioned in any report that we write,
though we might use quotes from your interview to illustrate findings, without using your name.
I would also like to record our conversation so that I can listen to it later as I complete my notes,
and I will not share the recording with anyone outside of the research team. Being part of this
discussion is voluntary, and you may choose not to answer a question if you wish, or to pause the
recording at any time. Do you have any objections to being part of this interview or to my
recording our discussion?
Okay, I’m going to turn on the audio recorder now.

Respondent Background
1. [If not already known for each person participating in the interview]
a. What is your official title and role?
b. What are your main responsibilities related to implementation of WIOA?

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection
displays a valid OMB control number. Public reporting burden for this collection of information is estimated to average 1 hour and 15 minutes
per response, including time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and
completing and reviewing the collection of information. The obligation to respond to this collection is voluntary. Send comments regarding the
burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to
[email protected] and reference the 0MB Control Number 1290-XXXX. Comments can also be mailed to: U.S. Department of
Labor, Chief Evaluation Office, 200 Constitution Ave., NW, S-2312, Washington, DC 20210. Note: Please do not return the completed interview
guide to the email or mailing address.

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 3.8

A. Governance and Planning
1. Local Plan Development
1.

Have there been changes in federal or state funding for the workforce system that
have affected your local plan or how the plan will be implemented?
a. If there have been significant cuts to funding, how are you adjusting the system in
response to these cuts?
b. What will be the impacts of these cuts on infrastructure, programming, and
service delivery?

2. Regional Designations and Regional Plans
1.

Has being part of a WIOA-designated region helped you to better engage with
regional employers? With regional economic development entities?

3. Local Workforce Development Board Role
1.

How, if at all, have the role and approaches of the board changed under WIOA?
a. Is it more strategic, more innovative, more focused on continuous improvement,
more focused on improving service integration?
b. Why has the role of the board changed?
i. New board mandates under WIOA?
ii. Required changes in membership?
iii. Some other reason?
a)

Employer Engagement

1.

What policies and strategies has your local board adopted to support employer
engagement? How successful have these been?

2.

How has employer representation on the Board changed, and how have these changes
affected employer engagement in the workforce system?

3.

How have employer engagement efforts and activities changed the workforce system
in your local area?
b)

Sector Strategies

1.

What policies and strategies has your local board adopted to increase the use of sector
strategies in the workforce system? Involving new partners, expanding services and
training options, targeting new populations, etc.?

2.

Has the focus on developing and expanding sector strategies been at the state, local,
or regional levels?

3.

What funding sources are being used to develop and support your sector strategy
programs and initiatives (e.g., local formula funds, WIOA state set-aside funds, Title
II or IV funding)?

3.80

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 3.8

B. AJC System
1. AJC Certification
1.

Did your AJC (as well as any others you manage) complete the certification process
successfully?
a. If not, what certification criteria were not met and how are you remedying these
issues?

2.

Have you faced any challenges with the certification process? If so, what were they
and how did you address them?

3.

What worked well with the certification process?

4.

How useful do you think the certification process is in ensuring AJC effectiveness,
accessibility, and continuous improvement? How could it be improved?

2. Infrastructure Funding and MOUs
1.

How does your MOU (excluding resource sharing components) differ from the last
one you had under WIA?

2.

What did the LWDB include in your One-Stop operating budget?
a. Other than IFAs, what was included?
b. Why did you include those components? [Make sure to get a copy or copies.]

3.

What process did the LWDB follow to negotiate the required MOU and One-Stop
operating budget(s)?
a. Did you negotiate umbrella budgets or separate budgets for each AJC?

4.

What methodologies did the LWDB choose to use for determining their various types
of costs?
a. Why did the LWDB choose those methodologies?
b. What are the advantages and disadvantages of these methodologies?

5.

How did the LWDB determine proportionate use and relative benefit for each
partner?
a. Why did the LWDB choose these methods?
b. How well do these methods work?
c. Are there differences in their appropriateness for on-site vs. off-site partners?

6.

What non-cash contributions do partners make?
a. Why were they not able to contribute cash?
b. How did the LWDB determine how to value non-cash contributions? How well
did that work?

7.

How has the WIOA RSA requirement affected partnerships (either positively or
negatively)?

3.81

WIOA IMPLEMENTATION EVALUATION

8.

OMB SUPPORTING STATEMENT: PROTOCOL 3.8

How often is the LWDB reviewing your agreement (for issues like value of non-cash
contributions, proportionate use and relative benefit)?
a. Are there any challenges related to this review process?

9.

What challenges did you face in the process of developing your MOU and One-Stop
operating budgets?

10.

Has your state has taken any steps to facilitate negotiation of One-Stop operating
budgets (e.g. meetings at the state level, with state partners, to facilitate their
participation)?

11.

Did you achieve consensus on your IFA?
a. [If so]: Why do you think your area succeeded?
b. [If not]: Which required partner(s) did not agree? What were the sticking points?
i. Where are you in the process of working with the state on the steps of the
SFM?
ii. How effective has the state been in carrying out the SFM? What could be
improved?

12.

How helpful was the guidance provided by your state and USDOL on resource
sharing and MOUs? What could have been more helpful?

13.

What TA have you participated in on MOUs and resource sharing?
a. How helpful was this TA?
b. What could have been more helpful?

14.

What additional guidance or technical assistance do you need from your state or
USDOL on MOUs and resource sharing?

3. One-Stop Operator
1.

Operators have different responsibilities in different areas; for your local area, what is
the operator’s role as it relates to:
a. Partner meetings?
b. Cross-training sessions?
c. Coordinating AJC operations & policies?
d. Managing AJC facilities?
e. Implementation of MOUs/RSAs?

2.

Does the operator have any other significant roles in your local area?

3.

How and why, if at all, has the operator’s role changed with the implementation of
WIOA?

4.

What is the role of the AJC manager?

3.82

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 3.8

5.

How and why has the AJC manager’s role changed, if at all, with the implementation
of WIOA?

6.

How, if at all, do you think these changes in what entity serves as the operator will
affect AJC operations?

4. Collocation at AJCs
1.

What partners are collocated at this AJC?

2.

If any required partners that are active in the local area are not collocated at
comprehensive AJC(s), are they providing access to services via training an onsite
staff member or via a “direct linkage”?

3.

Why did they opt for that approach and how is it being operationalized?
a. What challenges and benefits have you experienced with each approach?

4.

Has partner collocation at AJCs changed since WIOA implementation? If so, how
and why?

5.

What guidance have you received from the state on collocation of partners at AJCs
under WIOA?
a. Did that guidance change from what was provided under WIA? If so how? [make
sure to get copies of this guidance]
b. Is any additional guidance needed?

C. Overall AJC Streamlining & Accessible Services
1. Accessibility
1.

How integrated and streamlined is service delivery in the local area?

2.

Please describe any new state or local policies and actions that promote a more
seamless, accessible and customer-focused workforce service delivery network?

3.

Has the state or LWDB encouraged or required any changes to your AJC customer
intake process? What changes did this involve?

4.

To what extent has the state or LWDB emphasized customer or human-centered
design as a key method for making your state’s One-Stop career center system more
customer-focused?
a. What problems or goals do you plan to address through this approach?
b. What challenges and benefits have you encountered in operationalizing it?

5.

Please describe any state or local efforts to improve the accessibility of One-Stops for
individuals with disabilities (e.g. policies, additional funding, or training).

2. Referral Process
1.

How frequently are referrals are made between partner programs?

3.83

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 3.8

a. Which partners are most likely to initiate referrals?
b. Which partners are customers most frequently referred to?
2.

How are referrals made to each agency?
a. Do staff provide contact information for the program to a customer?
b. Do staff connect directly with the program they are referring a customer to?
c. Do staff submit a referral via shared MIS?
d. Or, is there some other referral process?

3.

Are referrals tracked? Is information shared about referred customers with the
program the customer is being referred to?

4.

What do you think is the most successful aspect of your referral process?

5.

Is there anything about the referral process you would like to improve?

3. Co-enrollment
1.

About how frequently are local customers are co-enrolled in partner programs?
Which programs and for what reasons?

1.

How, if at all, do co-enrollment processes differ from referral processes?

2.

How, if at all, is information about co-enrolled customers shared with partner
programs?

3.

What is the most successful aspect(s) of local co-enrollment processes?
a. What requirements particularly facilitate co-enrollment?
b. Are there factors that impede co-enrollment?

4.

Is there anything about local co-enrollment processes you would like to improve?

4. AJC Partner Meetings
1.

Do you have regular AJC partner meetings? (If no, skip to cross-training)

2.

What partners attend regularly?

3.

What level of staff attend these meetings? Program managers? Line staff? Both?

4.

Who organizes and facilitates these meetings?

5.

How often do they occur? For how long?

6.

What topics are covered in these meetings? [Try to get a copy of the agenda for the
most recent 1-2 mtgs.]

5. Cross-training
1.

Have you organized or participated in any cross-training sessions across different
programs? (If not, skip to 9)

2.

How often have these sessions occurred? At what locations?

3.84

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 3.8

3.

Are these in-person or virtual?

4.

How long did/do they last?

5.

Who attends regularly? Are these local or state-wide trainings?

6.

Who organizes/facilitates?

7.

What is covered in these sessions? [try to get an agenda for the last 1-2 cross-training
sessions.]

8.

How helpful have these sessions been?
a. Is there any way they could have been improved?

9.

In what other ways—if any—do you share information on partner programs?
a. How well do these other methods work?

6. Use of shared MIS
1.

Do any core or required partners use the same management information system
(MIS)? If so, what systems? (If not, skip to #5)
a. Which partners use them?
b. For what purposes?
i. Case management?
ii. Participant tracking?
iii. Performance reporting?
iv. Fiscal reporting?

2.

Are there differences in how partners can use any of these common systems?
a. Can all partners view data? Are there specific fields that are restricted?
b. Can all partners enter data into the system? Are there specific fields that only
some partners can enter?
c. Can all partners revise data? Are there specific fields than only some partners can
modify?

3.

How do you ensure that the data being entered into these systems by different
partners is accurate?

4.

How are you protecting the security of the data being entered and stored?

5.

Has the state or LWDB encouraged, required, or led the development of a shared case
management system?

6.

How important is it for seamless services that partner programs use the same MIS?

3.85

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 3.8

D. Performance Accountability
1. ETPL Changes
1.

How and why, if at all, have these ETPL changes impacted the number of providers
available for your local area?

2. Common Measures
1.

Have you faced any challenges implementing the WIOA common performance
measures?

2.

Which challenges have you faced in measuring the effectiveness of employer
services?
a. Do you have thoughts on measures other than the ones being used that might
better capture the effectiveness of serving employers?

3.

Have you faced any challenges in implementing state-specific performance
measures?

4.

Are you developing any local measures that you and your local partners will be
reporting on?

5.

Have you faced any challenges in implementing local performance measures?

E. Services for job seekers and employers
1.

Has WIOA changed your approach to providing services to businesses? In what way?

2.

Please describe your efforts to improve access and services to more barriered
customers, including disabled customers and those with low basic skills.

3.

Have you received any guidance or TA from the state on improving services to
customers with barriers? How helpful was it?

4.

What, if any, additional guidance should USDOL or your state should provide on
serving customers with barriers?

G. Overall Questions
1. Partnerships
1.

How would you assess relationships among core partners? Required partners? Other
partners?
a. Has this changed because of WIOA?
b. If not, why do you think WIOA’s focus on enhanced partnerships has not affected
your partnerships?

2.

Are there sufficient systems in place to ensure regular and effective communication
between partners?

3.86

WIOA IMPLEMENTATION EVALUATION

3.

OMB SUPPORTING STATEMENT: PROTOCOL 3.8

What have been the key challenges and successes related to increasing collaboration
across partners?
a. Are there differences in the extent of collaboration with your core partners versus
other required partners, such as UI, SCSEP, Veterans programs, and TANF?

4.

Overall, how integrated are workforce services now?
a. Has this changed because of WIOA?
b. What changes reflect this more integrated service delivery? For example, are case
managers of different programs meeting more often to discuss co-enrolled clients?

5.

[If respondent feels workforce system is not integrated]: Why do you think WIOA’s
focus on enhanced partnerships has not affected the integration of services?

2. Systems Change
1.

Overall, do you think the workforce system here has changed as a result of WIOA?
a. If so, what are the major changes you can attribute to WIOA? E.g. employer
engagement, partnerships, services more streamlined, data sharing, others?
b. If not, do you feel systems had already begun to change prior to WIOA, or that
efforts under WIOA have not yet been effective in changing the system?

2.

Have there been any negative, unintended consequences of implementing the WIOA
regulations?
a. Are there ways in which you feel the new regulations actually make it more
difficult for you to achieve your goals?

3.

[If respondent doesn’t think WIOA has changed the system]: Do you feel systems had
already begun to change prior to WIOA, or that efforts under WIOA have not yet
been effective in changing the system?

4.

What additional changes are planned or needed to transform your workforce system
to meet the goals of WIOA?
b. What changes would you like to see to make the workforce system better overall?

3. Guidance and Technical Assistance
1. Of the TA provided by DOL national and regional offices on WIOA implementation,
which assistance has been most helpful?
a. Which TA formats and types of materials have been most useful for you?
2. Are you satisfied with the level of TA DOL has provided on WIOA implementation?
3. Have you accessed the ION site to obtain TA materials or guidance on specific issues? If
so, which issues?
4. In what areas would you like to receive additional TA?

3.87

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 3.8

Wrap-up
1. How would you describe your overall progress to date with WIOA implementation?
a. Where do you still have plans to make significant changes?
2. Beyond what we’ve discussed today, are there other areas you would like to highlight?
3. Are there any other areas of WIOA implementation our study should explore?

3.88