Att 2.4 State Title II and IV Protocol

OMB No. 1290-XXXX
Exp. Date XX/XX/2021

2.4 PROTOCOL FOR STATE TITLES II AND IV STAFF
Instructions to site visitors: Bullets below each question represent probes for important details;
depending on how well you are doing on time, try to make sure respondents touch on each of
these issues. Text in brackets [ ] should be tailored based on the state or local area, as well as
Title II or IV respondent as appropriate. For states where the community college system is highly
centralized, and may be the most appropriate respondent for some Title II related questions,
questions specific to community colleges respondents have been identified. Sections in gray are
the high priority sections for all respondents.

Introduction
Thank you for taking the time to meet with me today. My name is [NAME] and I work for
[Mathematica Policy Research/Social Policy Research Associates]. I am part of an independent
research team conducting a study of WIOA implementation on behalf of the U.S. Department of
Labor. Thank you very much for agreeing to participate in this conversation.
The purpose of this three-year study is to help DOL understand how states are implementing
changes to the core workforce programs authorized under Titles I and III of WIOA, as well as
how those changes are influenced by integration with other partners such as adult education,
vocational rehabilitation, and TANF. To help us better understand WIOA’s implementation, we
are conducting site visits to 14 states, including two local areas in each state, to learn about their
experiences. The information you share will help us understand the range of experiences that
states have had, including key successes and challenges.
We have about [duration] for our conversation. I want to let you know that all interview data will
be reported in the aggregate and your name will never be mentioned in any report that we write,
though we might use quotes from your interview to illustrate findings, without using your name.
I would also like to record our conversation so that I can listen to it later as I complete my notes,
and I will not share the recording with anyone outside of the research team. Being part of this
discussion is voluntary, and you may choose not to answer a question if you wish, or to pause the
recording at any time. Do you have any objections to being part of this interview or to my
recording our discussion?
Okay, I’m going to turn on the audio recorder now.

Respondent Background
1. [If not already known for each person participating in the interview]
a. What is your official title and role?
b. What are your main responsibilities related to implementation of WIOA?
According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection
displays a valid OMB control number. Public reporting burden for this collection of information is estimated to average 1 hour and 20 minutes
per response, including time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and
completing and reviewing the collection of information. The obligation to respond to this collection is voluntary. Send comments regarding the
burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to
[email protected] and reference the 0MB Control Number 1290-XXXX. Comments can also be mailed to: U.S. Department of
Labor, Chief Evaluation Office, 200 Constitution Ave., NW, S-2312, Washington, DC 20210. Note: Please do not return the completed interview
guide to the email or mailing address.

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 2.4

2. Briefly describe your agency and department/division.
a. What is your department/division’s role in the agency? In WIOA implementation?
b. Do any of your agency’s other divisions/department play a significant role in
WIOA implementation? Which ones? What role do they play?
c. Do you have local or regional staff? How does your agency supervise/monitor the
work of those staff and the implementation of WIOA at the local and regional
levels?
[Title II Respondents]
a. Describe the structure of AEFLA service delivery in your state
i. What entities typically serve as local AEFLA grant recipients? How much
does this vary?
ii. Are there typically single or multiple grantees per local workforce area?
How much does this vary?
b. When did you last competitively award AEFLA grants? For how many years were
the most recent grants awarded?
c. What other adult education programs or funding operate within the state? How
integrated are AEFLA-funded programs with those other adult education efforts?
d. Have there been any major changes to the structure of AEFLA administration or
service delivery in your state since WIOA? If so, what changes and why did they
come about?
[Title IV respondents]
a. Have there been any major changes in administrative structure or responsibilities
for VR since WIOA? If so, what changes and why did they come about?
b. How well do the VR regions/service areas match up with WIOA regions? With
the geographic areas that define the local workforce areas in your state?
i. If VR and WIOA regions are different, is that a challenge? Describe.
ii. If VR service areas and local workforce areas are different, is that a
challenge? Describe.

A. Governance and Planning
1. State Plan Development
[Title II respondents]
1. How and to what extent were AEFLA/adult education staff involved in developing the
WIOA state plan? What other agencies were involved?
2. Did state partner staff meet regularly regarding the WIOA state plan?
a. From which agencies and programs?

2.46

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 2.4

b. How often did they meet?
i. Are they still meeting to discuss WIOA implementation? If so, how often?
ii. What do they now discuss?
3. Did the WIOA state plan process differ from previous (WIA) state plan development
efforts?
a. Were AEFLA/adult education staff more or less involved?
4. What, if any, significant challenges were encountered in the planning process? What
worked well?
5. Was your WIOA state planning process helpful for:
a. Developing and following through on your state’s workforce strategy?
b. Establishing, developing or improving partnerships with WIOA partners? If so,
which ones?
6. In what ways did the state plan revision process differ from the process used to develop
the initial plan?
7. What could be done to make the plan (or the planning process) more useful/strategic?

[Title IV respondents]
1. How and to what extent were VR staff involved in developing the WIOA state plan?
What other agencies were involved?
2. Did state partner staff meet regularly regarding the WIOA state plan?
a. From which agencies and programs?
b. How often did they meet?
i. Are they still meeting to discuss WIOA implementation? If so, how often?
ii. What do they now discuss?
3. Did the WIOA state plan process differ from previous (WIA) state plan development
efforts?
a. Were VR staff more or less involved?
4. What, if any, significant challenges were encountered in the planning process? What
worked well?
5. Was your WIOA state planning process helpful for:
a. Developing and following through on your state’s workforce strategy?
b. Establishing, developing or improving partnerships with WIOA partners? If so,
which ones?
6. In what ways did the state plan revision process differ from the process used to develop
the initial plan?
7. What could be done to make the plan (or the planning process) more useful/strategic?

2.47

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 2.4

2. State Workforce Development Board Role and Operations
[Title II respondents]
1. Overall, has the role or functioning of the state board changed significantly under WIOA?
If so, how and why?
a. Is the board more strategic? More streamlined? More innovative? More engaged
in the state plan and its development?
b. Have changes to the board’s role affected WIOA implementation? If so, how?
2. Has the board’s membership changed under WIOA?
a. Has AEFLA’s representation on the board and/or board committees been
enhanced? (e.g. Is the director of adult education now on the board? Are more
adult education staff on board committees)?
b. If the board’s membership has changed, what have new members brought to the
board?
3. If the board’s role or membership has changed since WIOA, have these changes affected
WIOA implementation? AEFLA? Describe.
[Community colleges only]
4. Are you a member of the state workforce development board or any committees or
workgroups affiliated with the state board? (If no, skip to 8).
a. If you are on committees or workgroups, which ones?
b. If you are on the state board or a committee, what has been your involvement in
any of the following:
i. Overall WIOA implementation?
ii. Career pathways and sector strategy policy or system development?
iii. Developing policies and procedures governing eligible training providers?
iv. Other workforce policy development?
5. When did you first become a member of the board/the committee? Was it prior to
WIOA?
a. Has the role or functioning of the state board or its committees changed
significantly under WIOA? If so, how and why?
i. Is the board more strategic? More streamlined? More innovative?
ii. How have those changes affected WIOA implementation?
6. Has your participation as a representative of the state’s community college system
affected the state board’s priorities or operations? The state’s workforce system as a
whole? If so, describe.
a. Have you helped the board to be more focused on the training and educational
needs of workforce?

2.48

WIOA IMPLEMENTATION EVALUATION

7.

OMB SUPPORTING STATEMENT: PROTOCOL 2.4

[If no to 8] Is there a representative from the state’s community college system on the
state board? If so, who is that?
i. [If there is someone else representing community colleges on the state
board] What is your connection to that representative? Is it helpful to
have a community college representative on the state board? If so, why?

8. [If no to 8] Have you worked with state board staff or the community college
representative on the state board to develop policies related to:
a. WIOA implementation?
b. Sector strategies, career pathways, or the ETPL?
9. From your perspective as a community college representative, what should be the state
board’s primary focus related to WIOA implementation?
a. Employer engagement?
b. Career pathways?
c. Sector strategies?
d. Apprenticeships?
10. How much of a focus does the board have on the role of the community college system
within the workforce system?

[Title IV respondents]
1. Has the role or functioning of the state board changed significantly under WIOA? If so,
how and why?
a. Is the board more strategic? More streamlined? More innovative? More engaged
in the state plan and its development?
2. Has the board’s membership changed under WIOA?
a. Has VR’s representation on the board and/or board committees been enhanced?
(e.g. is the director of VR now on the board? Are more VR staff on board
committees)?
b. If the board’s membership has changed, what have new members brought to the
board?
3. If the board’s role or membership has changed since WIOA, how have these changes
affected WIOA implementation? The VR program?

3. Local Area Designations and Local Plan Development
[Title II respondents]
1. What guidance did the state provide to local boards on the requirements for developing
their local plans? How, if at all, were AEFLA/adult education staff involved in the
development of that guidance?
2. How involved were AEFLA grantees in the development of local plans?

2.49

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 2.4

a. Did this differ from their involvement in local plans under WIA?
3. Describe the process you used for meeting WIOA’s requirement to have local boards
review AEFLA grantee funding applications and AEFLA grantees review local board
plans.
a. How useful were these reviews? In what ways were they useful?
b. Was it challenging to make this process work? Why?
4. How did the resulting local plans typically differ from the last WIA local plans?
5. Do you think the WIOA local plans are useful? How have they affected local workforce
service delivery?
a. Have they led to more integrated service delivery or enhanced sector planning
efforts across local areas within a region?
6. What challenges did AEFLA grantees typically experience regarding the development of
local plans? What worked well?
7. What could be done to make local plans (or the planning process) more useful/strategic?

[Title IV respondents]
1. What guidance did the state provide to local boards on the requirements for developing
their local plans? Were VR staff involved in the development of that guidance? Describe.
2. How involved were VR staff in the development of local plans?
c. Did this differ from their involvement in local plans under WIA?
3. How did the resulting local plans typically differ from the last WIA local plans?
4. Do you think the WIOA local plans are useful? How have they affected local workforce
service delivery?
a. Have they led to more integrated service delivery?
b. Have they led to enhanced sector planning efforts across local areas within a
region?
5. What challenges did VR staff typically experience regarding the development of local
plans? What worked well?
6. What could be done to make local plans (or the planning process) more useful/strategic?

4. Regional Designations and Regional Plans
[Title II respondents]
1. Were AEFLA/adult education staff involved in the development of the WIOA regions? If
yes, how?
a. Do you think the regional designations are appropriate? Why or why not?

2.50

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 2.4

b. How well do the WIOA regions match up with AEFLA/adult education service
delivery areas/regions?
i. If the AEFLA/adult education and WIOA regions are different, is that a
challenge? Describe.
2. What guidance did the state provide to local boards on the requirements for developing
regional plans? Were AEFLA/adult education staff involved in the development of that
guidance? Describe.
3. How involved were local AEFLA grantees in the development of regional plans?
4. What challenges did AEFLA grantees experience in the development of regional WIOA
plans? What worked well?
5. Do you think the WIOA regions and regional plans are useful? Have they affected local
workforce service delivery? Why or why not?
a. If they have affected service delivery, describe.
i. Have they led to more integrated service delivery or enhanced sector
planning efforts across local areas within a region?
6. How involved were/will AEFLA grantees be in the modifications of regional plans?
a. How did the regional plan revision process differ from the process used to
develop the initial regional plans?
7. What could be done to make regional plans (or the planning process) more
useful/strategic?

[Title IV respondents]
1. Were VR staff involved in the development of the WIOA regions? If yes, how?
a. Do you think the regional designations are appropriate? Why or why not?
b. How well do the WIOA regions match up with VR regions?
i. If VR and WIOA regions are different, is that a challenge? Describe.
2. What guidance did the state provide to local boards on the requirements for developing
regional plans? Were VR staff involved in the development of that guidance? Describe.
3. How involved were VR staff in the development of regional plans?
4. What challenges did VR staff experience in the development of regional WIOA plans?
What worked well?
5. Do you think the WIOA regions and regional plans are useful? How have they affected
local workforce service delivery?
a. Have they led to more integrated service delivery?
b. Have they led to enhanced sector planning efforts across local areas within a
region?
6. How involved were/will VR staff be in the modifications of regional plans?

2.51

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 2.4

a. How did the regional plan revision process differ from the process used to
develop the initial regional plans?
7. What could be done to make regional plans (or the planning process) more
useful/strategic?

5. Local Workforce Development Board Role and Focus
1. Has the role or functioning of local boards changed significantly under WIOA? If so,
how and why?
a. Are boards generally more strategic? More streamlined? More innovative?
Provide examples.
2. Has the typical membership of local boards changed significantly under WIOA?
a. Has AEFLA/VR’s representation on local boards changed? Describe.
i.

If AEFLA/VR representation on local boards has changed, how has this affected
the operation/priorities of boards?
1. Are boards focusing more on serving people with disabilities?

2. [Title IV only] Do most local boards in the state have a youth committee?
a. Are VR staff typically members of these youth committees? Why or why not?

B. AJC System
1. AJC Certification
[Title II respondents]
1. Were AEFLA/adult education staff involved in developing the state’s policy regarding
AJC certification? How?
2. What challenges, if any, have AEFLA/adult education staff faced related to the
certification process?
3. Overall, how well is the state’s certification process working? How helpful is it for
improving the state’s AJCs? In what ways could it be improved?
a. Has it improved access to AJCs by customers with low basic skills?

[Title IV respondents]
1. How were VR staff involved in developing the state’s policy regarding AJC certification?
How?
a. How specifically were they involved in developing criteria on AJC accessibility
for consumers with disabilities?
i. How is VR’s role in developing accessibility criteria different from under
WIA?

2.52

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 2.4

b. How is VR’s role in the AJC certification process generally different from under
WIA?
2. What challenges, if any, have VR staff faced related to the certification process?
3. Overall, how well is the state’s certification process working? How helpful is it for
improving the state’s AJCs? In what ways could it be improved?
a. Has it improved access to AJCs by disabled customers?

2. Infrastructure funding and MOUs
[Title II respondents]
1. What were the key aspects of the state guidance on MOUs, cost-sharing and IFAs? How
did it differ from the state’s MOU and cost-sharing guidance provided under WIA?
a. Were AEFLA/adult education staff involved in developing this guidance? If so,
how?
b. How clear and effective was this guidance? Any challenges? What worked well?
2. Did you provide guidance to AEFLA/adult education grantees to prepare them for their
role in negotiations? If yes, what were the key points of that guidance? [Request a copy.]
3. What was the experience of AEFLA/adult education grantees around the state in
negotiating MOUs, one-stop operating budgets, and IFAs?
a. How long did the process typically take?
b. What challenges arose in the negotiation process?
c. What were the major issues of concern? How were they resolved?
4. Were there any state-level activities that took place to facilitate the
development/agreement on of One-Stop operating budgets/IFAs (e.g. through state-level
partner meetings or MOUs)? Describe.
5. Did all local areas achieve consensus on their IFAs?
a. If yes, why do you think areas succeeded?
b. If not, how many areas failed? Which required partner(s) did not agree? What
were their objections?
6. [If the state implemented the State Funding Mechanism (SFM)] What was the SFM
process like for the AEFLA/adult education program in the affected local area(s)? What
was it like for your staff at the state-level?
7. Overall, was it relatively easy or difficult for local boards, AEFLA grantees and other
local partners to implement WIOA’s MOU, IFA, and cost-sharing requirements? What
challenges/promising practices did they encounter?
8. How have WIOA’s MOU, cost-sharing and IFA requirements affected AJCs’
partnerships with AEFLA?

2.53

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 2.4

[Title IV respondents]
1. What were the key aspects of the state guidance on MOUs, cost-sharing and IFAs? How
did it differ from the state’s MOU and cost-sharing guidance provided under WIA?
a. Were VR staff involved in developing this guidance? If so, how?
b. How clear and effective was this guidance? What challenges did you encounter in
using the guidance? What worked well?
c. What other types of input did the state VR agency have in planning or decisionmaking around MOU and cost-sharing/IFAs?
2. What was the experience of VR managers around the state in negotiating MOUs, onestop operating budgets, and IFAs?
a. How were VR managers prepared for their role in negotiations? Did they receive
training? Were they provided with negotiating instructions?
b. How were the MOUs, budgets and IFAs developed?
i. How long did the process typically take?
ii. Were the negotiations generally smooth or contentious?
iii. What were the major issues of concern? How were they resolved?
3. Typically, how often are IFAs and one-stop budgets reviewed? Are there any challenges
related to this review process? What has worked well?
4. Were there any state-level activities that took place to facilitate the
development/agreement on of One-Stop operating budgets/IFAs (e.g. through state-level
partner meetings or MOUs)? Describe.
5. Did all of the local areas achieve consensus on their IFAs?
a. If yes, why do you think areas succeeded?
b. If not, how many areas failed? Which required partner(s) did not agree? What
were their objections?
6. [If the state implemented the State Funding Mechanism (SFM)] What was the SFM
process like for the VR program in the affected local area(s). What was it like for your
staff at the state level?
7. Overall, how difficult was it for locals to implement WIOA’s MOU and cost-sharing
requirements? What challenges did they encounter? What worked well?
8. Overall, how have WIOA’s cost-sharing and MOU requirements affected AJCs?
Partnerships?
a. Are you aware of any changes to VR or other partner participation in AJCs
because of WIOA’s resource sharing requirements? If so, describe.

2.54

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 2.4

3. Collocation at AJCs
[Title II respondents]
1. How commonly are AEFLA/adult education grantee staff collocated at comprehensive
AJCs in the state?
a. What do you think are the primary reasons for this level of collocation?
i. State policies or requirements, lack of space at AJCs for adult education
classes, or some other reason?
b. Do you know of any plans by AEFLA/adult education grantees to collocate their
staff at the comprehensives where none are currently collocated? Explain.
2. Has collocation of AEFLA/adult education grantee staff at AJCs changed since WIOA?
If so, how? Why?
3. At the comprehensive AJCs where no AEFLA/adult education staff are collocated, how
are grantees meeting WIOA’s requirement for connection to AEFLA?
a. Are grantees training collocated partner staff to provide information to customers
about AEFLA/adult education services or providing a “direct linkage” via
technology to provide “meaningful information and services”?
b. Has it been relatively easy or difficult for grantees to meet this requirement?

[Title IV respondents]
1. How commonly are VR staff collocated at comprehensive AJCs in the state?
a. What do you think are the primary reasons for this level of collocation?
i. Is it required by state policies?
ii. Is it due to AJCs’ incentive to increase the accessibility of AJCs for
disabled consumers?
b. Are there any plans to collocate VR staff at the comprehensive AJCs where none
are currently collocated? Explain.
2. Has collocation of VR staff members at AJCs changed since WIOA? If so, how and why?
3. At the comprehensive AJCs where no VR staff are collocated, how are you meeting
WIOA’s requirement that you either train collocated partner staff to provide information
to customers about VR services or provide a “direct linkage” via technology to provide
“meaningful information and services”?
a. Has it been relatively easy or difficult to meet this requirement?

2.55

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 2.4

C. Overall AJC Streamlining & Accessible Services
1. Accessibility
1. Please describe any state policies and actions that promote a more seamless, accessible,
and customer-focused workforce service delivery network.
a. How common are integrated intake processes for AEFLA/VR and other AJC
partners?
i. Which partners participate?
ii. Did developing such processes involve state level activities, either by
AEFLA/VR or together with other partners (e.g., AEFLA/VR program
policy changes; meetings between state-level partners)?
b.

How common are integrated staffing arrangements like functional alignment at
AJCs involving AEFLA /VR program or grantee staffs?
i. Which partners participate?
ii. Did developing such arrangements involve state level activities, either by
AEFLA/VR or together with other partners such as Title IB managers
(e.g., AEFLA/VR staff supervision policy changes; meetings between
state-level partners)?

2. Have AEFLA/VR staff or grantees been involved in any human-centered design
processes at the local level? If so, how useful were these efforts in improving services?
3. [Title IV only] Please describe any state-wide efforts to improve the accessibility of AJCs
for individuals with disabilities (e.g. policies, additional funding, or training).
a. How involved have VR staff been in these efforts?

2. Co-enrollment
1. Do any new WIOA requirements particularly facilitate or impede co-enrollment for
AEFLA/adult education/VR customers?
a. Between what programs does co-enrollment occur? Which are the most common,
and why?
2. Is the state encouraging co-enrollment and if so, how? Between the AEFLA/adult
education/VR programs and which other programs?

3. Use of shared MIS
1. What management information system(s) (MIS) do AEFLA/adult education/VR
programs use?
a. How long have each of these systems been in place?
b. For what purpose is each system used?
i. Case management?

2.56

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 2.4

ii. Performance reporting?
iii. Fiscal reporting?
c. Are any other workforce partners using/accessing these systems? Which ones?
i. [If yes] How are they using/accessing the system(s)
1. Do they enter data? Just for their own programs or can they enter
data in shared fields, such as contact information?
2. Can they view other partners’ data? Which ones?
ii. [If no] Why are no other partners using these systems? Are there any plans
to have partners begin using them?
1. Does NOT using the same MIS impede WIOA’s goal of
integrated, streamlined services?
d. Are there plans to add other partners to these systems? If so, which partners?
i. If any core program partners are not using these systems, why not?
2. If not all core partners use the same MIS, is data shared (or planned to be shared) in some
other way? If so:
a. How are data shared across systems (e.g., via a data warehouse, etc.)?
b. What partners use this data sharing process?
c. For what purpose are these data shared?
3. How does data sharing help to achieve WIOA’s goal of integrated, streamlined services?
a. Is using the same MIS the best way to achieve these goals or is sharing data via
some other process just as effective?

D. Performance Accountability
1. Performance Measurement
[Title II respondents]
1. Was it relatively easy or difficult to implement the WIOA primary indicators of
performance for the AEFLA program?
a. Have you had any challenges with specific indicators related to the AEFLA
program? Which ones?
i. Measurable Skill Gains?
1. Challenges related to individuals who begin participating late in
the program year?
ii. Credential Attainment?
iii. The Employment Rate and Median Earnings indicators?

2.57

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 2.4

1. Challenges related to measuring employment and earnings two and
four quarters after exit?
2. Are the WIOA performance indicators appropriate for AEFLA participants? Why or why
not?
a. Are you concerned about the AEFLA program meeting negotiated targets for the
WIOA indicators? Why or why not?
3. Which two approaches to measuring the effectiveness in serving employers did your state
choose to implement for the pilot program?
a. What approaches other than the three proposed by DOL and the Department of
Education (ED) might better capture effectiveness in serving employers?
4. How important are these business-related indicators to improving business services?
a. Do you think these indicators will result in better integration of core program
business services?
b. Will these indicators result in more robust business services? Why or why not?

[Title IV respondents]
1. Was it relatively easy or difficult to implement the WIOA primary indicators of
performance for the VR program?
a. Have you had any challenges with specific indicators related to the VR program?
Which ones?
i. Measurable Skill Gains?
1. Challenges related to individuals who begin participating late in
the program year?
2. Challenges getting progress reports from employers or measuring
achievement of benchmarks for work-based training?
ii. Credential Attainment?
1. Challenges related to determining what credentials count under this
measure vs. what counted under WIA?
2. Challenges related to collecting data on credential attainment up to
1 year after exit?
iii. The Employment Rate and Median Earnings indicators?
1. Challenges related to measuring employment and earnings two and
four quarters after exit?
2. Are the WIOA performance indicators appropriate for VR consumers? Why or why not?
a. Are you concerned about the VR program meeting state negotiated targets for the
WIOA indicators? Why or why not?
3. Which two approaches to measuring the effectiveness in serving employers did your state
choose to implement for the pilot program?

2.58

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 2.4

a. What approaches other than the three proposed by DOL and the Department of
Education (ED) might better capture effectiveness in serving employers?
4. How important are these business-related indicators to improving business services?
a. Do you think these indicators will result in better integration of core program
business services?
b. Will these indicators result in more robust business services? Why or why not?

2. Reporting, Evaluation, and Labor Market Information
1. What progress has your state made related to meeting the requirements for WIOA’s new
record layout (PIRL)?
a. How challenging has it been to report on all the required fields required for the
PIRL? Do you think all of these fields are necessary? Useful?
2. How challenging has it been for [the AEFLA program and local grantees/VR] to meet
WIOA’s required reporting changes and timelines? Why?
a. What factors have made this process challenging?
b. How have you addressed these challenges?
3. To what extent can your state track an individual’s participation across more than one
partner via a common identifier, especially across the core partners?
a. Which partners use this identifier?
b. What efforts are currently underway to expand capacity to be able to do this—
especially across the core partner programs?
4. Have you faced any challenges in accessing data required for reporting?
a. If so, which types of data? UI wage data?
b. How have you resolved or attempted to resolve these challenges?

3. Eligible Training Provider Changes
[Title II respondents – Community Colleges only]
1. How involved are the state’s community colleges in providing training for Title I
programs (Adult, Dislocated Worker, and Youth)?
a. What approximate percentage of college programs are currently on the state’s
Eligible Training Provider List (ETPL)?
b. Has this percentage increased or decreased thus far under WIOA? Explain.
2. Have you been involved in revising the state’s the Eligible Training Provider policies and
procedures as required by WIOA? If so, how?
3. What challenges, if any, has the state faced in implementing WIOA’s changes to the ETP
system/list?
a. Is it difficult for community colleges to add a program to the list?

2.59

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 2.4

b. How easy/difficult is it for colleges to collect/submit required performance data to
keep their programs eligible? How is the state assisting colleges with doing so?
4. Have there been any challenges specifically related to including apprenticeships on the
ETP list? Describe.
5. What recommendations would you make regarding improvements to the state/local ETP
requirements?

E. Services for Job Seekers and Employers
1. Employer Services
[Title II respondents]
1. How do AEFLA grantees in your state typically work with employers? What, if any,
employer services do they typically provide?
a. Has this changed under WIOA? What prompted these changes? (e.g., because of
WIOA’s primary indicators of performance?)
2. How much do AEFLA/adult education grantees coordinate with other local core and
required partners when they work with employers? If coordination is limited, why?
a. If grantees do coordinate with partners, which ones are the most typical?
3. Is there a shared customer relationship management (CRM) database used by AEFLA
grantees for capturing data on outreach to employers? Which partners use this system?
How helpful is this system for coordination?
4. Has coordination around working with employers improved under WIOA? Why?
a. Because of the unified WIOA state plan requirements?
b. Because of the addition of workforce preparation activities as an AEFLA activity?
5. What challenges/promising practices have you encountered regarding coordination
around working with employers?

[Title IV respondents]
1. What is VR’s approach to serving businesses? What services are typically provided?
a. Has this approach or the typical business services provided changed under
WIOA? Why?
i. How has your approach become more streamlined, effective or innovative
under WIOA?
2. How integrated are business services among VR staff and staff from other core and
required partners in the state? If services are not well-integrated, why not?
a. With which core or required partners do VR staff typically work with

2.60

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 2.4

3. Is there a shared customer relationship management (CRM) database for capturing data
on business services? Which partners use this system? How helpful is this system for
partner coordination?
4. Has business services integration improved under WIOA? Why?
a. Has VR’s greater emphasis on business services under WIOA helped to improve
integration?
b. Has the integrated state planning process improved business services
coordination?
5. What challenges/promising practices have you encountered regarding
integrated/streamlined business services?

2. Sector Strategies
[Title II respondents]
1. Does the state have any sector focused initiatives/efforts? If so, describe.
a. How are the state’s community colleges involved in these initiatives? What other
partners, particularly core partner programs, are involved? What are their roles?
b. Are these primarily state, regional, or local-level efforts?
c. How are these initiatives being funded (state funding/local funding)? Is any
WIOA funding being used?
d. Who are the main targets of these initiatives (adult versus youth; individuals with
disabilities, ex-offenders, individuals with low basic skills)?
a. What are the main targeted industries/occupations?
i. How has the state used LMI in the selection of these industries? To
develop career pathways training programs?
e. How are employers involved in these efforts?
2. Did the state’s sector efforts precede WIOA?
a. If yes, has WIOA’s implementation affected these efforts? If so, how?
3. What challenges has the state encountered in its efforts to develop and/or sustain sector
initiatives? What has worked well?
4. Is there a need for additional TA or guidance from DOL or the state on sector initiatives?
6. How do AEFLA grantees in your state typically work with employers? What, if any,
employer services do they typically provide?

[Title IV respondents]
1. To what extent have you emphasized sector strategies for VR consumers?
a. What policies and strategies have you adopted to increase the use of sector
strategies for VR consumers?

2.61

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 2.4

b. Did the VR program’s efforts regarding sector strategies precede WIOA?
Describe.
ii. If so, what has changed under WIOA? Have you:
1. Involved new partners and collaborations?
2. Expanded services/training features?
3. Targeted different populations, such as youth, individuals with
disabilities, limited English and/or basic skills?
iii. Why have sector strategies under WIOA changed in these ways?
2. Do you coordinate with other core or required partners to develop or implement sector
strategies? Why or why not?
a. If you do coordinate with other partners, describe.

3. Work-based and job-driven training
[Title II respondents]
1. To what extent have you emphasized work-based training activities like internships,
subsidized work experience, or transitional employment for AEFLA participants that are
receiving integrated education and training services?
a. How typical is it for AEFLA participants to participate in such activities?
2. What are the most common types of such activities used by AEFLA participants?
3. Have you adopted policies and strategies to increase the use of work-based training
activities for AEFLA participants? Describe.
4. Do you coordinate with other core or required partners to develop or implement workbased training activities for AEFLA participants? Describe.
5. Did the AEFLA program’s efforts regarding work-based training precede WIOA?
Describe.
b. If so, what has changed under WIOA? Why?
i. Is there more coordination with core or required partners?

[Title IV respondents]
1. To what extent have you emphasized work-based training such as OJT, internships,
supported work experience, or transitional employment for VR consumers? How typical
is it for VR consumers to receive such training?
2. What are the most common types of such training used by VR consumers?
3. Have you adopted policies and strategies to increase the use of work-based training for
VR consumers? Describe.

2.62

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 2.4

4. Do you coordinate with other core or required partners to develop or implement workbased learning opportunities for VR consumers? Describe.
5. Did the VR program’s efforts regarding work-based training precede WIOA? Describe.
a. If so, what has changed under WIOA? Why?
ii. Is there more coordination with core or required partners

4. Registered Apprenticeship
[Title II respondents]
1. To what extent have you emphasized registered apprenticeship for AEFLA participants?
How typical is it for AEFLA participants to be apprentices?
2. Have you adopted policies and strategies to increase the registered apprenticeship for
AEFLA participants? Describe.
3. Do you coordinate with other core or required partners to develop or implement
registered apprenticeships for AEFLA participants? Describe.
4. Did the AEFLA program’s efforts regarding registered apprenticeship precede WIOA?
Describe.
a. If so, what has changed under WIOA? Why?
i. Is there more coordination with core or required partners?
b. Are there regulations around registered apprenticeship that could or should be
relaxed to improve their use in your state/region?
[Community colleges only]
5. What is the community college system’s involvement in the state’s efforts to increase the
number of apprentices? [If none, skip to the next section.]
a. Are many of the state’s community colleges involved in registered
apprenticeships?
i. If yes, what are colleges’ typical roles (classroom training provider,
sponsor, etc.)?
ii. If not, why not?
6. What challenges has the state faced in increasing the number of registered
apprenticeships?
a. In what ways is the state trying to deal with these challenges??
b. Are there any regulations related to apprenticeships that should be revised or
relaxed to improve the use of registered apprenticeship?
7. Has WIOA helped the state to increase the number of registered apprentices? If so why?

2.63

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 2.4

a. Has the state’s community college system become more involved in providing
apprenticeships as a result of WIOA? If so, why?
8. Is there a need for additional guidance or TA from DOL related to increasing the number
of apprentices?

[Title IV respondents]
1. To what extent have you emphasized registered apprenticeship for VR consumers? How
typical is it for VR consumers to be apprentices?
2. Have you adopted policies and strategies to increase the registered apprenticeship for VR
consumers? Describe.
3. Do you coordinate with other core or required partners to develop or implement
registered apprenticeships for VR consumers? Describe.
4. Did the VR program’s efforts regarding registered apprenticeship precede WIOA?
Describe.
a. If so, what has changed under WIOA? Why?
i. Is there more coordination with core or required partners?
b. Are there regulations around registered apprenticeship that could or should be
relaxed to improve their use in your state/region?

5. Career Pathways
1. To what extent have you emphasized career pathways for AEFLA/VR participants? How
typical is it for AEFLA/VR participants to take part in career pathways programs?
2. Are most of the career pathways programs that AEFLA/VR participants take part in at the
state, regional, or local levels?
a. How are they funded?
b. What are the main targeted occupations/industries?
c. [Title II only] What is the role of the AEFLA grantees?
3. Do you coordinate with other core or required partners to develop and implement career
pathways for AEFLA/VR participants? Describe.
4. Did your efforts related to career pathways precede WIOA? If yes:
a. How are you building on your pre-existing work?
b. What has changed under WIOA? Why?
i.

Is there more coordination with core or required partners?

2.64

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 2.4

5. What challenges have you encountered in your efforts to increase access to career
pathways for AEFLA/VR participants at the state, regional, and local levels? What has
worked well?
[Community colleges only]
6. Does the state have any career pathways initiatives/efforts? If so, describe.
a. How are the state’s community colleges involved in these initiatives? What other
partners, particularly core partner programs, are involved? What are their roles?
b. Are these primarily state, regional, or local-level efforts?
c. How are these initiatives being funded (state funding/local funding)? Is any
WIOA funding being used?
d. Who are the main targets of these initiatives (adult versus youth; individuals with
disabilities, ex-offenders, individuals with low basic skills)?
a. What are the main targeted industries/occupations?
i. How has the state used LMI in the selection of these industries? To
develop career pathways training programs?
e. How are employers involved in these efforts?
7. Is there a need for additional TA or guidance from DOL or the state on career pathways?

6. Access for Individuals with Disabilities and Other Barriers
[Title II respondents]
1. How is the state trying to improve and increase access to workforce services for
individuals with low basic skills?
2. Has the state provided guidance to local areas on improving services and increasing
access to customers with low basic skills? Were state AEFLA staff involved? Describe.
3. What are local boards typically doing in this area? What is the role of local AEFLA
grantees?
4. Overall, how accessible are workforce services and AJCs to participants with low basic
skills?
a. What would need to be done to meet WIOA’s goal of ensuring equal access to
services?

[Title IV respondents]
1. How is the state trying to improve and increase access to workforce services for
individuals with disabilities? How is the VR program involved? Describe.
2. Has the state provided guidance to local boards and AJCs on improving services and
increasing access to disabled customers? What is the involvement of VR?

2.65

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 2.4

3. What are local boards typically doing in this area? What is the role of local VR staff
members?
4. Overall, how accessible are workforce services and AJCs to disabled consumers?
a. What would need to be done to meet WIOA’s goal of ensuring equal access to
services by disabled consumers?

F. Youth
[Title IV respondents only]
1. To what extent has VR increased its focus on youth as a result of WIOA? Describe.
a. What new youth-related efforts and services are you providing?
i.

What challenges have you experienced? What has worked well?

2. Have you developed new youth-related partnerships as a result of WIOA (e.g., between
VR and the Title IB Youth program or AEFLA)? If so, describe.
a. Why were these partnerships developed?
b. What challenges and promising practices have you experienced related to these new
partnerships?

G. Overall Questions
1. Partnerships
1. How would you assess the strength of your state’s relationships among core partners?
Required partners?
a. Has this changed because of WIOA?
b. If not, why do you think WIOA’s focus on enhanced partnerships has not affected
your partnerships?
2. Are there sufficient systems in place to ensure regular and effective communication
between partners?
3. What have been the key challenges and successes, if any, related to increasing
collaboration across partners?
a. Are there differences in the extent of collaboration with your core partners versus
other required partners, such as UI, SCSEP, Veterans programs, and TANF?
4. Overall, how integrated are workforce services now?
a. Has this changed because of WIOA?
b. What examples can you give that show this more integrated service delivery?

2.66

WIOA IMPLEMENTATION EVALUATION

OMB SUPPORTING STATEMENT: PROTOCOL 2.4

5. [If respondent feels workforce system is not integrated]: Why do you think WIOA’s
focus on enhanced partnerships has not affected the integration of services?

2. Systems Change
1. Overall, do you think the workforce system here has changed as a result of WIOA?
a.

If so, what are the major changes you can attribute to WIOA? E.g. employer
engagement, partnerships, services more streamlined, data sharing, others?

b. If not, do you feel systems had already begun to change prior to WIOA, or that
efforts under WIOA have not yet been effective in changing the system?
2. Do you think the role of adult education/VR in the state’s workforce system has changed
because of WIOA? How so?
3.

[If respondent doesn’t think WIOA has changed the system]: Do you feel systems had
already begun to change prior to WIOA, or that efforts under WIOA have not yet been
effective in changing the system?

4. Have there been any negative, unintended consequences of implementing WIOA?
a. Are there ways in which you feel the new law makes it more difficult to provide
effective, integrated workforce services? If so, how and why?
5. Have there been positive, unexpected benefits from implementing WIOA?
6. What additional changes are planned or needed to transform your workforce system to
meet the goals of WIOA?
a. What changes would you like to see to make the workforce system better overall?

3. Guidance and Technical Assistance
1. Of the TA provided by DOL national and regional offices on WIOA implementation,
which assistance has been most helpful?
a. Which TA formats and types of materials have been most useful for you?
2. Are you satisfied with the level of TA DOL has provided on WIOA implementation?
3. Have you accessed the ION site to obtain TA materials or guidance on specific issues?
a. Which topics have you sought out guidance on?
4. In what areas would you like to receive additional TA?

Wrap-up
1. How would you describe your overall progress to date with WIOA implementation?
a. Where do you still have plans to make significant changes?
2. Beyond what we’ve discussed today, are there other areas you would like to highlight?
3. Are there any other areas of WIOA implementation our study should explore?

2.67

This page has been left blank for double-sided copying.