Eligible Training Provider Lists Survey

ETPL COORDINATOR SURVEY

The OMB Control Number for this information collection is 1205-0436 and the expiration date is 1/31/2017. According to the Paperwork Reduction Act of 1995, persons are not required to respond to this collection of information unless it displays a currently valid OMB control number and expiration date. Responding to this survey is voluntary. Public reporting burden for this collection of information is estimated to average 15 minutes per response, including time for reviewing instructions and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information to Richard Muller at [email protected].

Thank you for taking the time to complete this brief survey. The U.S. Department of Labor is working to develop a better understanding of how the Eligible Training Provider List (ETPL) in each state is produced, administered, and used. We’re hoping that you can provide some insight into how these processes work in your state. The information you provide will be used in combination with what we learn from others to produce an overview of ETPL administration processes. The staff member responsible for coordinating the ETPL is the intended – and best – respondent for the survey.  However, coordinators are encouraged to consult, as needed, with other staff members or supervisors and refer to ETPL records when responding to survey items requiring additional details or input. If there is a question you cannot answer exactly, please provide your best estimate.

Overview

First, we have a few questions about how the Eligible Training Provider List (ETPL) is administered in your state. The ETPL is the list of all training programs in a state that are eligible to receive individual training account (ITA) funds as part of the Workforce Investment Act (WIA). The exact name for this list may vary across states.

Q1. In which state are you located?

[PROGRAMMER NOTE: Present as a drop-down list; Select only one]

Alabama 01

Alaska 02

Arizona 03

Arkansas 04

California 05

Colorado 06

Connecticut 07

Delaware 08

District of Columbia 09

Florida 10

Georgia 11

Hawaii 12

Idaho 13

Illinois 14

Indiana 15

Iowa 16

Kansas 17

Kentucky 18

Louisiana 19

Maine 20

Maryland 21

Massachusetts 22

Michigan 23

Minnesota 24

Mississippi 25

Missouri 26

Montana 27

Nebraska 28

Nevada 29

New Hampshire 30

New Jersey 31

New Mexico 32

New York 33

North Carolina 34

North Dakota 35

Ohio 36

Oklahoma 37

Oregon 38

Pennsylvania 39

Puerto Rico 40

Rhode Island 41

South Carolina 42

South Dakota 43

Tennessee 44

Texas 45

Utah 46

Vermont 47

Virginia 48

Washington 49

West Virginia 50

Wisconsin 51

Wyoming 52

GO TO Q2.

Q2. What is the name of your organization?

­­­­­­­­­­­­­­­________________________________________ GO TO Q3

Q3. What is your position within this organization?

­­­­­­­­­­­­­­­­­________________________________________ GO TO Q4a

Q4a. Please indicate whether each of the following tasks of managing the ETPL is conducted at the state level and/or WIB level. (Check all that apply)

TASK	CONDUCTED AT STATE LEVEL	CONDUCTED AT WIB LEVEL
Setting program eligibility criteria	□	□
Verifying that programs are eligible to be included on the ETPL	□	□
Updating the ETPL	□	□
Reporting program performance	□	□

GO TO Q4b.

GO TO Q4c

Q4b. Are there any other important tasks related to managing the ETPL that are conducted at the state level?

• Yes (please specify) ______________ 01

• No 02

• Blank/Refused 99  Q4c

Q4c. Are there any other important tasks related to managing the ETPL that are conducted at the WIB level?

• 

  GO TO Q5

  Yes (please specify) _____________ 01

• No 02

• Blank/Refused 99  Q5

Q5. In [state/territory], what is the minimum state-wide requirement for how frequently the ETPL is updated?

GO TO Q5a

• At least once a month 01

• Every few months 02

• Once a year 03

• Less frequently than once a year 04

• There is no minimum state-wide requirement 05 Q5b

• Other (please specify) _________________ 06 Q5b

• Blank/Refused 99 Q5b

Q5a. What portion of WIBs updates the ETPL more frequently than the minimum state-wide requirement?

• 

  GO TO Q5b

  All WIBs 01

• Most WIBs 02

• Some WIBs 03

• No WIBs 04

• Blank/Refused 99 Q5b

Q5b. Are non-compliant training programs removed from the ETPL during the updating process?

[Programmer Note: This would be a check all that apply but respondent cannot check option 3 if either 1 or 2 are checked].

GO TO Q5c

• Yes, this is occurs at the state level 01

• Yes, this occurs at the WIB level 02

• No, this does not occur at either the state or WIB level 03

• Blank/Refused 99  Q5c

Q5c. In order to be included on the ETPL in [state/territory], is it required that training programs

address high demand occupations?

[Programmer Note: This would be a check all that apply but respondent cannot check option 3 if either 1 or 2 are checked].

GO TO Q5d

• Yes, this is required at the state level 01

• Yes, this is required at the WIB level 02

• No, this is not required at either the state or WIB level 03  GO TO Q5f

• Blank/Refused 99Q5f

Q5d. In [state/territory], how is it determined which occupations are considered high demand? (Check all that apply)

GO TO Q5e

• A list of high demand occupations is developed by the state 01

• Local WIBs develop lists of high demand occupations 02

• There is no formal list, staff rely on their knowledge of the local economy 03

• Other (please specify)_______________ 04

• Blank/Refused 99 Q5e

Q5e. Can exceptions be made for including programs on the ETPL that do not address high demand occupations?

[Programmer Note: This would be a check all that apply but respondent cannot check option 3 if either 1 or 2 are checked].

GO TO Q5f

• Yes, the state allows for these exceptions 01

• Yes, WIBs allow for these exceptions 02

• No, neither the state nor the WIBs allows for these

exceptions 03

• Blank/Refused 99 Q5f

Q5f. Does the ETPL in [state/territory] include training programs that are taught exclusively online?

GO TO Q5g

• Yes 01

• No 02

• Blank/Refused 99Q5g

Q5g. If a training program is approved to be included on another state’s ETPL, which of the following processes are followed at the state and/or WIB level to include that program on [state/territory]’s ETPL? (Check all that apply)

• Out-of-state training programs are not permitted to be included on

[state/territory]’s ETPL 01

• An out-of-state program would go through the same approval

process as in-state programs, regardless of whether or not it is

already included on another state’s ETPL 02

• There is an abbreviated approval process for adding individual programs that

are already included on another state’s ETPL 03

• There is a formal agreement in place where programs on certain

states’ ETPLs are automatically eligible for inclusion on

[state/territory]’s ETPL (also known as a reciprocal agreement) 04

• Other (please specify) 05

• Blank/Refused 99

If 04 is selected go to Q5h. Otherwise go to Q6.

Q5h. You noted that programs from certain states are automatically eligible for inclusion on [state/territory]’s ETPL (also known as a reciprocal agreement). With which states are there reciprocal agreements?  (Check all that apply)

Alabama 01

Alaska 02

Arizona 03

Arkansas 04

California 05

Colorado 06

Connecticut 07

Delaware 08

District of Columbia 09

Florida 10

Georgia 11

Hawaii 12

Idaho 13

Illinois 14

Indiana 15

Iowa 16

Kansas 17

Kentucky 18

Louisiana 19

Maine 20

Maryland 21

Massachusetts 22

Michigan 23

Minnesota 24

Mississippi 25

Missouri 26

Montana 27

Nebraska 28

Nevada 29

New Hampshire 30

New Jersey 31

New Mexico 32

New York 33

North Carolina 34

North Dakota 35

Ohio 36

Oklahoma 37

Oregon 38

Pennsylvania 39

Puerto Rico 40

Rhode Island 41

South Carolina 42

South Dakota 43

Tennessee 44

Texas 45

Utah 46

Vermont 47

Virginia 48

Washington 49

West Virginia 50

Wisconsin 51

Wyoming 52

GO TO Q6.

Determining Training program Initial eligibility

Now we’d like to ask you about the criteria that are used to identify training programs that are initially eligible for inclusion on [state/territory]’s ETPL. If there is a question you cannot answer exactly, please provide your best estimate.

Q6. Which of the following items are among the state-wide criteria for determining whether a program is initially eligibility for inclusion on [state/territory]’s ETPL? (This would not apply to programs that are automatically eligible to be included on the ETPL.) (Check all that apply)

• 

  GO TO Q7

  Length of time program has been in existence 01

• Program performance outcomes 02

• Program cost 03

• Program provider licensing/accreditation 04

• Qualifications of training staff 05

• Degree or credential training customers will receive upon program completion 06

• Program addresses a high growth industry 07

• Program addresses an in-demand occupation 08

• Other (please specify)____________________ 09

• Blank/Refused 99Q7

Q7. In [state/territory], have local WIBs set initial eligibility criteria that are more rigorous than the state-wide criteria?

• 

  GO TO Q8

  Yes, all local WIBs have more rigorous criteria 01

• Yes, most local WIBs have more rigorous criteria 02

• Yes, some local WIBs have more rigorous criteria 03

• No, local WIBs do not have more rigorous criteria 04

• Blank/Refused 99 Q8

Q8. Within [state/territory], which of the following data sources are used to measure whether providers meet initial eligibility criteria? (Check all that apply)

• 

  SEE BELOW

  Self-reported data from training providers 01

• Student records (e.g. program completion, courses completed, etc.) 02

• Unemployment Insurance wage records 03

• Other (please specify) _____________________ 04

• Blank/Refused 99

If state has a waiver (identifier in sample file), GO TO Q9, ELSE GO TO Q14a.

Waiver

We’d like to ask you about the waiver [state/territory] has in place that extends the time limit on the period of initial eligibility for training programs and releases programs from subsequent eligibility requirements. If there is a question you cannot answer exactly, please provide your best estimate.

Q9. For which of the following reasons did [state/territory] request this waiver extending the period of initial eligibility? (Check all that apply)

GO TO Q10

• Maximizing trainee choice in training programs 01

• Reducing reporting burden for training providers 02

• Limitations of sources of program performance information 03

• Other (please specify) _______________________ 04

• Blank/Refused 99 Q10

Q10. To which training programs does this waiver apply? (Check all that apply)

• 

  GO TO Q10a

  All programs on the ETPL 01

• Apprenticeship programs 02

• Two-year community college and technical college programs 03

• Four-year college/university programs 04

• On-the-job training programs 05

• Private, for-profit training provider programs 06

• Programs run by community-based organizations 07

• Union programs 08

• Other (please specify) ________________ 09

• Blank/Refused 99Q10a

Q10a. Although there is a waiver in place, does the state or do any local WIBs still require the collection of updated cost/tuition information from programs after they have been approved for initial eligibility?

• Yes, it is required by the state 01Q10b

• Yes, it is required by all local WIBs 02Q10b

• Yes, it is required by some local WIBs 03Q10b

• No, it is not required by the state or by local WIBs 04Q11

• Blank/Refused 99Q11

Q10b. Within [state/territory], would failure to keep program cost/tuition below a certain level influence the decision to remove a program from the ETPL?

• Yes 01

• No 02

• It varies by WIB 03

• Blank/Refused 99

GO TO Q11

Q11. Although there is a waiver in place, does [state/territory] or do any local WIBs still require the collection of performance outcome information for training programs after they have been approved for initial eligibility?

• 

  GO TO Q12

  Yes, it is required by the state 01

• Yes, it is required by all local WIBs 02

• Yes, it is required by some local WIBs 03

• No, it is not required by the state or by local WIBs 04 Q21a

• Blank/Refused 99Q21a

Q12. Does the state determine what performance outcome information is collected or do local WIBs?

• 

  GO TO Q13

  The state alone determines what performance information is collected 01

• The state sets minimum standards about what performance information

is collected but WIBs can collect additional information 02

• Local WIBs alone determine what performance information is collected 03

• Blank/Refused 99 Q13

Now we’d like to ask you about the program performance information that is collected after initial eligibility and how program performance is measured. If there is a question you cannot answer exactly, please provide your best estimate.

Q13. In [state/territory], for which of the following types of programs is performance outcome information collected after initial eligibility? (Check all that apply)

• 

  GO TO Q13a

  All programs on the ETPL 01

• Apprenticeship programs 02

• Two-year community college and technical college programs 03

• Four-year college/university programs 04

• On-the-job training programs 05

• Private, for-profit training provider programs 06

• Programs run by community-based organizations 07

• Union programs 08

• Other (please specify) ________________ 09

• Blank/Refused 99 Q13a

Q13a. For those programs where performance outcome information is collected, which of the following program performance indicators are measured after initial eligibility? (Check all that apply)

	Performance Indicator
	Program Completion Rate	Percent of Participants who have Obtained Unsubsidized Employment	Rate of Retention in Employment at Six Months	Wages at Placement	Wages at Six Months	Rate of Licensure, Certification, Attainment of Degrees, or Attainment of Other Industry Recognized Credentials	Other (please Specify)
Measured for all Program Participants	□	□	□	□	□	□	□
Measured for WIA Participants	□	□	□	□	□	□	□
We do not collect information for this performance measure	□	□	□	□	□	□	□

• Blank/Refused 99 13b

GO TO 13b

Q13b. Within [state/territory] do you remove programs from the ETPL that fail to meet required performance levels on these indicators?

• Yes 01 Q13c

• No 02 Q13h

• It varies by WIB 03 Q13c

• Blank/Refused 99 Q13h

Q13c. Who sets these minimum performance levels that training programs must meet in order to remain eligible to be included on the ETPL?

• The state 01 Q13d

• Local WIBs 02 Q13e

• Blank/Refused 99Q13e

Q13d. For those programs that are required to meet performance standards, what are the minimum performance levels on the following indicators for programs to maintain eligibility?

PROGRAMMER NOTE: Only include those items selected in Q13a. IF 13a = 99, LIST ALL

Performance Indicator	Minimum Performance Level for All Participants	Minimum Performance Level for WIA Participants
Program completion rate	%	%
Percent of participants who have obtained unsubsidized employment	%	%
Rate of retention in employment at six months	%	%
Wages at placement	$	$
Wages at six months	$	$
Rate of licensure, certification, attainment of degrees, or attainment of other industry recognized credentials	%	%

GO TO Q13e.

Q13e. On average, how frequently are programs that do not meet minimum performance levels reviewed for removal from the ETPL?

GO TO Q13f

• At least once a month 01

• Every few months 02

• Once a year 03

• Less frequently than once a year 04

• Never 05 Q13h

• Other (please specify) _____________ 06Q13f

• Blank/Refused 99Q13g

Q13f. Is there a probationary period in which these programs are given an opportunity to improve performance without being removed from the ETPL?

GO TO Q13g

• Yes 01

• No 02

• It varies by WIB 03

• Blank/Refused 99Q13g

Q13g. If it is determined that a program should be removed from the ETPL, who removes it?

GO TO Q13h

• State agency 01

• Local WIBs 02

• Other (please specify) _______________ 03

• Blank/Refused 99 Q13h

Q13h. Which of the following data sources are used to measure training program performance following initial eligibility? (Check all that apply)

GO TO Q13i

• Self-reported data from training providers 01

• Student records (e.g. program completion, courses completed, etc.) 02

• Unemployment Insurance wage records 03

• Other (please specify) _____________________ 04

• Blank/Refused 99 Q13i

Q13i. Which factors were the most important in determining which data source(s) to use for measuring program performance? (Check all that apply)

• 

  GO TO Q21a

  Availability of the data 01

• Reducing data reporting burden for training providers 02

• Legislative limitations on data use (e.g. FERPA, UI wage data regulations) 03

• Accuracy of the data 04

• Other (please specify) _____________ 05

• Blank/Refused 99 Q21a

Determining Training program Subsequent eligibility

Now we’d like to ask you about the criteria training programs must meet in order to comply with subsequent eligibility requirements. Subsequent eligibility refers to the reassessment of program performance outcomes and other criteria after the period of initial eligibility. This is used to determine if a program remains eligible to be included on the ETPL. If there is a question you cannot answer exactly, please provide your best estimate.

Q14a. Within [state/territory], are training programs required to report updated program cost/tuition information as part of the process of applying for subsequent eligibility?

• Yes 01Q14b

• No 02Q14c

• It varies by WIB 03Q14b

• Blank/Refused 99Q14c

Q14b. Within [state/territory], would failure to keep program cost/tuition below a certain level influence the decision to remove a program from the ETPL?

• Yes 01Q14c

• No 02Q14c

• It varies by WIB 03Q14c

• Blank/Refused 99Q14c

Q14c. In [state/territory], which of the following types of programs are required to meet performance standards in order to maintain subsequent eligibility to be included on the ETPL? (Check all that apply)

• 

  GO TO Q14d

  All programs on the ETPL 01

• Apprenticeship programs 02

• Two-year community college and technical college programs 03

• Four-year college/university programs 04

• On-the-job training programs 05

• Private, for-profit training provider programs 06

• Programs run by community-based organizations 07

• Union programs 08

• Other (please specify) ________________ 09

• Blank/Refused 99 Q14d

Q14d. For those programs that are required to meet performance standards to be included on the ETPL, for each of the following criteria, please provide the specific state-wide minimum performance levels necessary for programs to maintain subsequent eligibility.

Criteria	Minimum Performance Level
Minimum completion rate for ALL program participants	%
Minimum unsubsidized employment rate for ALL program participants	%
Minimum wages at placement for ALL program participants	$
Minimum unsubsidized employment rate for WIA participants who completed the program	%
Minimum rate of retention in employment at six months for WIA participants who completed the program	%
Minimum average wages at six months for WIA participants who completed the program	$
Minimum rate of licensure, certification, attainment of degrees, or attainment of other industry recognized credentials for WIA participants who completed the program	%

GO TO Q15

Q15. Has [state/territory] set any additional state-wide performance criteria for programs to maintain subsequent eligibility?

• Yes 01 Q16

• No 02Q17

• Blank/Refused 99Q17

Q16. Please list the additional state-wide criteria as well as the minimum performance levels necessary for programs to maintain subsequent eligibility. Please also indicate whether the criteria apply to all program participants, WIA participants only, or non-WIA participants only.

State-wide Criteria	Minimum Performance Level	Applies to all Participants	Applies to WIA Participants
		□	□
		□	□
		□	□
[Include a way to add additional rows if necessary]		□	□

GO TO Q17.

Q17. In [state/territory], have local WIBs set subsequent eligibility criteria that are more rigorous than the state-wide criteria?

• 

  GO TO Q18

  Yes, all local WIBs have more rigorous criteria 01

• Yes, most local WIBs have more rigorous criteria 02

• Yes, some local WIBs have more rigorous criteria 03

• No, local WIBs have not done this 04

• Blank/Refused 99 Q18

Q18. Who collects performance information about training programs for evaluating subsequent eligibility?

• A state agency 01 Q20a

• Local WIBs 02 Q19

• Blank/Refused 99Q20a

Q19. Do all local WIBs use the same data source(s) for measuring training program performance?

• 

  GO TO Q20a

  Yes 01

• No 02

• Blank/Refused 99 Q20a

Q20a. Which of the following data sources are used to measure training program performance for subsequent eligibility? (Check all that apply)

GO TO Q20b

• Self-reported data from training providers 01

• Student records (e.g. program completion, courses completed, etc.) 02

• Unemployment Insurance wage records 03

• Other (please specify) ____________________ 04

• Blank/Refused 99 Q20b

Q20b. Which factors were the most important in determining which data source(s) to use for measuring program performance? (Check all that apply)

• 

  GO TO Q21a

  Availability of the data 01

• Reducing data reporting burden for training providers 02

• Legislative limitations on data use (e.g. FERPA, UI wage data regulations) 03

• Accuracy of the data 04

• Other (please specify) _____________ 05

• Blank/Refused 99 Q21a

reporting training outcomes

Now we have some questions about the information related to ETPL training programs that is made publicly available to training customers by the state and/or WIBs. If there is a question you cannot answer exactly, please provide your best estimate.

Q21a. Within [state/territory] through which of the following sources are training customers able to access the ETPL? (Check all that apply)

GO TO Q21b

• Online 01

• American Job Center staff 02

• Local WIB staff 03

• Training providers themselves 04

• Printed publications such as brochures and reports 05

• Other (please specify) _______________ 06

• Blank/Refused 99 Q21b

Q21b. Within [state/territory], which of the following pieces of information regarding ETPL training programs are made publicly available by the state and/or WIBs? (Check all that apply)

• 

  IF ANY OF Q21b = 01, GO TO Q22a, ELSE GO TO Q21c

  Program performance outcomes 01

• Method of course delivery (e.g. online, classroom) 02

• Program cost 03

• Program length 04

• Availability of support services (e.g. childcare, career counseling) 05

• Disability adaptation 06

• Tuition and fees 07

• Availability of financial aid 08

• Instructor qualifications 09

• Other (please specify) ____________ 10

• None of the above 11

• Blank/Refused 99 Q23

Q21c. Are there efforts underway in [state/territory] to begin making training program performance information publicly available?

• 

  GO TO Q23

  Yes 01

• No 02

• Blank/Refused 99 Q23

Q22a. Which of the following ETPL program performance information is made publicly available by the state and/or WIBs? (Check all that apply)

	Performance Indicator
Made available in aggregate about…	Program Completion Rate	Percent of Participants who have Obtained Unsubsidized Employment	Rate of Retention in Employment at Six Months	Wages at Placement	Wages at Six Months	Rate of Licensure, Certification, Attainment of Degrees, or Attainment of Other Industry Recognized Credentials	Other (please Specify)
All program participants	□	□	□	□	□	□	□
WIA participants	□	□	□	□	□	□	□
We do not make this information publicly available	□	□	□	□	□	□	□

Q22b. Of all programs on [state/territory]’s ETPL, for what portion is performance information reported for each of the following indicators?

PROGRAMMER NOTE: Only populate rows with outcome indicators selected in Q22a, IF Q22a = 99, LIST ALL

	Information reported for all programs	Information reported for more than half but not all programs	Information reported for about half of programs	Information reported for less than half of programs
Program completion rate				
Percent of participants who have obtained unsubsidized employment				
Rate of retention in employment at six months				
Wages at placement				
Wages at six months				
Rate of licensure, certification, attainment of degrees, or attainment of other industry recognized credentials				

GO TO Q22c.

Q22c. For which ETPL training programs are performance data made publicly available? (Check all that apply)

• 

  GO TO Q22d

  All programs on the ETPL 01

• Apprenticeship programs 02

• Two-year community college and technical college programs 03

• Four-year college/university programs 04

• On-the-job training programs 05

• Private, for-profit training provider programs 06

• Programs run by community-based organizations 07

• Union programs 08

• Other (please specify) ________________ 09

• Blank/Refused 99 Q22d

Q22d. Through which of the following are training customers able to access program performance information? (Check all that apply)

GO TO Q22e

• Online – through the ETPL website 01

• Online - through a website separate from the ETPL 02

• American Job Center staff 03

• Local WIB staff 04

• Training providers themselves 05

• Printed publications such as brochures and reports 06

• Other (please specify) _______________ 07

• Blank/Refused 99Q22e

Q22e. Is the training program performance information that is made publicly available, the same information that is used to determine ETPL program eligibility?

• 

  GO TO Q22f

  Yes 01

• No 02

• It varies by WIB 03

• Blank/Refused 99 Q22f

Q22f. On average, how frequently is reported performance information about each training program updated?

GO TO Q23

• At least once a month 01

• Every few months 02

• Once a year 03

• Less frequently than once a year 04

• Never 05

• Blank/Refused 99 Q23

ETPL Challenges

We’d like to briefly ask you about some potential challenges in administering the ETPL.

Q23. At the state level, please indicate the ease of the following tasks related to working with ETPL training providers.

Task	Very Easy	Easy	Difficult	Very Difficult	N/A
Getting new providers onto the ETPL					
Including enough qualified providers on the ETPL					
Communicating ETPL requirements to training providers					
Gathering information about ETPL programs from training providers					
Removing training providers from the ETPL					

GO TO Q24

Q24. At the state level, please indicate the ease of the following tasks related to maintaining ETPL program information.

Task	Very Easy	Easy	Difficult	Very Difficult	N/A
Collecting quality data about ETPL program performance					
Ensuring that programs on the ETPL correspond to high-demand occupations					
Ensuring that information about training programs on the ETPL is up-to-date					
Disseminating information about training programs to training customers					

GO TO Q25

Q25. At the state level, please indicate the ease of the following tasks related to state and local ETPL coordination.

Task	Very Easy	Easy	Difficult	Very Difficult	N/A
Ensuring local WIB compliance with state-wide ETPL requirements					
Sharing information between local and state offices					

GO TO Q26.

Finally, we’d like to give you an opportunity to provide any additional information that you think may be useful.

Q26. Please share any additional thoughts or comments you have that would enhance our understanding of how ETPL administration and management works in [state/territory].

PROGRAMMER NOTE: Provide text box

GO TO THANK YOU.

Thank you for taking the time to complete this survey. Your answers will be used in combination with others to provide feedback on how ETPLs are administered across the United States.

0