Use of Technology-Based Learning for Basic Skills in American Job Centers

OMB Control # 1205-0436

Expiration Date: 06/30/2020

12/11/2017 --Not for Distribution.

Survey Questionnaire

Technology-Based Learning (TBL) in American Job Centers

Dear AJC Director:

The U.S. Department of Labor’s Employment and Training Administration (ETA) is conducting a survey on how American Job Centers (AJCs) use technology-based learning (TBL). The survey focuses on use of TBL for teaching pre-vocational and job search skills, not on its use in occupational training. The information you provide will be critical in gaining a systematic understanding of how TBL is used on the front lines of the workforce system and will be important for planning and policy purposes.

Completing the questionnaire will take about 25 minutes. Please feel free to consult with other staff as needed in filling it out. Also, please note that your responses will be kept confidential; they will be combined with those from other AJCs and shared only in the aggregate in all reports and communications with ETA.

We very much appreciate your time and effort in filling out this questionnaire, and thank you in advance for helping in this important research.

• The Division of Research and Evaluation, Office of Policy Development and Research, Employment and Training Administration, U.S. Department of Labor

Note: The survey is being conducted by the Employment and Training Administration (ETA) under OMB control number 1205-0346.  ETA’s Division of Research and Evaluation (DRE), with input and support from the relevant program offices in ETA, is directing the survey, which is being administered by the research firm, Abt Associates, and its partner, MEF Associates. The survey will take approximately 25 minutes to complete and your identity and that of your center will be kept private, including not sharing it with ETA, to the extent permitted by law. If you have any questions or comments regarding this survey, please contact DRE staffer, Sande Schifferes at [email protected], or 202/693-3655.

SECTION XX: IDENTIFICATION AND CENTER CHARACTERISTICS

Name of your AJC:

City/State:

Your role at the AJC:

What type of area does your AJC serve? Check all that apply.

• Urban

• Rural

• Suburban

Estimated total number of AJC customers (participants and reportable individuals) for PY 2016 (ending June 30, 2017):

Definition:

In this survey, technology-based learning (TBL) is defined as teaching or learning activities that use electronic technology, including:

• Computer-based classes, workshops or courses, whether in-house or on-line,

• Educational software (with simulations or game-like elements), and

• Live or recorded video or audio.

1. In the past 12 months, did your AJC provide or offer access to technology-based learning in the following skill areas? (Check all that apply.)

   • Employability or workplace basic “soft” skills (i.e., punctuality, communication, etc.);

   • Basic reading and math below the 9^th grade level;

   • Building job search skills (e.g., instruction on how to write a resume, interviewing, find job openings, etc.);

   • Technological literacy/basic computer skills;

   • Other: (Please specify)

   • None - we do not offer or support any form of TBL [Skip to VERSION B, Question B2]

VERSION A of Questionnaire

Section XX

SELF-DIRECTED USE OF TBL:

2. Many AJCs provide access to technology-based learning as a self-directed service (i.e., without any staff assistance). Examples of this include online, self-paced courses, recorded training sessions, or video tutorials, all delivered without staff involvement. If you offer such self-directed TBL, please indicate how customers access it for each of the skill areas below. Check all that apply.

	Customers must visit the AJC and register there before accessing TBL for this skill area	Customers must register online but do have to visit the AJC	Anyone can access online without registering	We do not offer self-directed TBL for this skill area.
Employability or “soft” skills				
Basic reading and math below the 9th grade level				
Building job search skills				
Technological literacy/basic computer skills				

[Skip to Question 6 if respondent indicates that AJC does not offer self-directed TBL activities for any of the four areas]

3. In general, where do customers access these self-directed TBL services? Check all that apply.

• At the AJC

• Through a specific AJC partner (e.g., library, community college, employer, etc.)

• At any location with internet access

• Don’t know.

4. Please indicate below how you track self-directed customer use of TBL. Check all that apply.

   • Track “hits” on various web-pages

   • Monitor registration to access the courses

   • Monitor customer time logged in

   • Monitor customer completion of modules, units, sections, courses

   • Monitor customer progress in fixed time period

   • Receive report(s) from partners/providers

   • Receive customer self-report

   • Other (please specify): __________

   • Not applicable/Do not track self-directed service use.

5. For self-directed TBL, please indicate the software programs used for each skill development areas. Check all that apply.

GCF LearnFree

Metrix Learning

ed2go’s Career Online High School

LearningExpress

KeyTrain

Career Ready 101

Lexia

Acellus

MathXL

Odyssey

Khan Academy

Edgenuity

Achieve 3000

Lynda.com

ProveIt

Other (please specify):

Employability or “soft” skills































____

Basic reading and math below the 9th grade level































____

Building job search skills































--

Technological literacy/basic computer skills































____

Section XX

STAFF-ASSISTED TECHNOLOGY-BASED LEARNING:

Staff or instructor assisted TBL refers to learning activities in which a staff person delivers content, feedback, technical assistance, or instruction alongside use of technology-based learning. This could be in an in-person or online workshop, class, or course..

6. Please indicate below if your AJC offers staff or instructor assisted TBL in the following four skills areas.

SKILL AREA	Yes	No
Employability or “soft” skills		
Basic reading and math below the 9th grade level		
Building job search skills		
Technological literacy/Basic Computer skills		

[If no for all, skip to question 14]

7. Please indicate below if you provide staff/ instructor-assisted TBL for the following groups for each of the skill areas. Check all that apply.

SKILL AREA	Adults	Dislocated workers	Veterans	Youth	Individuals with disabilities	Prisoners	Other (please specify):
Employability or “soft” skills							
Basic reading and math below the 9th grade level							
Building job search skills							
Technological literacy/basic computer skills							

8. For staff/ instructor-assisted TBL, please indicate which software programs are used in each skill development area. Check all that apply.

GCF LearnFree

Metrix Learning

ed2go’s Career Online High School

LearningExpress

KeyTrain

Career Ready 101

Lexia

Acellus

MathXL

Odyssey

Khan Academy

Edgenuity

Achieve 3000

Lynda.com

ProveIt

Other (please specify):

Employability or “soft” skills































____

Basic reading and math below the 9th grade level































____

Building job search skills































____

Technological literacy/basic computer skills































____

9. Some AJCs provide instructor-assisted TBL that is conducted remotely (e.g., through an online course where an instructor provides video-conference lectures, feedback, etc.). Please indicate below if you offer staff/instructor-led on-line classes for the following skill areas. Check all that apply.

	Yes	No	Don’t know
Employability or “soft” skills			
Basic reading and math below the 9th grade level			
Building job search skills			
Technological literacy/basic computer skills			

10. For staff/ instructor-assisted TBL classes, which of the following tools are used for sharing course material or for communicating with students? Check all that apply.

    • Online collaboration tools (e.g., SharePoint, Google Drive)

    • Learning Management System (e.g., Blackboard)

    • Online discussion boards or forums

    • Webinars (live/recorded)

    • Videoconference technology

    • Online messaging (such as Skype, Slack, Google Hangouts, etc.)

    • Other (please specify): __________

    • Not applicable

11. For staff/ instructor-assisted TBL classes in the following skill areas, to what extent do the programs use game-like elements (“gamification”) or simulations?

	No use	Limited use	Moderate use	Significant use
Employability or “soft” skills
Basic reading and math below the 9th grade level
Building job search skills
Technological literacy/basic computer skills

12. For staff/ instructor-assisted TBL, please indicate in which settings it is offered, for each skill area. Check all that apply.

	One-on-one Assistance	Short Workshop (e.g., one or two times)	Longer Course (Multiple sessions)	Other
Employability or “soft” skills				
Basic reading and math below the 9th grade level				
Building job search skills				
Technological literacy/basic computer skills				

SECTION XX: ASSISTIVE TECHNOLOGY

13. Which of the following assistive technology tools are available for instructor-assisted TBL? Check all that apply.

    • Screen reading technology for mobile devise (e.g., iPhone’s Voiceover or Android’s TalkBack)

    • Screen reading technology for desktop or laptop (e.g. WindowEyes screen reader)

    • Closed captions(e.g., webinars, online videos)

    • Screen/text enlargement capability

    • Ubiduo devices (two-way speech generating software)

    • Word prediction software

    • Other (please specify): __________

    • Not applicable

SECTION XX: SUPPORTS FOR STAFF/INSTRUCTOR-ASSISTED TBL

14. How do you assess participants’ basic computer skills and ability to successfully use technology? Check all that apply.

    • Formal technological literacy assessment Name of assessment _______

    • Informal assessment

    • Staff recommendation

    • Customer self-assessment

    • Other (please specify): _________

    • Don’t formally assess these skills

15. Which of the following professional development opportunities are available to staff to support them in their delivery of TBL? Check all that apply.

    • Training (e.g., conferences, webinars, online courses)

    • Ongoing mentoring

    • In-person demonstration of TBL

    • Peer learning

    • Other (please specify): __________

    • Not applicable

16. SkillsCommons is an online repository of curricula, course materials, training guides, and other workforce training materials. Please indicate below if staff have used materials from SkillsCommons to develop TBL for the following skill areas.

	Yes	No	Don’t know
Employability or “soft” skills			
Basic reading and math below the 9th grade level			
Building job search skills			
Technological literacy/basic computer skills			

17. A key to successful use of TBL is ensuring that IT systems function properly. Who provides technology system support to participants and staff using TBL? Check all that apply.

    • IT support staff

    • Other AJC staff

    • Individual instructors

    • External contractor

    • Partners

    • Other (please specify): __________

    • Not applicable

SECTION XX: SATISFACTION AND PERCEIVED EFFECTIVENESS OF STAFF/INSTRUCTOR-ASSISTED TBL

18. Please rate the perceived effectiveness of staff/ instructor-assisted TBL instruction as compared to non-TBL approaches.

	More effective	The same	Less effective	Don’t know
Employability “soft” skills				
Basic reading and math below the 9th grade level				
Building job search skills				
Technological literacy/basic computer skills				

19. How do you evaluate the effectiveness of staff/ instructor-assisted TBL for the following skill areas? Check all that apply.

	Measure student progress (Examples may include assessments, pre/post tests, graded assignments, etc.)	Customer feedback	Instructor feedback	Partner feedback	Employer feedback	Other performance metrics	None
Employability or “soft” skills							
Basic reading and math below the 9th grade level							
Building job search skills							
Technological literacy/basic computer skills							

SECTION XX: IMPLEMENTATION BARRIERS

20. What type of internet connectivity does your center have? Check all that apply.

    • Dial-up

    • Broadband/High-speed (e.g., cable, DSL)

    • Other

21. Does your center offer wireless internet to customers?

    • Yes

    • No

22. To what extent do the following factors pose barriers to customers using technology-based learning programs:

	No barrier	Minimal barrier	Significant barrier	Don’t know
Low levels of basic literacy				
Low levels of technological literacy				
Lack of access to hardware				
Lack of access to software				
Lack of training to use the TBL tool				
Lack of IT systems support				
Lack of interest in using TBL				
Lack of accommodations or assistive technology ( for customers with disabilities or special needs)				

23. To what extent do the following instructor/staff related factors pose barriers to more extensive use of TBL for delivery of career services?

	No barrier	Minimal barrier	Significant barrier	Don’t know
Low levels of technological literacy				
Lack of access to hardware				
Lack of access to software				
Lack of training to use the TBL tool				
Lack of continuous supports needed to use the TBL tool (e.g., IT staff to troubleshoot)				
Lack of instructor/staff interest in using the TBL tool				

24. To what extent do the following technological factors pose barriers to more extensive use of TBL for building skills?

	No barrier	Minimal barrier	Significant barrier	Don’t know
Lack of availability or compatibility of hardware and/or software needed to use TBL tools				
Security requirements (e.g., unable to disable firewalls or pop-up blockers)				
Issues with internet connectivity or insufficient bandwidth				
Lack of responsiveness or capacity of the AJC staff responsible for IT				
Difficulty maintaining current technology (e.g., software updates)				

SECTION XX: DECISION MAKING ABOUT TBL

25. Are there plans in your AJC to expand use of TBL in any of the following skill areas? Check all that apply.

    • Employability or “soft” skills

    • Basic reading and math below the 9^th grade level

    • Building job search skills

    • Technological literacy/basic computer skills

    • Other (please specify): >>>>>>>>>>>>>>>

    • No/None [Skip to Question 27]

26. How does your AJC plan to expand TBL use in any of the four skill areas? Check all that apply.

    • Add new software programs to the curriculum

    • Add new/more hardware options

    • Expand to new populations

    • Expand to additional services (e.g., training)

    • Develop new partnerships to support TBL

    • Expand use of TBL with existing partners

    • Other (please specify): __________

27. Some AJCs involve their partners in various aspects of programming. Please indicate the role of partners in the provision of TBL. Check all that apply.

	Public libraries	Community and technical colleges	Employers	Community-based organizations	School districts/LEAs	Public housing authority	Other (please specify):
Delivery of staff-assisted TBL							___
Hardware provision							___
Oversight							___
Advising on curricula							___
Selection of TBL platforms and software							___
Data sharing on participants							___
Other (please specify):	___	___	___	___	___	___	___

28. Regarding your AJC, please indicate below who has significant input into decisions about TBL use and implementation for the following Check all that apply.

	Local Workforce Development Board	AJC Leadership	AJC staff	Partner agencies	State leaders	State Workforce Board	State workforce agency	Employers	Other training providers	For Other s, including training providers (please specify):
Selection of TBL programs										___
How content is delivered										___
Resource use (staff time, classroom space, hardware, etc.)										___
Priority populations for TBL use										___

[If do not select “AJC Leadership” or “AJC staff” skip to Question 28]

29. Please rank the top three factors you consider when deciding to use TBL in your AJC. [Only if respondent selected at least one row in the “AJC staff” or “AJC leadership” column]

___ Cost

___ Support/training available

___ Alignment with partner curricula or training

___ Ease of use

___ Staff readiness

___ Customer readiness

___ Compatibility with existing AJC hardware

___ Demonstrated effectiveness in supporting job retention or advancement

___ Adaptability to customers’ circumstances/needs

___ Scalability

___ Gaps in existing services

Thank you for your participation.

[END OF SURVEY – Version A]

VERSION B

[For respondent answering “None” to Question 1 in Version A.]

B2. Has your AJC ever used TBL for employability or “soft” skills, basic reading and math below the 9^th grade level, building job search skills, or technological literacy/basic computer skills?

• Yes [Continue to B3]

• No [Skip to B5]

B3. Why did your AJC stop using TBL for employability or “soft” skills, basic reading and math below the 9^th grade level, building job search skills, or technological literacy/basic computer skills? Check all that apply.

• Cost of software

• Cost of hardware

• Lack of customer interest

• Lack of instructor interest

• Customer readiness

• Instructor readiness

• Lack of applicability to jobs or educational pathways

• Perceived ineffectiveness of TBL tools

• Other (please specify): __________

B4. Who has input into the following decisions about TBL use and implementation? Check all that apply.

	Local Workforce Development Board	AJC Leadership	AJC staff	Partner agencies	State leaders	State Workforce Board	State workforce agency	Employers	Other training providers	Other (please specify):
Selection of TBL platforms										____
Content delivery										____
Resource use (staff time, classroom space, hardware, etc.)										____
Priority populations for TBL use										____

B5. Are there plans in your AJC to implement use of TBL in any of the following skill areas? Check all that apply.

• Employability or “soft” skills

• Basic reading and math below the 9^th grade level

• Building job search skills

• Technological literacy/basic computer skills

• Other (please specify): _________________

• None [Skip to QB7]

B6. How does the AJC plan to implement or resume use of TBL in the future?

• Add software programs to the curriculum

• Add hardware

• Develop partnerships to support TBL

• Implement TBL within existing partnerships

• Other (please specify):

B7. What specific supports/assistance would be helpful in implementing TBL? Check all that apply.

• More information on TBL tools/software

• Learning opportunities from other AJCs

• Needs assessments of partners to understand if TBL is useful

• Needs assessments of customers to understand if TBL would be beneficial

• Training opportunities for AJC staff

• Expanded partnerships around TBL

• Resources to purchase new hardware

• Other (please specify): __________

B8. To what extent do the following customer related factors pose barriers to more extensive use of TBL for delivery of career services?

	No barrier	Minimal barrier	Significant barrier	Don’t know
Low levels of literacy				
Low levels of technological literacy				
Lack of access to hardware				
Lack of access to software				
Lack of training to use the TBL tool				
Lack of support needed to use TBL (e.g., IT staff to troubleshoot)				
Lack of interest in using TBL				
Lack of accommodations or assistive technology for customers with disabilities or special needs

B9. To what extent do the following instructor/staff related factors pose barriers to more extensive use of TBL for delivery of career services?

	No barrier	Minimal barrier	Significant barrier	Don’t know
Low levels of technological literacy				
Lack of access to hardware				
Lack of access to software				
Lack of training to use the TBL tool				
Lack of continuous supports needed to use the TBL tool (e.g., IT staff to troubleshoot)				
Lack of instructor/staff interest in using the TBL tool				

B10. To what extent do the following technological factors pose barriers to more extensive use of TBL for delivery of career services?

	No barrier	Minimal barrier	Significant barrier	Don’t know
Lack of availability or compatibility of hardware and/or software needed to use TBL tools				
Security requirements (e.g., unable to disable firewalls or pop-up blockers)				
Issues with internet connectivity or insufficient bandwidth				
Lack of responsiveness or capacity of the AJC staff responsible for IT				
Difficulty maintaining current technology (e.g., software updates)				

Thank you for your participation.

[END OF SURVEY – Version B]