Attachment L - Question by Question Justification - Staff Survey

Attachment L Staff Survey QxQ.DOCX

OPRE Evaluation: Evaluation of Employment Coaching for TANF and Other Related Populations [Experimental impact study and an Implementation study]

Attachment L - Question by Question Justification - Staff Survey

OMB: 0970-0506

Document [docx]
Download: docx | pdf

Shape1

DRAFT


ATTACHMENT L


STAFF SURVEY Question-BY-Question JUSTIFICATION




This page intentionally left blank for double sided copying

This document provides the source and justification for each question on the Staff Survey (Attachment E).

Question #

Question text

Source

Justification

SECTION A. BACKGROUND AND PROGRAM INVOLVEMENT

1

What is your title in your current position with [program name or agency]?

STED/ETJD

(OMB No. 0970-0413)

These items measure staff background demographic characteristics.

2

How long have you been working in the position of [title from Q1] at [program name or agency]?

3

How long have you worked for [organization name]?

4

How would you describe yourself?

SOGI Report

5

What is your age?

STED/ETJD

(OMB No. 0970-0413)

6

Are you Hispanic, Latino, or of Spanish origin?

OMB

7

What is your race?

8

What is the highest level of education you have completed?

Adapted from cobra

(OMB No. 1291-0001)

9

In your position of [title from Q1] at [program name or agency], are you a full-time employee, part-time employee, full-time contractor or part-time contractor?

Adapted from STED/ETJD

(OMB No. 0970-0413)

10-M

Which of the following are among your responsibilities in your position of [title from Q1]?

Adapted from HPOG-NIE

(OMB No. 0970-0394)

These items measure the role of the staff member.

10-S

Which of the following are among your responsibilities in your position of [title from Q1]?

11-M

On average, which of your responsibilities do you spend the most time on in a typical week?

11-S

On average, which of your responsibilities do you spend the most time on in a typical week?

12

How much total work experience (including your current and prior positions) do you have in performing responsibilities similar to those you carry out as part of [program name or agency]?

JSA

(OMB No. 0970-0400)

These items measure the staff member’s employment background.

12a

How much total work experience (including your current and prior positions) do you have in coaching and/or supervising coaching?

Adapted from JSA

(OMB No. 0970-0400)

13a-M

In your position of [title from Q1] at [program name or agency], do you formally manage/supervise staff on an ongoing basis?

Adapted from HPOG-NIE

(OMB No. 0970-0394)

These items measure the role the staff member plays in the organization.

13b-M

If yes, how many staff do you typically manage/supervise?

13c-M

Do you supervise: coaches, case managers, workshop instructors, employment assistance frontline staff (e.g., job developers, trainers), administrative staff, and/or other?

14a

In your position of [title from Q1] at [program name or agency], are you responsible for working with a number of participants on an ongoing basis (i.e., do you carry a “dedicated caseload” or fill in for frontline staff when needed)?

14b

Do you work with: only participants in [treatment group], only participants in [control group], both, do not work with participants, or don’t know?

14c

On average, how many participants do you typically work with (i.e., what is your caseload) each month?


14d

During a typical session with a participant, what percentage of your time do you spend supporting the participant to set goals or work on plans to achieve goals?

New

Developed by Mathematica/Abt

These items measure the extent to which staff use the goal setting approach.

14e

During a typical session with a participant, what percentage of your time do you spend describing and directing the participant to support services?

15a

In your position of [title from Q1] at [program name or agency], have you participated in coaching training?

These items measure the extent to which staff have been trained in coaching.

15b

If yes, approximately when did you complete your initial coaching training at [program name or agency]?

15c

If yes, approximately when did you complete your most recent coaching training at [program name or agency]?

15d

What method best describes your initial coaching training and your most recent coaching training?

15e

How many hours of coaching training have you participated in, including initial and follow-up sessions?

15f

Did your coaching training include a manual or other written materials?

15f-S

Do you think the manual or other written materials are useful?

15g

Has your training included follow-up sessions or check-ins after your initial coaching training through the following methods?

  1. Peer-to-peer coaching meetings

  2. Peer-to-peer observations

  3. Case reviews

  4. Webinars

  5. Other method (please specify)

16a

Does your supervisor or manager reinforce coaching techniques and strategies with you?

16b

If yes, please select all that apply.

16c

How frequently does your supervisor or manager use coaching techniques and strategies when meeting with you?

SECTION B. TYPES OF [PROGRAM NAME] SERVICES PROVIDED

17-M

Based on your discussions with coaches, what is your perception of how often participants in [program name or agency] are able to accomplish the following:

  1. Identify a long-term goal

  2. Identify short-term goals

  3. Break down goals into small achievable steps on their own

  4. Use an identified goal to motivate themselves

  5. Identify potential challenges that may get in the way of accomplishing a goal

  6. Make adjustments to steps if unable to accomplish goal

  7. Celebrate their achievement of a goal

New

Developed by Mathematica/Abt

These items measure staff beliefs in program participants’ capacity to set and achieve goals.

17a-S

Using a scale of 1 to 5, where 1 = Never and 5 = Always, please indicate how often participants who have been in [program name or agency] for less than one month are able to work with their [coach/case manager] to accomplish the following:

  1. Identify a long-term goal

  2. Identify short-term goals

  3. Break down goals into small achievable steps on their own

  4. Use an identified goal to motivate themselves

  5. Identify potential challenges that may get in the way of accomplishing a goal

  6. Make adjustments to steps if unable to accomplish goal

  7. Celebrate their achievement of a goal

17b-S

Using a scale of 1 to 5, where 1 = Never and 5 = Always, please indicate how often participants who have been in [program name or agency] for more than one month are able to work with their [coach/case manager] to accomplish the following:

  1. Identify a long-term goal

  2. Identify short-term goals

  3. Break down goals into small achievable steps on their own

  4. Use an identified goal to motivate coaches

  5. Identify potential challenges that may get in the way of accomplishing a goal

  6. Make adjustments to steps if unable to accomplish goal

  7. Celebrate their achievement of a goal

18

Please indicate about how often you use the following [resources/process from curricula] with participants. Using a scale of 1 to 5, where 1 = Never and 5 = Always, please indicate how often these resources are used during one-on-one sessions with participants:

  1. [TOOL A]

  2. [TOOL B]

  3. [TOOL C]

  4. [TOOL D]

  5. [TOOL E]

  6. [TOOL F]

  7. [TOOL G]

  8. [TOOL H]

  9. Tangible incentives (e.g., gift cards, tokens used to buy items, other items)

  10. Other (please specify)

JSA

(OMB No. 0970-0400)

These items measure staff use of treatment intervention materials.


19

Please indicate about how valuable you find each of these resources in helping participants gain and retain employment:

  1. [TOOL A]

  2. [TOOL B]

  3. [TOOL C]

  4. [TOOL D]

  5. [TOOL E]

  6. [TOOL F]

  7. [TOOL G]

  8. [TOOL H]

  9. Tangible incentives (e.g., gift cards, tokens used to buy items, other items)

  10. Other (please specify)

20

Please indicate how much time it takes on average to complete each of the following [program name or agency] resources with a participant:

  1. [TOOL A]

  2. [TOOL B]

  3. [TOOL C]

  4. [TOOL D]

  5. [TOOL E]

  6. [TOOL F]

  7. [TOOL G]

  8. [TOOL H]

  9. Tangible incentives (e.g., gift cards, tokens used to buy items, other items)

  10. Other (please specify)

SECTION C. AMOUNT OF STAFF SUPERVISION AND MONITORING

21-M

On average, how often do you

  1. Communicate with case managers about participants’ individual situations (e.g., participant progress, strengths, challenges to participation)?

  2. Communicate with coaches about participants’ individual situations (e.g., participant progress, strengths, challenges to participation)?

  3. Communicate directly with participants about their individual situations (e.g., participant progress, strengths, challenges to participation)?

  4. Observe case managers conduct sessions with participants?

  5. Observe coaches conduct sessions with participants?

Adapted from HPOG-NIE

(OMB No. 0970-0394)

These items measure staff frequency and mode of contact with participants and supervisors.

21a-S

On average, how often do you have contact with participants through each of the following methods?

  1. In person, one-on-one session

  2. In person, group session

  3. Over the phone

  4. By email or other electronic communication

  5. Other method (please specify)

Adapted from JSa

(OMB No. 0970-0400)


21b-S

On average, how often do you work with your supervisor on an ongoing basis to practice or review coaching techniques through each of the following methods?

  1. In person, one-on-one session

  2. In person, group session

  3. Over the phone

  4. By email or other electronic communication

  5. Other method (please specify)

22-S

On average, how often do you

  1. Communicate with program management or supervisors about participants’ individual situations (e.g., participant progress, strengths, challenges to participation)?

  2. Communicate with other staff about participants’ individual situations (e.g., participant progress, strengths, challenges to participation)?

Adapted from HPOG-NIE

(OMB No. 0970-0394)

23a-S

Are follow-ups conducted as part of [program name or agency] with participants after they have found a job?

Adapted from STED/ETJD

(OMB No. 0970-0413)

23b-S

On average, how often do you use the following contact methods with participants after they have found a job?

  1. In person, one-on-one session

  2. In person, group session

  3. Over the phone

  4. By email or other electronic communication

  5. Other method (please specify)

SECTION D. CHALLENGES TO EMPLOYMENT

24

Based on your experience, on average, how frequently do participants experience the challenges listed below?

  1. Limited education

  2. Limited prior work or volunteer experience

  3. Limited relevant vocational skills

  4. Child care or dependent care issues

  5. Transportation problems

  6. Low motivation to find employment

  7. Mental health condition(s)

  8. Physical health condition(s)

  9. Limited problem-solving abilities

  10. Drug and/or alcohol addiction

  11. Learning disabilities

  12. Intimate partner violence issues

  13. Other domestic issues (e.g., divorce, child custody)

  14. Homelessness or housing problems

  15. Criminal history

  16. Legal problems

  17. Unable to focus on what’s important

  18. Procrastination

  19. Unable to follow through to the completion of a goal

  20. Inability to set goals

  21. Limited soft skills (e.g., communication skills, social skills, teamwork)

  22. Limited number of jobs that match the education, skills, and abilities of participants

  23. Limited number of good jobs (e.g., well-paying, benefits)

  24. Participant doesn’t know where to find jobs

  25. Participant afraid to approach employers

  26. Other (please specify)

Adapted from JSA

(OMB No. 0970-0400)

These items measure perceived challenges to participant employment.

SECTION E. PROGRAM PARTICIPANTS AND NONPARTICIPATION

25

Using a scale of 1 to 5, where 1 = Strongly Disagree and 5 = Strongly Agree, please indicate how much you agree or disagree with the following statements about staff at [program name or agency]:

  1. Staff in this program closely monitor the progress of participants assigned to [program name or agency]

  2. Staff in this program learn quickly about participants who did not attend planned activities

  3. Staff in this program learn quickly about a participant who quit or lost a job

  4. Staff in this program explain program expectations and consequences of nonparticipation to participants

  5. Staff in this program consistently use tangible incentives (e.g., gift cards, tokens used to buy items, other items) to encourage participants

  6. Staff in this program use positive reinforcement to encourage participation (e.g., praise, clapping)

  7. Staff in this program impose sanctions on participants who do not participate or comply with program rules

  8. Staff in this program are consistent in their use of a sanctions (e.g., apply consistent criteria)

  9. Staff implement sanctions quickly after a participant stops participating in program activities

  10. Staff work hard to reengage sanctioned participants in program activities

Adapted from JSA

(OMB No. 0970-0400)

These items measure staff efforts to increase program participation.

SECTION F. [PROGRAM NAME or AGENCY] AND ORGANIZATIONAL PERFORMANCE

26

Do you think coaching is an effective approach within [program name or agency]?

New

Developed by Mathematica/Abt

These items measure staff perception of program effectiveness.

27a

On average, how many participants benefit from the coaching approach?

27b

In your opinion, what are the characteristics of participants who benefit most from the coaching approach?

28a

Do you think a directive and prescriptive approach with participants in [program name or agency] is effective?

28b

In your opinion, what are the characteristics of participants who benefit most from a directive and prescriptive approach?

29

Using a scale of 1 to 5, where 1 = Strongly Disagree and 5 = Strongly Agree, please indicate how much you agree or disagree with the following statements about [program name or agency]:

  1. Staff make an effort to learn about participants’ personal and family situations

  2. Staff make an effort to learn about participants’ career and employment goals

  3. Staff make an effort to learn about participants’ motivation to work

  4. Staff provide information about available support services

  5. Staff assign activities for participants to do before sessions

  6. Staff help participants set personal goals

  7. Staff offer tangible incentives (e.g., gift cards, tokens used to buy items, other items) to increase participant motivation and persistence

  8. Staff check-in with participants between sessions

Adapted from JSA

(OMB No. 0970-0400)

These items measure staff perception of organizational environment.

30

Using a scale of 1 to 5, where 1 = Strongly Disagree and 5 = Strongly Agree, please rate how strongly you agree or disagree with each of the following statements about [organization name] and your experience in your position:

  1. Frequent staff turnover is a problem for your organization

  2. Staff are able to spend the time needed with participants

  3. Staff have the skills they need to do their jobs

  4. The [program name or agency] has enough staff to meet current participants’ needs

  5. [Program name or agency] staff are well-trained

  6. A larger support staff is needed to help meet needs in the [program name or agency]

  7. Staff training and professional development are priorities in the [program name or agency]

  8. The [program name or agency ] holds regular in-service training

  9. The budget of the [program name or agency] allows staff to attend professional training

  10. The [program name or agency ] is managed well

  11. The [program name or agency] has supervisors who are capable and qualified

  12. When needed, [program name or agency] supervisors devote much time and attention to staff supervision

  13. Management decisions for the [program name or agency] are well considered

  14. You have confidence in how decisions in the [program name or agency] are made

JSA

(OMB No. 0970-0400)

30

  1. You meet frequently with supervisors about participants’ needs and progress

  2. Staff concerns are ignored by management when making decisions about the [program name or agency]

  3. Some staff members seem confused about the main goals for the [program name or agency]

  4. The [program name or agency] operates with clear goals and objectives

  5. Your job duties are clearly related to the goals for the [program name or agency]

  6. Management for the [program name or agency] has a clear plan for its future

31

Using a scale of 1 to 5, where 1 = Strongly Disagree and 5 = Strongly Agree, please rate how strongly you agree or disagree with each of the following statements about [organization name] and your experience in your position:

  1. The [program name or agency] encourages and supports professional growth for the staff

  2. Keeping your knowledge and skills up-to-date is a priority for you

  3. You do a good job of regularly updating and improving your skills

  4. You regularly seek to learn new techniques or updates in the field

  5. You are satisfied with your present job

  6. You feel appreciated for the job you do

  7. You give high value to the work you do

  8. You are proud to tell others where you work

  9. You like the people you work with

  10. You would like to find a job somewhere else

  11. The heavy staff workload reduces the effectiveness of the [program name or agency]

  12. You are under too many pressures to do your job effectively

  13. Staff members at the [program name or agency] often show signs of high stress and strain

  14. Staff frustration is common where you work

  15. Staff performance measures do not align with the coaching approach

Sources: geniuss (gender-related measures overview), National implementation evaluation of the Health profession opportunity grants (HPOG-NIE), jsa (job search assistance), subsidized and transitional employment demonstration (STED) / enhanced transitional jobs demonstration (ETJD).



File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
File TitleTANF BASELINE QXQ
SubjectQxQ
AuthorBethany Boland
File Modified0000-00-00
File Created2023-08-18

© 2024 OMB.report | Privacy Policy