Frontline Staff Web Survey

INSTRUMENT 4:

FRONT LINE STAFF SURVEY

Insert logo here

OMB Control No: 0990-new

Expiration Date: XX/XX/XXXX

IMAGIN Study

Frontline Staff Survey

January 2019

THE PAPERWORK REDUCTION ACT OF 1995

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0990-new. The time required to complete this information collection is estimated to average 30 minutes per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: U.S. Department of Health & Human Services, OS/OCIO/PRA, 200 Independence Ave., S.W., Suite 336-E, Washington D.C. 20201, Attention: PRA Reports Clearance Officer

INTRODUCTION AND INSTRUCTIONS

Thank you for helping with this important study. Your input on this survey will help us to understand the early stages of implementation of your sexual risk avoidance education or sexual risk reduction program. This survey asks questions about (1) the program you are implementing and its target population, (2) your role and prior experiences, (3) the training and preparation provided to staff, (4) your experiences with implementation and data collection for evaluation, (5) key lessons learned, and (6) your background. The survey should take about 30 minutes to complete.

Note: Program is defined as the entire program model including any specific curricula, components, and services; Curriculum is the structured SRA/SRR content (e.g. a series of lessons) that need to be delivered as part of the broader program model.

We invite you to participate in this survey. We hope you will find it interesting to describe your experiences implementing this program.

PRIVACY

Your responses will be kept private. Your responses will not be shared with your supervisor, coworkers, or your organization. Your identity will not be revealed in reports, presentations, or articles and will not be recognizable to anyone beyond the research team. We will use a study identification number to track responses and follow up with non-respondents.

In any reporting, we will not include your name or title. All responses will be reported as a group response only, for example, “Most program staff reported that . . .”

Please give your most honest and complete answers so that your thoughts and opinions can help provide a better understanding of innovative sexual risk avoidance or risk reduction programs, how they’re operating, and what is needed to ensure that they are ready for implementation and evaluation. Your responses will be used for research purposes only. Your individual responses will not be shared with the funder, other staff from your organization, or anyone outside the research team; and, again, your name will not be on this survey.

COMPENSATION AND FREEDOM TO WITHDRAW

Completion of this survey is voluntary. There is no compensation for completing this survey. You may refuse to answer specific questions or discontinue your participation at any time without any penalty. There are no right or wrong answers to these questions.

Please answer questions to the best of your ability.

If you have a comment or a question about the survey or would like to clarify or amend an answer in any way, we have included a space at the end of the survey where you can record your additional thoughts or comments.

Thank you for your participation!

PLEASE MARK ALL ANSWERS WITHIN THE WHITE BOXES PROVIDED.

PLEASE READ EACH QUESTION CAREFULLY. There are different ways to answer the questions in this survey. It is important that you follow the instructions when answering each kind of question.

This first section includes questions about the program and your target population. Note: Program is defined as the entire program model including any specific curricula, components, and services; Curriculum is the structured SRA/SRR content (e.g. a series of lessons) that need to be delivered as part of the broader program model.

A1. Please confirm the name of the program you are currently providing: [Prefilled]

1 □ Yes, this is correct.

0 □ No, the name of my program is: _________________________________

A2. Does the program have a defined curriculum?

1 □ Yes

0 □ No SKIP TO A3

A2a. What is the name of the curriculum? If there is more than one, please list all curricula included in the program.

Curriculum A

Curriculum B

Curriculum C

Curriculum D

A3. Which of the following topics are part of the program?

Mark all that apply

1 □ General health topics

2 □ Healthy decision-making

3 □ Adolescent development

4 □ How to avoid or prevent sexual risk

5 □ How to reduce sexual risk

6 □ Healthy relationships

7 □ Positive youth development

8 □ Enhancing protective factors (among youth, among families and caregivers, and on a systems-level)

9 □ A holistic focus on physical, social and emotional health (in other words, optimal health)

10 □ Other (specify)

A4. Were you involved in the development or selection of the program (including any curricula) you are delivering?

MARK ONE ONLY

1 □ Yes, I was very involved

2 □ Yes, I was a little bit involved

0 □ No, I was not involved

A5. Typically, programs are designed to serve a specific population. Please describe who you are delivering the program to.

A6. What are the primary needs of the population you deliver the program to?

Mark all that apply

1 □ Reducing teen pregnancy

2 □ Reducing risky sexual behaviors

3 □ Reducing other risky behaviors, such as smoking, alcohol or drug abuse

4 □ Completing education

5 □ Improving healthy decision-making among youth

6 □ Improving mental and emotional health

7 □ Other (specify)

A7. In your opinion, does the program you are delivering need to add more services in order to meet the needs of your target population?

1 □ Yes

0 □ No

A8. Apart from the program you are delivering, are there similar programs that address the needs of your target population?

MARK ONE ONLY

0 □ We need many more similar programs in the community.

1 □ We could use a few more similar programs in the community.

2 □ We have a sufficient number of similar programs in the community.

3 □ We have too many similar programs in the community.

This section includes questions about your role, professional experiences, and background.

B1. What is your job title?

B2. How long have you worked in this position?

MARK ONE ONLY

1 □ Less than 1 year

2 □ 1 - 3 years

3 □ 4 - 7 years

4 □ 8 - 10 years

5 □ More than 10 years

B3. Which of the following activities do you take part in as part of this job?

MARK ALL THAT APPLY

1 □ Provide sexual risk avoidance or sexual risk reduction education

2 □ Provide health education

3 □ Provide education on academic subjects (e.g., English language arts, math)

4 □ Provide counseling

5 □ Manage or supervise individuals at your organization

6 □ Train staff at your organization

7 □ Other activities (specify):

B4. Including your time in your current position, how many years of experience do you have taking part in these activities?

MARK ONE ONLY

1 □ None

2 □ Less than 6 months

3 □ 6 to 11 months

4 □ 1 to 2 years

5 □ 3 to 5 years

6 □ More than 5 years

B5. Prior to your current position, in which types of fields did you work?

MARK ALL THAT APPLY

1 □ Sexual and reproductive health counseling

2 □ Other counseling

3 □ Education

4 □ Vocational rehabilitation

5 □ Juvenile justice

6 □ Psychology

7 □ Social work/human services

8 □ Medicine

9 □ Administration

10 □ Child development

11 □ Child welfare

12 □ Other (specify):

B6. How many years of experience do you have working with the target population of this program?

MARK ONE ONLY

1 □ None

2 □ Less than 6 months

3 □ 6 to 11 months

4 □ 1 to 2 years

5 □ 3 to 5 years

6 □ More than 5 years

B7. What is the highest level of education you have completed?

MARK ONE ONLY

1 □ Some high school

2 □ High school diploma or equivalent

3 □ Postsecondary vocational or technical training

4 □ Some college, no degree

5 □ Associate’s degree

6 □ Bachelor’s degree

7 □ Master’s degree

8 □ Doctorate or other professional degree

B8. Do you currently have a professional license, certification, or credential?

1 □ Yes (specify):

0 □ No

The next questions ask about the types and frequency of training you needed or have received in order to deliver the program. Note: Program is defined as the entire program model including all components, services, and any specific curricula; Curriculum is the structured SRA/SRR content (e.g. a series of lessons) that need to be delivered as part of the program.

C1. Did you receive training on how to deliver the program?

1 □ Yes

0 □ No SKIP TO C10

C2. To date, how many trainings for the program have you attended? Please include training that occurred both before and while you were delivering the program model, as applicable.

________________________ NUMBER OF TRAININGS

C3. What types of training(s) did you receive on how to deliver the program?

MARK ALL THAT APPLY

1 □ ln-person training

2 □ Online training

3 □ Webinar

4 □ Self-review of training manual

5 □ Self-review of implementation plans

6 □ Other (specify):

C4. When did the training(s) occur?

MARK ONE ONLY

1 □ Before you began delivering services

2 □ While you were delivering services

3 □ Both

C5. Who provided the training on how to deliver program?

MARK ALL THAT APPLY

1 □ Supervisor

2 □ Developer of the program

3 □ Colleague or peer

4 □ Someone else (specify):

C6. What topics were covered in the training(s)?

MARK ALL THAT APPLY

1 □ General health topics

2 □ Healthy decision-making

3 □ Adolescent development

4 □ How to avoid or prevent sexual risk

5 □ How to reduce sexual risk

6 □ Healthy relationships

7 □ Positive youth development

8 □ Teaching strategies

9 □ How to work with the target population

10 □ Enhancing protective factors (among youth, among families and caregivers, and on a systems-level)

11 □ A holistic focus on physical, social and emotional health (in other words, optimal health)

12 □ Other (specify)

C7. How prepared did you feel to deliver the program after completing the training(s) for your program?

MARK ONE ONLY

1 □ Very prepared

2 □ Somewhat prepared

3 □ Not prepared at all

C8. Thinking across all your training, please rate the usefulness of the training(s) in preparing you to deliver the program.

MARK ONE ONLY

1 □ Very useful

2 □ Somewhat useful

3 □ Not at all useful

C9. Which of the following statements most accurately describes how the training(s) affected your ability to overcome any challenges you faced as you delivered the program.

MARK ONE ONLY

1 □ The training helped me overcome the challenges I faced a great deal.

2 □ The training helped me overcome the challenges I faced a little bit.

3 □ The training did not help me overcome the challenges I faced.

4 □ I already had the skills I needed to overcome the challenges I faced.

N □ I have not faced any challenges in delivering the program.

C10. Please read the following statements about delivering the program and rate the extent to which you agree or disagree:

	SELECT ONE RESPONSE PER ROW
	STRONGLY DISAGREE	DISAGREE	AGREE	STRONGLY AGREE
a. I received the support I needed to deliver the program	1 □	2 □	3 □	4 □
b. I have enough time to deliver the program as planned	1 □	2 □	3 □	4 □
c. I know how to work with this target population	1 □	2 □	3 □	4 □
d. I understand the content and how to use the strategies required by the program	1 □	2 □	3 □	4 □

The following questions focus on your experiences delivering the program in the past 6 months. Note: Program is defined as the entire program model including all components, services, and any specific curricula; Curriculum is the structured SRA/SRR content (e.g. a series of lessons) that need to be delivered as part of the program.

D1. To what extent was the program delivered as planned?

MARK ONE ONLY

0 □ The program was not delivered during the past 6 months

1 □ The majority of the program was not delivered as planned (in other words, there were extensive changes to the program or only 25 percent of the program was delivered as planned)

2 □ Some of the program was delivered as planned (in other words, about 50 percent of the program’s components were delivered as planned)

3 □ Most or all of the program was delivered as planned (in other words, 75 percent or more of the program’s components were delivered as planned)

D2. To what extent were participants engaged (e.g. actively involved, participating in activities, and paying attention) in the program?

MARK ONE ONLY

0 □ Less than a quarter of participants were engaged in the program

1 □ About a quarter of participants were engaged in the program

2 □ About half of the participants were engaged in the program

3 □ About three quarters of the participants were engaged in the program

4 □ All of the participants were engaged in the program

The next set of questions ask you about the types of supervision you receive.

D3. Do you receive supervision as you deliver the program?

1 □ Yes

0 □ No SKIP TO D8

D4. What kind of supervision do you receive as you deliver the program?

MARK ALL THAT APPLY

1 □ My supervisor helps me complete difficult activities and tasks

2 □ My supervisor asks me to work independently and seek resources that I need on my own

3 □ My supervisor sets high, but reasonable standards for my work

4 □ My supervisor sets deadlines and timelines for me

5 □ My supervisor asks that I collaborate with other staff

6 □ My supervisor has helped staff develop into an effective team

7 □ My supervisor regularly asks me about the progress I have made

8 □ My supervisor is available when I ask for help

9 □ My supervisor reinforces the training I received

10 □ My supervisor helps me learn and improve

11 □ My supervisor listens to me

12 □ My supervisor has an attitude that helps me be enthusiastic about my work

13 □ My supervisor provides strategies to help me deliver the program and work with the target population

14 □ My supervisor supports my understanding of the curriculum or program content and mission

15 □ Other (specify):

D5. How frequently do you communicate with your supervisor about program delivery?

MARK ONE ONLY

1 □ Daily

2 □ A few times a week

3 □ Once a week

4 □ Once every two weeks

5 □ Once a month

6 □ Other (specify):

D6. Please rate the adequacy of the supervision you received as you delivered the program.

MARK ONE ONLY

1 □ Very supportive

2 □ Somewhat supportive

3 □ Not supportive

D7. Please rate the quality of supervision since you began delivering the program.

MARK ONE ONLY

1 □ Decreased

2 □ Remained the same

3 □ Improved

D8. If you could make any changes to improve the supervision you receive, which of the following would you include?

MARK ALL THAT APPLY

0 □ No changes

1 □ Increased communication

2 □ Decreased communication

3 □ More support and guidance

4 □ More feedback on my work

5 □ Less feedback on my work

6 □ More structure and rules

7 □ Less structure and rules

8 □ More encouragement of collaboration

9 □ Other (specify):

D9. Beyond general supervision, have you received targeted feedback on delivering the program (including any specific curriculum, if applicable)?

1 □ Yes

0 □ No SKIP TO D13

D10. How frequently do you receive targeted feedback on delivering the program (including any specific curriculum, if applicable)?

MARK ONE ONLY

1 □ Daily

2 □ A few times a week

3 □ Once a week

4 □ Once every two weeks

5 □ Once a month

6 □ Other (Please describe):

D11. Who provided you with targeted feedback?

MARK ALL THAT APPLY

1 □ Supervisor

2 □ Program trainer

3 □ Colleague or peer

4 □ Someone else (Specify):

D12. Please rate the usefulness of the targeted feedback you received in helping you deliver the program.

MARK ONE ONLY

1 □ Very useful

2 □ Somewhat useful

3 □ Not at all useful

The next set of questions focus on how the program was implemented. Note: Program is defined as the entire program model including any specific curricula, components, and services; Curriculum is the structured SRA/SRR content (e.g. a series of lessons) that need to be delivered as part of the broader program model.

D13. Do you use a written manual that is a guide on how to deliver the program (including any curricula)?

1 □ Yes

0 □ No

D14. Are the expectations for how the program should be delivered (including any curricula) clear?

MARK ONE ONLY

0 □ Not at all clear

1 □ A little clear

2 □ Somewhat clear

3 □ Very clear

D15. How do you know if the program is delivered as it should be (including any curricula)?

MARK ALL THAT APPLY

1 □ There are specific content, lessons, and/or curriculum that must be provided

2 □ There is clear guidance on the number of program sessions and activities that must be provided

3 □ There are clear instructions and standards for the quality and methods of program delivery

4 □ I don’t know how the program should be delivered

5 □ Other (specify):

D16. On your first day delivering the program....

	SELECT ONE RESPONSE PER ROW
	NOT READY	SOMEWHAT READY	READY
a. To what degree did you feel ready to deliver the program?	1 □	2 □	3 □
b. To what degree was the setting or location ready for program delivery?	1 □	2 □	3 □
c. To what degree was the program ready to be delivered? (for example, curricula, lesson plans, manuals and guides)	1 □	2 □	3 □
d. To what degree was the organization ready for program delivery? (for example, facilities, finances, administrative support, leadership support)	1 □	2 □	3 □

D17. In thinking about program delivery in the upcoming year....

	SELECT ONE RESPONSE PER ROW
	NOT READY	SOMEWHAT READY	READY
a. Do you feel ready to deliver the program?	1 □	2 □	3 □
b. To what degree is the setting or location ready for program delivery?	1 □	2 □	3 □
c. To what degree is the program ready to be delivered? (for example, curricula, lesson plans, manuals and guides)	1 □	2 □	3 □
d. To what degree is the organization ready for program delivery? (for example, facilities, finances, administrative support, leadership support)	1 □	2 □	3 □

Sometimes staff have to make changes on how a program is delivered to meet the needs of program participants, the timeline, and/or organizational resources, or some other factor. The next questions are about adjustments or adaptations you made to your program or services for any reason.

D18. Which of the following best describes what adjustments or adaptations you have made while delivering the program?

MARK ONE ONLY

1 □ I have been able to use the program exactly as designed (without any adjustments or adaptations) to meet the needs of my participants SKIP TO D20

2 □ I have made minor adjustments or adaptations to the program design in order to meet the needs of my participants

3 □ I have made significant adjustments or adaptations to the program’s design (for example, structure, content, methods) in order to meet the needs of my participants

D19. Which of the following adjustments or adaptations did you make to the program’s design in order to meet the participants’ needs?

MARK ALL THAT APPLY

1 □ Changed procedures and methods

2 □ Changed the sequence of sessions or activities

3 □ Increased the number of sessions or visits

4 □ Decreased the number of sessions or visits

5 □ Increased the length of sessions or visits

6 □ Decreased the length of sessions or visits

7 □ Adjusted how youth were recruited into the program

8 □ Adjusted the setting of the program

9 □ Changed program content (specify):

10 □ Changed program materials ( specify):

11 □ Other (specify):

The following questions focus on key factors that may have affected your program’s implementation.

D20. Please read the following statements about your organization and rate the extent to which you agree or disagree about how each relates to delivering your program well:

	SELECT ONE RESPONSE PER ROW
	STRONGLY DISAGREE	DISAGREE	AGREE	STRONGLY AGREE
a. My organization has enough money to deliver program activities	1 □	2 □	3 □	4 □
b. My organization has adequate space for program activities	1 □	2 □	3 □	4 □
c. My organization has adequate equipment and supplies for program activities	1 □	2 □	3 □	4 □
d. My organization has enough of the right staff to deliver the program activities	1 □	2 □	3 □	4 □
e. My organization has enough of the right staff to work with the target population	1 □	2 □	3 □	4 □
f. My organization has enough administrative or support staff	1 □	2 □	3 □	4 □
g. My organization is able to solve the challenges we face in implementing the program	1 □	2 □	3 □	4 □

D21. Please read the following statements about staffing and delivering the program and rate the extent to which you agree or disagree:

	SELECT ONE RESPONSE PER ROW
	STRONGLY DISAGREE	DISAGREE	AGREE	STRONGLY AGREE
a. Staff in my organization agree on what our program participants’ needs and interests are.	1 □	2 □	3 □	4 □
b. Staff in my organization have the right skills or qualifications.	1 □	2 □	3 □	4 □
c. Staff in my organization offer program activities at times convenient for participants.	1 □	2 □	3 □	4 □
d. Staff in my organization get enough participants engaged in services.	1 □	2 □	3 □	4 □
e. There is frequent staff turnover.	1 □	2 □	3 □	4 □

D22. Please rate how much you agree or disagree with the following statements on the alignment of the program with the needs of the target population:

	SELECT ONE RESPONSE PER ROW
	STRONGLY DISAGREE		DISAGREE		AGREE		STRONGLY AGREE		DON’T KNOW
a. This program is making a difference for the target population	1 □		2 □		4 □		5 □		d □
b. This program is addressing the primary needs of the target population	1 □		2 □		4 □		5 □		d □
c. This program fits well with the values and philosophy of this setting and population	1 □		2 □		4 □		5 □		d □
d. This community is supportive of this program and others that address teen sexual risk behaviors	1 □	2 □		4 □		5 □		d □
e. The target population is interested in the program offered	1 □	2 □		4 □		5 □		d □
f. Members of the target population were involved in the developing or selecting the program	1 □	2 □		4 □		5 □		d □

D23. If the program is not well-aligned with the needs of the target population, please describe why:

D24. Please describe any challenges you faced as you delivered the program.

The following questions ask about any data you may have gathered as you delivered the program and how it was used.

D25. Did you collect information or data on how the program was delivered? This can include program attendance, surveys of participants to ask about what they learned or their perceptions about the program, or about other program activities in which participants were engaged.

1 □ Yes

0 □ No SKIP TO SECTION E

D26. How did you collect information or data on how the program was delivered?

MARK ALL THAT APPLY

1 □ Surveys or questionnaires filled out by program participants

2 □ Focus groups or discussions with program participants or their families

3 □ Attendance (e.g., session rosters, sign in sheets)

4 □ Coursework, homework, written activities

5 □ Observations of program sessions

6 □ Surveys or questionnaires filled out by staff members who implemented the program

7 □ Program schedules or calendars

8 □ Curricula or other program materials

9 □ Other (specify) __________________________________________________________

D27. Which of the following types of information or data did you collect on how the program was delivered?

MARK ALL THAT APPLY

1 □ The number of sessions or activities participants attended or engaged in

2 □ The number of program activities or sessions offered to participants

3 □ Participants’ satisfaction with the program

4 □ Participants’ attitudes about topics covered in the program (e.g., how to avoid risky behavior)

5 □ Participants’ knowledge of program objectives

6 □ Participants’ knowledge of the program content

7 □ Changes or adjustments to content or activities

8 □ Other (specify) __________________________________________________________

d □ Don’t’ know

D28. Did supervisors use this data to provide feedback to you or other staff members to improve the ways the program was delivered?

1 □ Yes

2 □ No

3 □ Don’t know

This section focuses on the lessons you learned as you implemented this program.

E1. Overall, how prepared did you feel your organization was in delivering the program in the first 6 months?

MARK ONE ONLY

1 □ Not at all prepared

2 □ Somewhat prepared

3 □ Very prepared

E2. How would you rate your experience delivering the program in the past year?

MARK ONE ONLY

1 □ Difficult

2 □ Somewhat difficult

3 □ Somewhat easy

4 □ Easy

E3. Looking ahead to when you will use this program again, is there anything you would do differently or change to improve program delivery?

E4. What are some successes you experienced in delivering the program over the past year?

F1. What is your age?

| | | years

F2. Which of the following best describes you?

1 □ Male

2 □ Female

3 □ Prefer not to say

F3. Are you Hispanic or Latino?

1 □ Yes

0 □ No SKIP TO F5

F4. Which of the following best describes you?

MARK ALL THAT APPLY

1 □ Mexican

2 □ Puerto Rican

3 □ Cuban

4 □ South or Central American

5 □ Of another Spanish culture or origin

F5. Which of the following best describes you?

MARK ALL THAT APPLY

1 □ American Indian or Alaska Native

2 □ Asian

3 □ Black or African American

4 □ Native Hawaiian or Other Pacific Islander

5 □ White

6 □ Prefer not to say

Thank you for taking the time to complete this survey.

This page has been left blank for double-sided copying.