IE Technical Reviewers Survey 7-19-07.rev.03-04-08.xls

The Independent Evaluation of the SAPT BG Program

Dear Technical Reviewer:

As you know, the Substance Abuse Prevention and Treatment Block Grant (SAPT BG) Program was authorized by Congress to provide funds to States, Territories, and one Indian Tribe for the purpose of planning, implementing, and evaluating activities to prevent and treat substance abuse and is the largest Federal program dedicated to improving substance abuse prevention and treatment systems. The sponsors of the program, the Substance Abuse and Mental Health Services Administration’s Center for Substance Abuse Prevention and Center for Substance Abuse Treatment, have contracted with Altarum Institute (formerly Health Systems Research, Inc.), a public policy consulting firm in Washington D.C., to conduct an independent evaluation of this program.

We are soliciting feedback about the SAPT BG from key program stakeholders. As a past Technical Reviewer, you have important insights and views about the intent, implementation, and impact of the SAPT BG from your site visits to States. We would greatly appreciate your assistance with the evaluation through the completion of this survey. Most of the questions are closed-ended questions where you will be asked to check the appropriate answer or answers. In addition, there are several open-ended questions where you have the opportunity to comment. We urge you to be as honest and thorough as possible.

We will only report aggregated responses to the questions, and we will never attribute specific comments to particular individuals. Your responses will not be used by CSAT to assess State compliance with the requirements and will not have any repercussions for any particular State; they will be used solely for the purpose of evaluating the overall SAPT BG Program.

Thank you very much for taking the time to participate.

OMB Clearance Number:

Expiration Date:

Estimates of Burden for the Collection of Information.

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid Office of Management and Budget (OMB) control number. The valid OMB control number for this collection is XXXX-XXXX. The time required to complete this information collection is estimated to average 60 minutes per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collected.

1)

Have you participated in Core Elements Technical Reviews? [If yes, continue. If no, skip to question 28.]

Yes

No

Cannot answer

2)

If yes, which of the following was your role? (Check all that apply)

Management reviewer

Clinical reviewer

Fiscal Team Leader

Other (please describe):

Team Leader

3)

What is the purpose of the Core Elements Technical Review?

4)

How often does a State representative identify compliance issues for the reviewers during the Core Elements Technical Review site visit?

Never

Rarely

Sometimes

Usually

Always

Cannot answer

5)

When conducting Core Elements Technical Review site visits, how often do you identify potential issues for Federal action?

Never

Rarely

Sometimes

Usually

Always

Cannot answer

6)

When conducting Core Elements Technical Review site visits, how often do you identify potential issues for State action?

Never

Rarely

Sometimes

Usually

Always

Cannot answer

7)

How are issues that require action communicated to the States? (Check all that apply)

E-mail contact

Phone contact

Exit interview at the conclusion of site visit

Site visit report

Other site visit products

(please describe)

Other (please describe)

8)

How are issues that require action communicated to CSAT? (Check all that apply)

Email contact

Phone contact

Exit interview at the conclusion of site visit

Site visit monitoring report

Other site visit products

(please describe)

Other (please describe)

9)

What are the strengths of the Core Elements Technical Review process?

10)

What are the weaknesses of the Core Elements Technical Review process?

11)

To what extent do you agree that the Core Elements Technical Review site visits are useful to States?

Strongly disagree

Somewhat disagree

Neither disagree nor agree

Somewhat agree

Strongly agree

Cannot answer

12)

Please list any ways in which you feel the Core Elements Technical Review site visits could be more useful to States.

13)

What types of training, if any, did you receive to prepare you for conducting the Core Elements Technical Review site visits? (Check all that apply)

In-person training

Written instructions

No training received

Shadowing of experienced site visitors

Other (please describe)

14a)

If you received in-person training, please rate how useful you feel the training you received was.

Not useful

A little useful

Somewhat useful

Mostly useful

Extremely useful

Cannot answer

14b)

If you received in-person training, how long before the actual site visits began did the training occur? (Check one)

Within 1 week

Within 2-3 weeks

Within 1 month

Within 2 months

Longer than 2 months

Other (please describe)

15a)

Do you have any recommendations for improving the training and preparation for the site visits to make you a more effective reviewer?

Yes

No

15b)

If yes, what are they?

16a)

If you received written instructions, please rate how useful they were.

Not useful

A little useful

Somewhat useful

Mostly useful

Extremely useful

Cannot answer

16b)

If you received written instructions, how long before the actual site visits began did you receive them? (Check one)

Within 1 week

Within 2-3 weeks

Within 1 month

Within 2 months

Longer than 2 months

Other (please describe)

16c)

How useful was the Core Element Review Protocol in helping to gather the information needed to prepare the report?

Not useful

A little useful

Somewhat useful

Mostly useful

Extremely useful

Cannot answer

17a)

Were you provided any other materials in preparation for the site visits?

Yes

No

17b)

If yes, what other materials?

18a)

Do you think that there are requirements of the SAPT BG program that are not addressed adequately in the protocols for the Core Elements Technical Reviews?

Yes

No

18b)

If yes, please discuss.

19a)

Did you receive information about the State’s responsibilities in the Core Elements Technical Review site visit?

Yes

No

Cannot answer

19b)

If yes, what information did you receive?

20)

On average, how prepared are the States that you have visited on Technical Review site visits?

Very unprepared

Somewhat unprepared

Minimally prepared

Very prepared

Completely prepared

Cannot answer

21)

What products result from the Core Elements Technical Review site visits to States? (Check all that apply)

Site visit report and recommendations

Technical assistance plan

Strategic plan

Other (please describe)

22)

How long after a site visit do you typically submit your drafts of site visit products to CSAT? (Check one)

Within 1 week

Within 2-3 weeks

Within 1 month

Within 2 months

Longer than 2 months

Other (please describe)

23)

How long after the submission of the technical review report draft to CSAT do States typically receive a copy of the site visit report? (Check one)

Within 1 week

Within 2-3 weeks

Within 1 month

Within 2 months

Longer than 2 months

Other (please describe)

24a)

Do you know if Federal program staff and grants management use site visit products?

Yes

No

Don't know

24b)

If yes, how?

25a)

Do you know if States use site visit products?

Yes

No

Don't know

25b)

If yes, how?

26a)

Do you have any recommendations for improving the dissemination of Core Elements Technical Review site visit products?

Yes

No

Don't know

26b)

If yes, what are they?

27a)

Do you have any recommendations for more effective uses of the Core Elements Technical Review site visit products?

Yes

No

Don't know

27b)

If yes, what are they?

28)

Have you participated in State-Requested Technical Reviews? [If yes, continue. If no or cannot answer, the survey is complete.]

Yes

No

Cannot answer

29)

What is the purpose of the State-Requested Technical Review?

30)

For what issues have you conducted a State-Requested Technical Review?

31)

What have been the results of the State-Requested Technical Reviews in which you have participated?

32)

What are the strengths of the State-Requested Technical Reviews?

33)

What are the weaknesses of the State-Requested Technical Reviews?

34)

To what extent do you agree that the State-Requested Technical Review site visits are useful to States?

Strongly disagree

Somewhat disagree

Neither disagree nor agree

Somewhat agree

Strongly agree

Cannot answer

35)

Please list any ways in which you feel the State-Requested Technical Review site visits could be more useful to States.

36)

What types of training, if any, did you receive to prepare you for conducting the State-Requested Technical Review site visits? (Check all that apply)

In-person training

Written instructions

No training received

Shadowing of experienced site visitors

Other (please describe)

37a)

If you received in-person training, please rate how useful you feel the training you received was.

Not useful

A little useful

Somewhat useful

Mostly useful

Extremely useful

Cannot answer

37b)

If you received in-person training, how long before the actual State-requested site visits began did the training occur? (Check one)

Within 1 week

Within 2-3 weeks

Within 1 month

Within 2 months

Longer than 2 months

Other (please describe)

38a)

If you received written instructions, please rate how useful they were.

Not useful

A little useful

Somewhat useful

Mostly useful

Extremely useful

Cannot answer

38b)

If you received written instructions, how long before the actual State-requested site visits began did you receive them? (Check one)

Within 1 week

Within 2-3 weeks

Within 1 month

Within 2 months

Longer than 2 months

Other (please describe)

39a)

Were you provided any other materials in preparation for the State-requested site visits?

Yes

No

39b)

If yes, what other materials?

40)

Please rate how prepared the last State you visited was for the State-requested site visit.

Very unprepared

Somewhat unprepared

Minimally prepared

Very prepared

Completely prepared

Cannot answer

41)

What products resulted from the State-Requested Technical Review site visits to States? (Check all that apply)

Site visit report and recommendations

Technical assistance plan

Strategic plan

Other (please describe)

42)

How long after a State-requested site visit do you typically submit your drafts of site visit products to CSAT? (Check one)

Within 1 week

Within 2-3 weeks

Within 1 month

Within 2 months

Longer than 2 months

Other (please describe)

43)

How long after the submission of the draft State-requested technical review report to CSAT do States typically receive a copy of it? (Check one)

Within 1 week

Within 2-3 weeks

Within 1 month

Within 2 months

Longer than 2 months

Other (please describe)

44a)

Do you know if Federal program staff and grants management use State-requested site visit products?

Yes

No

Don't know

44b)

If yes, how?

45a)

Do you know how States use State-Requested Technical Review products?

Yes

No

Don't know

45b)

If yes, how?

46a)

Do you have any recommendations for improving the dissemination of State-Requested Technical Review site visit products?

Yes

No

Don't know

46b)

If yes, what are they?

47a)

Do you have any recommendations for more effective uses of the State-Requested Technical Review site visit products?

Yes

No

Don't know

47b)

If yes, what are they?