Site visit and key informant data collection: Partner survey (Appendix C)

OMB No.: 0970-0527

Expiration Date: 05/31/2022

Partner Survey

Regional Partnership Grants

National Cross-Site Evaluation

Draft

Public reporting burden for this collection of information is estimated to average 20 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number (0970-0527).

May 2021

CONTENTS

Section Page

I. INTRODUCTION 4

A. YOUR ORGANIZATION 6

B. PARTNER GOALS 10

C. PARTNERSHIP OUTPUTS 15

D. END OF SURVEY 17

Partner Survey – Regional Partnership Grants National Cross-Site Evaluation: Questionnaire and programming specifications

Draft Dated: 5/1/2021

Programming and operational assumptions:

• Modes. The survey will be administered via web only.

• Population. Representatives from partner organizations working with Regional Partnership Grant (RPG) grantees.

• Target respondent. RPG partner organization representatives.

• Length. The questionnaire is designed to take about 20 minutes to complete.

• Language. The questionnaire is available in English only.

• Administration and design specifications. Each item in the web questionnaire specifications includes: which respondents receive the item; dynamic fills, designated by text [in brackets]; emphasis text, designated in italic font; and soft and hard checks that help improve data quality (designated in boxes below applicable items). Response options shown with boxes indicate a “select all that apply” response format, whereas those shown in circles denote a “select one only” response format. The web survey will be optimized to deploy easily on mobile devices, tablets, and personal computers.

• Login. Respondents will receive an email with a direct link to the web survey.

• Critical items have soft and hard checks added throughout the instrument. Any case that starts the survey but does not finish it should be considered a “partial.” At a later date the Survey team will ask TSG to provide a list of all cases statused as partial and the last question answered. We will use this list to determine what should ultimately be designated a “complete.”

PROGRAMMER: do not display item numbers.

Questionnaire sections:

I	Introduction
A	Your Organization
B	Partner Goals
C	Partnership Outputs
D	End of Survey

WEB PROGRAMMING NOTES:

• Include section header titles, but no logo, at the top of each page within each section. The logo should appear on the introduction and end screens only. Section header IDs (I, A, B, etc.) should not display.

• Include a progress bar starting on the introduction screen.

• All items should be optimized for presentation on mobile devices.

• Next and Back buttons should appear in the same location on each page.

• Only one question/item should be presented on the same page. Section breaks should appear before each new section.

• Answers should save automatically if the respondent closes out of the survey.

• Standard web formatting rules apply (e.g., “Select one only” should not display on the screen).

Frequently Used Fills

In the boxes below, please list fills that are repeated frequently in your questionnaire requirements. These must come from a single source (whether from a preload or a question). The fills specified here do not need to be specified in the fill condition box each time they appear in a question.

Fill	Source / Condition	Used at Question #:
[GRANTEE_NAME]	Sample file	Introduction
[ORG_NAME]	Sample file Min/max for given [GRANTEE_NAME]: 3/30 (est.)	Introduction, A1, B3, B3a, C2
[RPG_NAME]	Sample file	A3-A5, B1-B5, C1-C2, D1

I. INTRODUCTION

The Regional Partnership Grant (RPG) program supports interagency collaborations and program integration designed to increase the well-being, improve the permanency, and enhance the safety of children who are in, or at risk of, out-of-home placements as a result of a parent or caretaker’s substance abuse. The Children’s Bureau within the U.S. Department of Health and Human Services, Administration for Children and Families (ACF) has contracted with Mathematica to complete the national cross-site evaluation of the program. The evaluation will describe the interventions that were implemented, the nature of the partnerships, the types of services provided, and their impacts.

You are being asked to complete this survey because you were identified as a representative of a partner organization working with the RPG grantee, [GRANTEE_NAME]. Representatives from partner organizations are asked to complete this survey to provide information about their own organizations, relationships with the grantee and other collaborating organizations, and program implementation. The length of this survey is different for different people, but on average it should take about 20 minutes.

Your participation in this survey is important and will help us understand more about the partnerships implementing RPG-funded programs. Please provide responses for your organization, [ORG_NAME]. If you represent a specific branch or program within your organization that is engaged with the RPG partnership, rather than the organization as a whole, please provide information about that branch or program rather than the organization as a whole. If you are unsure of how to answer a question, please give the best answer you can rather than leaving it blank.

Your responses will be kept private and used only for research purposes. They will be combined with the responses of other staff and reported in the aggregate; and no individual names will be reported. Participation in the survey is completely voluntary and you may choose to skip any question.

If you have any questions about the survey, please contact the team at Mathematica by emailing [email protected] or calling 866-627-9538 (toll-free).

Please read and answer the statement on the next page and then click the Next button to begin the survey.

All

I1. I have read the introduction and understand that the information I provide will be kept private and used only for research purposes. My responses will be combined with the responses of other staff and no individual names will be reported.

 I agree with the above statement and will complete the survey 1

 I do not agree with the above statement and will not complete the survey 0 GO TO END2

NO RESPONSE M

SOFT CHECK: IF I1=0; You have indicated that you will not complete the survey. Please check that this is correct and either keep your answer or change your answer below.

To keep your answer without making changes, click the Next button.

HARD CHECK: IF I1=M; Please indicate whether you agree to complete the survey and then click the Next button.

A. YOUR ORGANIZATION

PROGRAMMER: DISPLAY A_INTRO ON THE SAME PAGE AS A1.

A_INTRO. The first questions are about your organization, [PARTNER ORGANIZATION].

IF I1=1

FILL [ORG_NAME] FROM SAMPLE FILE

A1. Which of the following best describes your organization?

Select one only

 Child welfare services provider 1

 Substance abuse treatment provider 2

 Mental health services provider 3

 School district, school, or early childhood education or services provider 4

 Housing/homeless services provider 5

 Medical or dental services provider 6

 University 7

 Court/judicial agency 8

 Corrections or law enforcement agency 9

 Home visiting services provider 10

 Department in state or tribal government 11

 Department in local government 12

 Foundation 13

 Research/evaluation organization 14

 Mental health and substance abuse treatment provider 15

 Community resource organization 16

 Substance abuse prevention organization 17

 Other (specify) 99

Specify (STRING 255)

NO RESPONSE M

SOFT CHECK: IF A1=99 AND A1 Specify=EMPTY; Please specify what best describes your organization.

To continue to the next question, click the Next button below.

IF I1=1

A2. What are the main activities your organization conducts in general?

Select all that apply

 Regulation and oversight 1

 Child welfare services 2

 Substance abuse treatment 3

 Family therapy 4

 Medical or dental services 5

 Education or early childhood intervention 6

 Legal processes 7

 Law enforcement 8

 Home visiting 9

 Funding 10

 Evaluation 11

 Program planning and policy development 12

 Advocacy 13

 Case management/coordination 14

 Research 15

 Education (general) 16

 Housing 17

 Mental health services 18

 Substance abuse prevention 19

 Recovery support 20

 Other (specify) 99

Specify (STRING 60)

NO RESPONSE M

SOFT CHECK: IF A2=99 AND A2 Specify=EMPTY; Please specify the main activities your organization conducts in general.

To continue to the next question, click the Next button below.

IF I1=1

FILL [RPG_NAME] FROM SAMPLE FILE

A3. Does your organization currently provide program/other services, or does it plan to serve [RPG_NAME] clients in the future?

Select one only

 Currently provides services to [RPG_NAME] clients 1

 Plans to provide services to [RPG_NAME] clients 2

 No 3 GO TO B1

NO RESPONSE M GO TO B1

SOFT CHECK: IF A3=M; Your response to this question is important. Please provide a response and continue.

To continue to the next question, click the Next button below.

IF I1=1 AND (A3=1 OR 2)

FILL [RPG_NAME] FROM SAMPLE FILE

A4. Did your organization receive any funding for [RPG_NAME] this fiscal year?

PROGRAMMER: DO NOT DISPLAY “Select one only” ON THE SCREEN.

Select one only

• Yes 1

 No 0 GO TO A5

NO RESPONSE M

IF I1 AND (A4=1 or M)

A4a. Approximately how much funding did your organization receive this fiscal year? Your best estimate is fine.

$X,XXX,XXX

AMOUNT OF FUNDING RECEIVED

 Don’t know D

NO RESPONSE M

SOFT CHECK: IF A4a GT $500,000; You indicated that your organization received [fill A4a] for this program this fiscal year. Please check your answer and keep or change your response.

To continue to the next question, click the Next button below.

SOFT CHECK: IF A4a LT $500; You indicated that your organization received [fill A4a] for this program this fiscal year. Please check your answer and keep or change your response.

To continue to the next question, click the Next button below.

SOFT CHECK: IF A4a=M; Your response to this question is important. Please indicate how much funding your organization received for this program this fiscal year. Your best estimate is fine.

To continue to the next question, click the Next button below.

programmer box A4a:

IF VALUE GT 0 ENTERED IN TEXT BOX, DISABLE OPTION D.

If A4a=D, DISABLE TEXT BOX.

ALLOW FOR AMOUNTS GT $1,000,000 (EMPLOY SOFT CHECK for A4a GT $500,000).

IF I1=1 AND (A3=1 OR 2)

FILL [RPG_NAME] from SAMPLE FILE

A5. Which of the following in-kind resources is your organization contributing to [RPG_NAME] this fiscal year?

Select all that apply

 Staff time 1

 Office space 2

 Volunteers 3

 Office supplies 4

 Program materials 5

 Computer/Internet, telephone, or fax service 6

 Information 7

 Transportation 8

 Other (specify) 99

Specify (STRING 150)

 None of these 97

NO RESPONSE M

SOFT CHECK: IF A5=99 AND A5 Specify=EMPTY; Please specify the in-kind resources your organization is contributing to the program this fiscal year.

To continue to the next question, click the Next button below.

programmer box A5:

If A5=97, DISABLE OPTIONS 1-8 and 99.

B. PARTNER GOALS

IF I1=1

FILL [RPG_NAME] FROM SAMPLE FILE

PROGRAMMER: IN B1, “(LIMIT: 1,000 CHARACTERS)” SHOULD DISPLAY ON THE SCREEN.

PROGRAMMER: ADD CHARACTERS REMAINING INDICATOR TO TEXT BOX.

B1. In your own words, what are the main goals of the [RPG_NAME] partnership? (Limit: 1,000 characters)

(STRING 1000)

NO RESPONSE M

SOFT CHECK: IF B1=M; Your response to this question is important. Please indicate in your own words the main goals of the partnership.

To continue to the next question, click the Next button below.

IF I1=1

FILL [RPG_NAME] FROM SAMPLE FILE

B2. Do you currently serve on a steering, implementation, governance, or some other committee for the [RPG_NAME] grant?

PROGRAMMER: DO NOT DISPLAY “Select one only” ON THE SCREEN.

Select one only

 Yes 1

 No 0

NO RESPONSE M

IF I1=1

FILL ALL [ORG_NAME] FROM SAMPLE FILE FOR GIVEN GRANTEE_NAME

FILL [RPG_NAME] FROM SAMPLE FILE

B3. Other than formal [RPG_NAME] partnership meetings that you may attend, how frequently does your organization communicate about [RPG_NAME] with other organizations?

Please indicate if you do not communicate at all, if you communicate infrequently (a few times each month), or if you communicate regularly (every day or nearly every day) with each given partner. Please choose the answer that best represents the frequency of communication.

PROGRAMMER: DISPLAY ONE [ORG_NAME] VALUE PER PAGE WITH RESPONSE OPTIONS AS ROWS. DO NOT DISPLAY ROW FOR GIVEN RESPONDENT’S [ORG_NAME].

Select one only

PROGRAMMER: CODE ONE PER ROW
	We do not communicate at all	We communicate infrequently (a few times each month)	We communicate regularly (every day or nearly every day)	NO RESPONSE
FILL [ORG_NAME]	1 	2 	3 	M 
	1 	2 	3 	M 
	1 	2 	3 	M 
	1 	2 	3 	M 
	1 	2 	3 	M 
	1 	2 	3 	M 
	1 	2 	3 	M 

IF I1=1

FILL ALL [ORG_NAME] FROM SAMPLE FILE FOR GIVEN GRANTEE_NAME

FILL [RPG_NAME] FROM SAMPLE FILE

B3a. Among the organizations listed below, please select the ones whose members you were working with as part of the [RPG_NAME] partnership prior to the beginning of the [RPG_NAME] grant in 2017.

PROGRAMMER: DO NOT DISPLAY GIVEN RESPONDENT’S [ORG_NAME] AS A RESPONSE OPTION. Select all that apply

 [ORG_NAME] 1

NO RESPONSE M

IF I1=1

FILL [RPG_NAME] from SAMPLE FILE

B4. To what extent do you disagree or agree with each of the following statements about the current status of the collaboration among [RPG_NAME] partner organizations?

PROGRAMMER: DISPLAY ITEMS B4a-u ONE PER PAGE WITH RESPONSE OPTIONS AS ROWS.

Select one only

PROGRAMMER: CODE ONE PER ROW	Strongly Disagree	Disagree	Agree	Strongly Agree	NO RESPONSE
a. Our collaborative effort was started because we wanted to do something about an important problem.	1 	2 	3 	4 	M 
b. Our [RPG_NAME] program’s top priority was having a concrete impact on the real problem.	1 	2 	3 	4 	M 
c. The organizations involved in our [RPG_NAME] program included those organizations affected by the issue.	1 	2 	3 	4 	M 
d. Participation was not dominated by any one group or sector.	1 	2 	3 	4 	M 
e. Our partner organizations have access to credible information that supports problem solving and decision making.	1 	2 	3 	4 	M 
f. [RPG_NAME] partner organizations agree on what decisions will be made by the group.	1 	2 	3 	4 	M 
g. Partner organizations agree to work together on this issue.	1 	2 	3 	4 	M 
h. Organizations involved in our [RPG_NAME] program have set ground rules and norms about how we will work.	1 	2 	3 	4 	M 
i. We have a method for communicating the activities and decisions of the group to all partner organizations.	1 	2 	3 	4 	M 
j. There are clearly defined roles for [RPG_NAME] partner organizations.	1 	2 	3 	4 	M 
k. Partner organizations are more interested in getting a good decision for the [RPG_NAME] program than improving the position of their own organization.	1 	2 	3 	4 	M 
l. Staff who participate in [RPG_NAME] program meetings are effective liaisons between their home organizations and the group.	1 	2 	3 	4 	M 
m. Partner organizations trust each other sufficiently to honestly and accurately share information, perceptions, and feedback.	1 	2 	3 	4 	M 
n. Partner organizations are willing to let go of an idea for one that appears to have more merit.	1 	2 	3 	4 	M 
o. Partner organizations are willing to devote whatever effort is necessary to achieve the goals.	1 	2 	3 	4 	M 
p. Divergent opinions are expressed and listened to.	1 	2 	3 	4 	M 
q. The openness and credibility of the process helps partner organizations set aside doubts and skepticism.	1 	2 	3 	4 	M 
r. Our group sets aside vested interests to achieve our common goal.	1 	2 	3 	4 	M 
s. Our group has an effective decision-making process.	1 	2 	3 	4 	M 
t. Our group is effective in obtaining the resources it needs to accomplish its objectives.	1 	2 	3 	4 	M 
u. The time and effort of the collaboration is directed at achieving our goals rather than keeping the collaboration in business.	1 	2 	3 	4 	M 

IF I1=1

FILL [RPG_NAME] FROM SAMPLE FILE

B5. Indicate the degree to which you disagree or agree with each of the following statements about [RPG NAME] programming.

PROGRAMMER: DISPLAY ITEMS B5a-f ONE PER PAGE WITH RESPONSE OPTIONS AS ROWS.

Select one only

PROGRAMMER: CODE ONE PER ROW	Strongly Disagree	Disagree	Agree	Strongly Agree	Does not apply/Don’t know	NO RESPONSE
a. We developed strategies to recruit community participation.	1 	2 	3 	4 	98 	M 
b. Community members are included in program planning and development.	1 	2 	3 	4 	98 	M 
c. We developed formal mechanisms to solicit support and input from community members and consumers.	1 	2 	3 	4 	98 	M 
d. Frontline staff have up-to-date resource directories for family support centers and resources.	1 	2 	3 	4 	98 	M 
e. Community-wide accountability systems are used to monitor substance abuse and child welfare issues.	1 	2 	3 	4 	98 	M 
f. Consumers, patients in recovery, and program graduates have active roles in planning, developing, implementing, and monitoring services.	1 	2 	3 	4 	98 	M 

c. partnership outputs

PROGRAMMER: DISPLAY C_INTRO ON THE SAME PAGE AS C1.

C_INTRO. The next questions are about the outputs or products of the [RPG_NAME] partnership of which your organization is a part.

IF I1=1

FILL [RPG_NAME] FROM SAMPLE FILE

PROGRAMMER: DISPLAY ITEMS C1a-l ONE PER PAGE WITH RESPONSE OPTIONS AS ROWS.

C1. Indicate the degree to which you disagree or agree with each of the following statements about [RPG_NAME] programming.

Select one only

PROGRAMMER: CODE ONE PER ROW	Strongly Disagree	Disagree	Agree	Strongly Agree	Does not apply/Don’t know	NO RESPONSE
a. Services provided to families are coordinated across multiple partners.	1 	2 	3 	4 	98 	M 
b. Case management is coordinated across both substance abuse treatment providers and child welfare agencies.	1 	2 	3 	4 	98 	M 
c. Families receiving joint case management receive regular cross-agency assessments.	1 	2 	3 	4 	98 	M 
d. Staff from both substance abuse treatment providers and child welfare agencies participate in joint case management activities, such as family team conferences, case plan reviews, or intake or permanency staffings.	1 	2 	3 	4 	98 	M 
e. Judicial officers and attorneys are viewed as partners in developing new approaches to serve families with substance use disorders in the child welfare system.	1 	2 	3 	4 	98 	M 
f. Substance abuse and child welfare agencies and the courts have negotiated shared principles or goal statements.	1 	2 	3 	4 	98 	M 
g. The region/partnership developed responses to conflicting time frames associated with child welfare services, substance abuse treatment, Temporary Assistance for Needy Families, and child development.	1 	2 	3 	4 	98 	M 
h. Substance abuse treatment and child protective service case plans are coordinated.	1 	2 	3 	4 	98 	M 
i. Formal working agreements have been developed on how courts, child welfare, and treatment agencies will share client information.	1 	2 	3 	4 	98 	M 
j. Data tracking of child welfare and substance abuse clients across systems is done to monitor outcomes.	1 	2 	3 	4 	98 	M 
k. Substance abuse agencies, child welfare agencies, and court systems have developed shared outcomes for families and agree on how to use information on outcomes with families.	1 	2 	3 	4 	98 	M 
l. Joint training programs for the three main systems staff have been developed to help staff and providers work together effectively.	1 	2 	3 	4 	98 	M 

IF I1=1

FILL ALL [ORG_NAME] FROM SAMPLE FILE FOR GIVEN GRANTEE_NAME

FILL [RPG_NAME] FROM SAMPLE FILE

C2. Which of the following [RPG_NAME]-related services does your organization coordinate with or collaborate on with each given organization?

PROGRAMMER: DISPLAY ONE [ORG_NAME] VALUE PER PAGE WITH RESPONSE OPTIONS AS ROWS. DO NOT DISPLAY ROW FOR GIVEN RESPONDENT’S [ORG_NAME].

Select all that apply

PROGRAMMER: CODE ALL THAT APPLY	Screening and/or Assessment	RPG Program Referrals	Case Management or Coordination	Substance Abuse Treatment	Mental Health / Trauma Services	Other Social or Family Services	We do not collaborate with this organization on any of these services	NO RESPONSE
FILL [ORG_NAME]	1 	2 	3 	4 	5 	6 	97 	M 
	1 	2 	3 	4 	5 	6 	97 	M 
	1 	2 	3 	4 	5 	6 	97 	M 
	1 	2 	3 	4 	5 	6 	97 	M 
	1 	2 	3 	4 	5 	6 	97 	M 
	1 	2 	3 	4 	5 	6 	97 	M 
	1 	2 	3 	4 	5 	6 	97 	M 
	1 	2 	3 	4 	5 	6 	97 	M 
	1 	2 	3 	4 	5 	6 	97 	M 

programmer box C2:

If [ORG_NAME]=97 FOR A GIVEN ROW, DISABLE COLUMNS/OPTIONS 1-6.

D. END OF SURVEY

IF I1=1

FILL [RPG_NAME] FROM SAMPLE FILE

PROGRAMMER: IN D1, THE TEXT “(LIMIT: 1,000 CHARACTERS)” SHOULD DISPLAY ON THE SCREEN.

PROGRAMMER: ADD CHARACTERS REMAINING INDICATOR TO TEXT BOX.

D1. Thank you for your participation in this survey. If there is anything else that you would like to tell us about your work on the [RPG_NAME] program or about the partnership as a whole, please share it here. (Limit: 1,000 characters)

(STRING 1000)

NO RESPONSE M

IF I1=1

END1. Thank you for completing the Regional Partnership Grant Partner Survey! Please click the Submit button to submit your completed survey. Note: You will not be able to make any changes after you click Submit.

PROGRAMMER: REDIRECT RESPONDENT TO MATHEMATICA HOMEPAGE.

IF I1=0

END2. Thank you for considering participation in this survey. Please click the Submit button so that we have a record of your desire not to participate. This will result in your removal from our contact list.

PROGRAMMER: REDIRECT RESPONDENT TO MATHEMATICA HOMEPAGE.