Instrument 3: Interview Guide

April: Interviews

State Child Welfare Data Linkages Descriptive Study
Interview Guide

Draft – Do Not Share

April 2022

THE PAPERWORK REDUCTION ACT OF 1995 (Pub. L. 104-13): Public reporting burden for this collection of information is estimated to average one hour per response, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB number for this information collection is XXXX-0XXX and the expiration date is XX/XX/20XX. If you have any comments on this collection of information, including estimated time to complete, please contact Dr. M.C. Bradley at Mathematica at (XXX) XXX-XXXX or by email at: [email protected].

Phone Interview Introduction and Consent Statement

Thank you for joining us in this interview. My name is __________ and I am joined by my colleague, ______. We work for Mathematica, an independent social policy research firm. We are independent researchers working on behalf of the U.S. Department of Health and Human Services, Office of Planning, Research and Evaluation in collaboration with the Children’s Bureau. We are conducting a study to learn more about states’ child welfare administrative data and practices related to sharing and linking those data with data from other sources. The study aims to inform the ongoing and accurate surveillance of child maltreatment and to identify opportunities and challenges related to linking data.

As part of our study, we conducted two surveys in 2022 to collect information from states about child welfare data systems, data sharing agreements, data exchanges, and linked data sets. We would like to talk with you to learn more about connected data efforts (integrated and linked data) in [YOUR STATE], including your experience and perspective about what has helped you share and link data, and what has been a challenge. Our team will use your responses, along with information we have collected from the surveys, to describe national themes. Your state or your role may be identified in public reports of study findings. However, public reports will not identify you by name. We may also archive a summary of responses from everyone we interview in [YOUR STATE] for restricted use by other researchers. Archived data may identify states and counties but will not include any names of individuals or organizations. It is up to you whether you want to be part of this interview, and you can choose not to answer a particular question or stop the interview any time you want to. The interview will take about an hour.

We would like your consent to audio record our interview so we can listen to it later when we write up our notes. No one outside our research team will listen to the recording or have access to raw notes.

If you want to say something that you don’t want recorded, please let us know, and we will pause the recorder.

Do you have questions about the study?

If you think of questions about the study after the interview or have questions about your rights as a research participant, please contact the project director, M.C. (Cay) Bradley, at [use 800#].

Before we can start, I need to confirm your participation by asking you two questions:

[Consent Question 1] Do you agree to participate in the study? [yes/no]

[If yes, Consent Question 2] Do you agree to the audio recording of the interview? [yes/no]

Protocol for the Follow-Up Interview

Instructions for interviewers

The purpose of the interview is to collect additional information about the connected data efforts in the particular state/county, and to ask for more details about the barriers and facilitators that impact connected data efforts in the state/county. The interviewer should review the survey responses for the state/county as part of preparing for the interview. Do not ask questions that the survey already provided information for, but do use the survey responses as a foundation for questions when appropriate.

Below, we have organized topics into sections with suggested questions. You do not need to ask the questions verbatim. If the respondent indicates there is relevant written documentation, encourage them to send it to you and send a follow-up email reminding them of their offer to share information.

Before you begin the interview, read the verbal consent script to describe the study to respondents, answer any questions they have about the study, and ask for their consent to participate in the study and to audio record the interview.

A. Roles, responsibilities, and experience

1. Can you please tell us about your position, role, and responsibilities with [employer name]?

2. How long have you been in this role? (If less than two years:) What role did you have before this?)

B. Key characteristics of state system

[This section is specific to the child welfare director or designee. Remaining sections can be asked of all respondents.]

3. [See Questions 5–7 of the Initial Survey for state’s responses about their system of record for child maltreatment data.] How old is [SYSTEM]? If you are currently transitioning to a new system, what is the status of the transition? Are you planning to make significant changes or updates to it over the next five years?

If there are plans to change/update:

• What implications does this change have for obtaining new data or data linking plans? (Will the updates/new system have an impact on plans to obtain new data or link child welfare data with other data?)

• Are there multiple systems used to collect child welfare data throughout the state?

If yes:

• Can you describe how those systems interact with each other? If the state has a county-administered or hybrid child welfare system: Does each county develop and administer its own system?

4. How are the child maltreatment data compiled across the state (or from child welfare contributing agencies) for the National Child Abuse and Neglect Data System (NCANDS) and other statewide reporting purposes?

5. If the state indicated in Q2 of the Initial Survey that it has a child welfare division within a larger agency: Does your child welfare division need data sharing agreements to share child maltreatment data with some or all other divisions within the same agency?

6. If child welfare contributing agencies were indicated in Q4 of the Initial Survey: What data do child welfare contributing agencies [CWCAs] provide to the state? How do they provide the data? Please describe how data from CWCAs are processed by the state and incorporated with the state’s other child welfare data.

• How does the state get the data? Are they entered directly into [SYSTEM]?

  • If not, what data format and process does the state use to integrate the data? How often are the data integrated?

• Is there a statewide process or county-level or CWCA-specific differences in how data are received?

• Are there any challenges with the process or data quality, and/or with relying on the data for analytic purposes? (for example, to better understand child maltreatment incidence)

• Are there any challenges related to what demographic data is collected by different entities, and variations in their definitions, quality, and completeness?

7. If there are data systems connected to the legacy system/SACWIS/CCWIS/CMS:

Can you describe what led to the [linking/integration] of [data systems identified in the Survey of Connected Data]?

• What were the goals?

• How did your state arrive at the method for [linking/integration]?

• How are the data used? Who uses the data? Are they used by case managers? If so, how? Are there analytic data sets that encompass child maltreatment data and [linked/integrated data] used for research or evaluation? Are these data sets used for continuous quality improvement?

• What challenges did you encounter to establish these [linked/integrated datasets]?

• Were there any challenges related to what demographic data is collected by different entities, and variations in their definitions, quality, and completeness?

• Do the connected (integrated or linked) data include information about child or parent health and functioning? (Physical health including health services and immunizations? Mental/behavioral health? Substance use? Developmental milestones? Academic performance?)

C. Data sharing and partnerships

8. Please provide details about any data sharing agreements in place.

• I’d like to confirm you have data sharing agreements with the following organizations to make sure we are not missing any, and then ask some questions. The list includes: [list the organizations identified in the Survey of Connected Data].

• Are there any data sharing agreements with other organizations? For any data sharing agreements not previously identified:

  • When were these data sharing agreements established?

  • What organizations are involved in the data sharing agreements?

  • What data does each organization provide?

  • Do the connected (integrated or linked) data include information about child or parent health and functioning? (Physical health including health services and immunizations? Mental/behavioral health? Substance use? Developmental milestones? Academic performance?)

  • For what purpose(s)?

  • Does this agreement relate to any linked data sets that include your child maltreatment data?

• Describe the process(es) followed to get the data sharing agreements established. If you don’t know the details for a particular agreement, just let me know that.
  
  
  

  • What factors were instrumental in initiating and finalizing agreements?

  • Were there specific policies and/or resources that were instrumental?

  • Are there state laws that influence your ability to establish data sharing agreements, such as laws that prohibit or regulate data sharing across agencies in the state?

  • Does your agency need to identify legal authority to share records?

  • What was the resource investment to make this happen? (About how much time? How many staff? Were there dedicated staff?)

  • Any other challenges?

  • Were there challenges related to differences in demographic data collected by different entities, and variations in their definitions, quality, and completeness?

  • How did your state/county overcome these challenges?

9. For states that do not have linked data sets, ask questions to get details on the goals, challenges, and barriers involved in achieving data sharing agreements and data linkages. Skip to closing remarks after this topic (only for states that do not have linked data sets):

• Are you aware of any data sharing agreements that your agency tried to establish but ultimately didn’t complete or get off the ground, or that are no longer active? If yes:

  • Why were these efforts unsuccessful/What were the specific challenges/Why are they no longer active?

  • Does your agency need to identify legal authority to share records?

  • When were data sharing agreements attempted, or when were they in place?

  • What were the goals for the data sharing agreements?

  • Were all goals met? If not, what were the challenges or barriers to completing or maintaining the agreements?

  • Were there specific polices and/or resource factors that hindered your efforts? Are there any state laws that prohibit or regulate data sharing agreements? If so, were these a factor?

• What other factors may have changed the outcome of unsuccessful efforts? Are you aware of any data sets for which there was an attempt to link to your state’s child maltreatment data, but it didn’t work out, or any data sets that were linked, but aren’t any more? If yes:

  • When were those attempts made?

  • What organization did you try to link data with?

  • What are/were the goals for the linked data?

  • Why was this unsuccessful/why are the agreements no longer active?

    • When did any of these earlier agreements end?

  • What were the challenges or barriers to establishing or maintaining links between your child maltreatment data and other data sets?

  • Were there specific polices and/or resource factors that kept these agreements from moving forward? Are there any state laws that prohibit or regulate collecting/using data elements for data linking? Were these a factor?

10. For states that have linked data sets (Q.9–17):

Please tell us about any data sharing agreements that are not in place anymore, and/or any attempts to obtain data sharing agreements. (probe for details)

• What were the goals of these agreements?

• Why did those agreements end, and/or did you start or try to start data sharing agreements that were never completed?

11. When did your state begin linking your child maltreatment data? (How long have the linked data been in place?)

12. How are linked data set(s) managed and stored? Within the child welfare agency, within another state agency, or external to the state? (If there are multiple linked data sets, cycle through these questions as time permits.)

• If external to the state, what organization manages the linked data?

• Is it a university, research firm, or another kind of organization?

• What type of agreement governs the sharing and use of the data? (Data use agreement, Business Associates Agreement, other?)

• Who does the linking?

• How often are the data linked?

• What resources have been necessary to maintain linked data?

  • Internal or external funding? Staff expertise/skills? Technical capacity?

  • If funding is mentioned: What are the funding sources?

• What kinds of things have helped you obtain the resources required to maintain the linked data?

13. Do you have a written version of the methods or procedures for linking the data and accessing them? Can you describe the methods or procedures for linking the data?

• How are the data linked? (that is, what are your methods?)

  • What specific program(s) are you using? (for example, LinkPlus, Choicemaker, R)

  • What data elements are used for data linking?

• Where are the data hosted during the linkage process?

D. What are the connected (integrated or linked) data used for?

14. Practice?

• How so? (to assist with casework/case management, service delivery, other?)

• What factors are instrumental in using these linked data for practice? Are there specific policies and/or resources that are instrumental?

• Are there plans to use the linked data in a different way for practice?

• What are challenges to using the linked data for practice? (Data quality issues? Other?) How is your state/county addressing them?

15. Research?

• What types of research questions? Descriptive purposes and/or evaluation?

• State policy analysis research?

• Program evaluation research? To build evidence for a clearinghouse?

• Who does this analysis?

• What factors are instrumental in using these linked data for research? Are there specific policies and/or resources that are instrumental?

• Are there additional research projects planned with these linked data?

• What are challenges to using the linked data for research? (Access issues? Other?)

16. Performance monitoring?

• In what ways? (to examine access to or consistency of services, required reporting, other?)

• Government performance monitoring?

• Contracted agency performance monitoring?

• What factors are instrumental in using these linked data for performance monitoring? Are there specific policies and/or resources that are instrumental?

• What are challenges to using the linked data for performance monitoring?

17. Identifying differences across demographic groups to advance equity? (And if so, does this demographic information come from [CW AGENCY] or elsewhere?)

18. Other internal/external purposes? (Check that Q14–Q16 have covered the uses of any connected data mentioned in Q7–Q8 that include information about child or parent health and functioning, including physical health, mental/behavioral health, developmental milestones, and academic performance)

19. Are there any other challenges or barriers to using the linked data? (Data quality issues? Access issues? Other issues?)

20. Thinking about all of the administrative data available to your agency—including data that are not linked—are there opportunities for your state to more fully utilize these data for practice, research, or performance monitoring?

- How could the state more fully utilize these data?

- What are anticipated facilitators to those opportunities?

- What are anticipated barriers to those opportunities?

E. Closing remarks and follow-up

• Thank you so much for your time today. Your input will help us better understand states’ experiences sharing and linking child welfare administrative data.

• We also know that after a conversation like this, there’s often a moment of, “Oh, I forgot to mention …” As a follow-up to this conversation, we’re going to send an email [for any follow-up items discussed during the call] and ask you to share any other thoughts or information that might have occurred to you since we spoke. We’d appreciate it if you could share any follow-up thoughts in the next two weeks.

DRAFT – DO NOT SHARE 1