Instrument 1: Master Site Visit Interview Protocol

virtual SITE VISIT INVITATION

T o: [DIRECTOR OF STATE OR LOCAL APPROACH]

Subject: We would like to learn more about [STATE OR LOCAL APPROACH NAME]!

Dear [DIRECTOR OF STATE OR LOCAL APPROACH]:

[IF STATE OR LOCAL APPROACH PARTICIPATED IN TELEPHONE INTERVIEW] Thank you again for helping us learn more about [STATE OR LOCAL APPROACH NAME] by participating in a telephone interview about your coordinated services approach. As part of the Assessing Models of Coordinated Services for Low-Income Children and Their Families (AMCS) study, we are following up to invite you and other staff from [STATE OR LOCAL APPROACH NAME] to participate in a series of follow up interviews (“virtual site visit”) so we can learn more about your coordinated services approach.

[IF STATE OR LOCAL APPROACH WAS NOT INVITED TO PARTICIPATE IN TELEPHONE INTERVIEW]. We are interested in learning more about [STATE OR LOCAL APPROACH NAME] as part of an important project that the Administration for Children and Families (ACF) within the U.S. Department of Health and Human Services is conducting, called Assessing Models of Coordinated Services for Low-Income Children and Their Families. ACF has partnered with Mathematica to conduct this study. The goal of this study is to improve our understanding of approaches to coordinate early care and education with family economic security and/or other health and human services. We are would like to invite you and other staff from [STATE OR LOCAL APPROACH NAME] to participate in a series of interviews (“virtual site visit”) so we can learn more about your coordinated services approach.

[ALL APPROACHES] We would like to talk with you and other staff at [STATE OR LOCAL APPROACH NAME] about your experiences at [STATE OR LOCAL APPROACH NAME] in order to understand how coordinated services approaches operate, including topics such as partnerships, strategies for service delivery, staffing, and data sharing. If you would like to participate in this series of interviews (“virtual site visit”), we ask that you identify a few dates across which we could schedule interviews. We are happy to discuss an approach to selecting staff to participate if that would be helpful in selecting dates they are available.

Please let us know your, and your colleagues, interest and availability to participate in a series of interviews (“virtual site visit”). We will call you in the next week to answer any questions you have and discuss this opportunity. In the meantime, please contact me by email [EMAIL] or phone [PHONE] if you have questions or would like to discuss the study.

Sincerely,

[NAME]

Paperwork Reduction Act Statement: The referenced collection of information is voluntary. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB control number for this collection is 0970-0535 and it expires 10/31/2020.

virtual SITE VISIT follow-up phone script

Protocol for virtual SITE VISIT invitation

Interviewer instructions: The goal of this telephone call is to follow-up on an email request to invite staff from state and local coordinated services approaches to participate in a series of follow up interviews (“virtual site visit”). Please ask to speak to the director of the coordinated services approach (or someone in the equivalent role). If that person is not available, please use the information below to briefly describe the study and the request and ask if there is anyone else to speak to or if the director of the coordinated services approach could return your call. Once you are speaking to the director, or other person who is able to address the request, please follow the protocol below.

Telephone protocol:

INTRODUCTION

Hello, my name is ________ and I am from an organization called Mathematica. I am calling about a study called Assessing Models of Coordinated Services for Low-Income Children and Their Families. I’m following up on an email I sent last week inviting you to participate in a series of follow up interviews (“virtual site visit”) to learn more about your approach to coordinating services for children and their families. Did you receive that email?

[IF NO]. I am sorry to hear you did not receive the email. I can tell you about the opportunity briefly now. Do you have about five minutes to talk now?

[If YES] I’d like to tell you a little more about the series of follow up interviews (“virtual site visit”) and see if you are interested in participating. Do you have five minutes to talk now?

[IF YES, OK TIME] Great. Thank you. [PROCEED WITH “ABOUT THE STUDY AND VIRTUAL SITE VISIT”]

[IF NO, NOT OK TIME] No problem, is there a better time for me to reach you? [IF YES, PLEASE CONFIRM ALTERNATE TIME]. I will call you back then. Thank you very much. [IF NOT INTERESTED AT ALL/REFUSES, SKIP TO “IF DECLINED” AT END]

ABOUT THE STUDY AND VIRTUAL SITE VISIT

The Assessing Models of Coordinated Services study is being conducted for the Administration for Children and Families in the U.S. Department of Health and Human Services. The goal of the study is to learn more about how states and communities are coordinating services to serve children and their families. We would like to talk with you and other staff at [STATE OR LOCAL APPROACH NAME] about your experiences at [STATE OR LOCAL APPROACH NAME] in order to understand how coordinated services approaches operate, including topics such as partnerships, strategies for service delivery, staffing, and data sharing. We will use this information to help inform others in the field about how coordinated services can best support families. We will conduct a series of individual or small group interviews (6-8 interviews) via audio/video conferencing, and will work with you to schedule interviews with key staff who support the operation of the coordinated services approach. The interviews will range in length, up to a maximum of two hours each, and we can be flexible in scheduling them to meet your availability. We would like to schedule them over the course of two weeks. If you would like to participate, we would ask you to identify a few weeks that we could schedule interviews. We would like to speak to various types of staff (leaders, services providers) who could provide information about the coordination process, from both the development of the coordinated services approach to the current operations, and in light of COVID-19, would also like to ask some questions to understand how your work may have been and/or continues to be affected. We would also like to conduct a one-hour virtual focus group with parents to understand their experience receiving services.

SCHEDULING THE VIRTUAL SITE VISIT

We would like to schedule the series of interviews between [DATE] and [DATE] [REFER TO TIME PERIOD]. When would be a good time for you and your staff? We can follow-up after this call by email if that is best.

CLOSING

[IF AGREED] Thanks so much, I look forward to scheduling a time to speak with you and your colleagues.

[IF DECLINED]. Thank you for speaking with me. We have enjoyed finding out about your coordinated services approach and we wish you continued success.

NOTE: Not all questions will be asked of all respondents. This is a master protocol. There are two ways in which questions will be selected: (1) by respondent: after we learn which staff are employed by the site, we will tailor this protocol to only ask the respondent questions in the sections that they would be knowledgeable about; and (2) italicized questions represent information that may end up being obtained prior to a particular staff interview. This could occur a number of ways, such as through non-burden project tasks such as the literature synthesis or national scan, or interviews with other staff at the same particular site. If the information has already been collected, the italicized question will not be asked.

AMCS Master VIRTUAL SITE VISIT Interview Protocol

Thank you for your willingness to speak with me today about [State/Local Approach name]. My name is [Interviewer 1], and I work for Mathematica. My colleague, [Interviewer 2] has also joined us today.

Let me provide some background. As we’ve mentioned in previous communication, the Office of Planning, Research and Evaluation at the Administration for Children and Families within the U.S. Department of Health and Human Services, contracted with us to study how states and communities coordinate different types of services for children and families. When we talk about coordinating or integrating services, we mean providing families with high quality early care and education, and simultaneously addressing additional family needs, like income supports, health and mental health services for children and their families, job search and training, and other types of health and human services. In the fall, we conducted a nationwide scan of coordinated services approaches in operation, which is how we came to learn about [State/Local approach]. A few months ago, we sent a profile with information we gathered on [State/local approach] [IF TELEPHONE INTERVIEW COMPLETED] and more recently we conducted a telephone interview with several of your colleagues. We appreciate you participating in these activities to help us learn more about your work. Today, we would like to ask some additional questions so that we can learn more about your work.

We are not evaluating [State/local approach]. Instead, we are gathering information across several state and local approaches, to better understand how coordination looks in different settings. I expect our conversation will take up to two hours. We have a lot to cover in our discussion today so I may have to move our conversation along at times.

Your participation in this interview is voluntary, and what you say today will be considered private to the extent provided by law. Information will be summarized across interviews and your name(s) will not be associated with your comments in any reports. The reports might list and describe coordinated services approaches that contributed information, but we will not quote you by name or title. You may refuse to answer any question and may stop the interview at any time. If it is ok with you, we will take notes over the course of the interview so that we can remember the information we collect. Do you mind if I also record the conversation? It will help us ensure we accurately capture our discussion today. [If yes, start recording.] Our notes and the recording will be stored on a secure drive, and destroyed when this project is over.

An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid Office of Management and Budget control number. The Office of Management and Budget number for this information collection is 0970-0535 and the expiration date is 10/31/2020.

Do you have any other questions? If not, then I’ll get started.

We are primarily interested in the usual way that [State/local approach] operates. However, we recognize that COVID-19 has likely affected your work. Throughout the interview, we will ask you about the influence of COVID-19 on particular aspects of your work, but otherwise encourage you to share how [State/local approach] operated prior to COVID-19.

[Interviewer Note: Review the state/local model profile and telephone interview data. Italicized questions request information about topics that might be collected across the literature synthesis, national scan and model profile confirmation and completion, and telephone interviews. These questions should only be asked if this particular information has not already been collected. Questions and topics that are prioritized for this interview are bolded]

A. Respondent roles and responsibilities

1. What is your official job title or position?

2. What are your roles and responsibilities in your job?

3. How long have you been in your current role? Have you held other positions in [Program name]?

B. Target population

Next I’d like to talk about the population of families and children you serve.

1. Based on our background research and initial telephone conversation, I understand that [Program name] targets [description of target population]. Is that right?

   • Has this changed as a result of COVID-19?

2. About how many families do you serve? Has this number changed over time? How and why has [Program name]’s capacity changed? [PROBE: Has it changed due to resources or funding? Demand? Eligibility criteria? Changes at the state level? Changes due to COVID-19? Etc.]

3. Why was the particular target population selected?

   • Has this changed as a result of COVID-19?

4. Do you find you are, in fact, serving the population you target? Do you have trouble identifying or finding people to serve or does [Program name] have to expand to meet family needs? Is there a waiting list for your services?

   • Has this changed as a result of COVID-19?

5. Do participants in [Program name] receive services from other programs in the community? If so, what services do they receive? How did that influence what services [Program name] provides or coordinates?

   • Has this changed as a result of COVID-19?

6. What are the primary needs of the people you serve?

   • Has this changed as a result of COVID-19?

7. To what extent is [Program name] meeting the needs of parents, children, and families? Are there services you do not provide that families have trouble finding?

   • Has this changed as a result of COVID-19?

8. Have you considered expanding and making [Program name]’s services available to a wider population, or serving a different group? [PROBE: For example, if program focuses on single mothers has the program considered serving fathers; or has the program considered serving Spanish-speakers, etc.]

   • Has this changed as a result of COVID-19 (for example, children and families of first responders, health care workers, essential services workers)?

C. Services

Now I’d like to learn about the services [Program name] provides to families and children.

1. Our understanding is that [Program name] provides [X], [Y], and [Z] services. Is that right? Are there other services you provide? What are the main services are available?

   • Are some services in higher demand than others? Has this changed as a result of COVID-19?

2. Can you walk us through a typical family’s progression through [Program name]?

• Do they receive services in a particular order or based on needs? Do they receive multiple services at the same time or one at a time?

• How are service needs determined? Do participants go through screenings? If so, what screening tools do you use? How do [Program name] staff use screenings to inform service provision? Do you use particular curricula or methods? Do you use research evidence or data to guide your choices?

• Where are the services located? Are services for parents and children located in the same place/building?

• What is the schedule for offering services? Are services that are offered for parents available during the same time of the day as services that are offered for children?

• Do parents and children participate in any services together?

• Do different program or partner staff work with families at different points in program participation? Or do staff work with families all throughout their time in [Program name]?

• Has this changed as a result of COVID-19?

3. On average, how long are families enrolled in [Program name]?

   • Has this changed as a result of COVID-19?

4. I’d like to learn more about how [Program name] and its partners work together to provide services for families. Could you tell me a little about how coordinated these efforts are? How closely do you work with your partner agencies in providing services? How do you partner with state/local partners for service delivery?

• There are many ways to coordinate. Some programs use referrals, other programs share information and data, and other programs provide services that are fully integrated to serve multiple family needs and work with both children and adults. Where would you say your organization is within the range of possible ways to coordinate?

• Are you moving toward more or less coordination or do you feel like you are already working together in the way you prefer?

  • [If not where they would like to be] Is there anything that prevents you from collaborating more intensively? Have you worked with the state to change these barriers?

• Has this changed as a result of COVID-19?

5. [If respondent indicates program is coordinated or collaborating] I’d like to ask about how [Program name] and its partners work together to provide a coordinated set of services to families. Who leads or coordinates cross-agency or organizational work? How do you work with state/local partners to coordinate services and make decisions?

• You mentioned [Program name] provides [X], [Y], and [Z] services. What agency or organization delivers each service?

• How did [Program name] decide which organizations would offer which services?

• What is the process for decision making among all the partners involved in [Program name]? Who is involved in decision-making? Do they meet regularly? What are they responsible for? How are decisions made? How are they made when decision-makers disagree?

• Is there anyone you wish were involved in decision making? Do you have plans to involve them soon? How will you engage them?

• Does [Program name] have an advisory or policy council? Who is on the council and what is the process for joining the council? What is the council responsible for? Is the council involved in decision-making?

• Has this changed as a result of COVID-19?

6. How do staff providing different services communicate? How does information flow between the partners? How do you communicate with state/local partners?

   • Has this changed as a result of COVID-19?

7. How do staff plan for and facilitate transitions for families and children as they transfer from one service to another? Are there supports to help families transition from early care and education into kindergarten? Do staff talk with families about the transition ahead of time? Do program staff create a plan for the transition? Is there a hand-off or a referral? How do families transition from the program? Is there a plan?

   • Has this changed as a result of COVID-19?

8. What services are available through referral?

• Are the referral agencies formal program partners or regularly available community resources?

• To what extent do program staff follow up on the referral? Do program staff “check in” on referred families after the referral?

• Has this changed as a result of COVID-19?

9. How much input do families have about the services they receive? Do they decide on which services they will receive?

   • Has this changed as a result of COVID-19?

10. What supports, services, or incentives do you offer to keep families engaged in [Program name]?

    • Has this changed as a result of COVID-19?

D. Program overview

Let’s talk about [Program name] more generally.

1. What is the main goal of [Program name]?

   • Has this changed as a result of COVID-19?

2. As we discussed, [Program name] provides [X], [Y], and [Z] services together for families. Many programs provide one service to families. Why did your program decide to coordinate multiple services? How did [Program name] and partners come together to create this effort to coordinate services? What was/is the role of the state?

• Who or what drove this effort?

• Had you worked with the partners before or are these partnerships new?

• Who led planning activities?

• Was family or client voice used to design, refine, or govern the approach?

• What mechanisms were used? [PROBE: steering committees, work groups, parent policy council, etc.]

• Are these partnerships formalized through a contract or agreement? How did you define partner roles and responsibilities?

• When did you begin providing coordinated services? [We know [Program name] began coordinating services in X]. How long did the planning/start-up take?

• What were the most critical decisions in the planning of [Program name]? What are the most critical decisions that have come up over the course of your effort (after the planning phase)?

• What were the perceived opportunities?

• What were the perceived challenges?

• Has any of this changed as a result of COVID-19?

3. What is the organizational structure among the partners? What is the relationship between state/local partners?

• Is there a lead organization? If so, which organization is the lead?

• Which organization(s) have oversight?

• Which organizations are involved in decision-making? How are decisions made?

• Has this changed as a result of COVID-19?

4. How much support was there for coordinating services when you first implemented such services?

• How much support is there for [Program name] now? [PROBE: among the community, among funders, among staff, etc.] What type of support did you have – funding, in kind, training, TA?

• Has this changed as a result of COVID-19?

5. What goals do you have for the families you work with? Were goals set in collaboration with partners? Families?

• Have your expectations about your anticipated goals or outcomes changed since you started working at [Program name]?

• If so, how and why?

• Has this changed as a result of COVID-19?

6. What benefits do you expect [Program name] to have?

• How does [Program name] benefit families and children?

• What benefits does [Program name] have in the community?

• Have you noticed other benefits of [Program name] as a result of COVID-19?

7. Do you think [Program name]’s efforts to coordinate services will have any benefits to [state name]?

   • Has this changed as a result of COVID-19?

E. Eligibility and enrollment

Let’s discuss how families enroll in [Program name]

1. How do families learn about [Program name]?

2. Do you recruit families to participate?

• What types of outreach/marketing do you use? What organizations and staff are involved in outreach?

• How do you advertise [Program name]?

• Do you receive referrals from other agencies/organizations? If so, can you please describe the referral process?

• Have you made changes to recruitment as a result of COVID-19?

3. What are the eligibility criteria for participating in [Program name]?

• Do different services have different eligibility criteria for participation? If so, did you do anything to reconcile the differences in eligibility criteria? If so, please describe.

• How has this improved the process of determining eligibility for families?

• What challenges do you face with eligibility criteria? What have you done to deal with these challenges?

• Has any of this changed as a result of COVID-19?

4. What is your process for determining who is eligible? Which agencies that are a part of [Program name] are responsible for determining if families are eligible?

• Are all members of the household assessed for eligibility to participate in [Program name]?

• Which program or partner staff makes the eligibility determination?

• What is the process for enrolling families into [Program name]? Do partner organizations enroll families into [Program name]? [PROBE: How soon after enrollment do families start receiving services? What is the process for obtaining services after being enrolled?]

• Were there any differences in enrollment processes between you and the partner organizations? If so, did you do anything to reconcile the differences in enrollment processes? If so, please describe.

• How do you determine the needs of the parents, children, and families? Is there a process or are there tools you use to determine family needs after they have enrolled in [Program name]? If so, please describe this process.

• [IF THERE IS A TOOL]

• How long is the tool? Who conducts the assessment? In what format is the assessment conducted (test, meetings, group discussions, and so on)?

• What do you do with the information you gather from the assessment?

• Has any of this changed as a result of COVID-19?

5. Are you and partner agencies doing anything differently in your approach to eligibility and assessment because of your partnership? Do you think you would approach program eligibility differently if you were a standalone program rather than part of a coordinated services effort? [PROBE: For example, a “no wrong door” approach that allows each agency to conduct similar eligibility and assessment processes and share the results with partners. Or dividing responsibilities for eligibility determination and assessment to reduce overlapping processes across agencies.]

   • Have you and partner agencies made any changes to your approach to eligibility and assessment as a result of COVID-19?

F. Staffing and organizational structure

I have a couple questions about how [Program name] is organized and how program staff work together.

1. What is your staffing and management structure? [Interviewer note: Request copy of organizational chart, if one wasn’t provided previously.]

• Do you work on [Program name] full-time or do you have other responsibilities for other programs in your organization? How much time do you spend on these other responsibilities?

• Has this changed as a result of COVID-19?

2. How are staff trained? What trainings do staff go through? Please describe the training you received/gave.

   • Has this changed as a result of COVID-19?

3. How often do staff turnover?

   • Has this changed as a result of COVID-19?

4. Who makes hiring decisions for [Program name] the lead organization, partners, or both?

   • Has this changed as a result of COVID-19?

5. When [Program name] was getting started, did you hire new staff or bring on existing staff?

• Who made those hiring decisions?

• [If new staff were hired] What kinds of qualifications did staff need to be hired for [Program name]?

6. What are the unique staffing needs of programs that coordinate multiple services for families as compared with programs that only provide one type of service? Has this changed as a result of COVID-19? How much time do you estimate it takes to coordinate services? In other words does coordinated service delivery require more staff time than delivering one type of service would? [PROBE: Do you have staff dedicated to coordinating services for families?]

7. You’ve mention you regularly work with [X], [Y], and [Z] to coordinate services. What other agencies, organizations, and providers have contributed or are involved in [Program name]?

• Have they contributed time? Expertise? Connections to other stakeholders? Funding?

• What are each of their roles?

• Has this changed as a result of COVID-19?

G. Partnership building

Next, I’d like to talk the process of working with partners and activities you engage in to maintain your partnerships.

1. Did any of the partnerships change over time? If so, why and when? How did it affect [Program name]?

• Were any added over time? If so, why and when? How did it affect [Program name]?

• Are there any partners you would like to add to [Program name]? Why?

• Has any of this changed as a result of COVID-19?

2. Has collaboration between partner organizations or their roles in [Program name] changed over time?

   • Has this changed as a result of COVID-19?

3. How do you maintain effective communication and relationships with partners?

   • Has this changed as a result of COVID-19?

4. Did you anticipate or would you have liked more involvement from any of [Program name]? Please describe.

   • Has this changed as a result of COVID-19?

5. What are the relationships like among the partners involved in [Program name]?

• What are the most effective elements of the collaboration among partners? Why?

• What aspects of the collaboration could be improved?

• What have you learned about establishing and maintaining these collaborations?

• Has any of this changed as a result of COVID-19?

H. Data and data systems

Next I’d like to talk about how [Program name] measures progress for families and children.

1. What types of data do you collect about your services or the families that participate in [Program name]?

2. Staff in telephone calls indicated you use [data management system] to manage your data. Is this still correct?

3. What program and partner staff have access to the data system? Do state/local partners have access to the data system?

• [If partner staff have access to data] How do you share data? Do you share it with state/local partners? What rights or capabilities do partner staff have in your data system? Are there any challenges in sharing data? Were there challenges in initially setting up a shared data system? How were any challenges overcome?

• Are any data systems linked? If so, what data systems are linked? What kind of data do you have access to as a result?

• Do you use data to track participants’ progress across different services? Do you use data provided by other services or partners?

• What challenges associated with linking data have you faced? How did you address these challenges?

• What is the benefit of linked data?

• Is there additional data you wish you had access to or are currently trying to access? If so, what data and why?

• Has any of this changed as a result of COVID-19?

4. What types of data are collected in the data system?

• Demographic information

• Income

• Eligibility

• Screening data

• Service receipt data

• Assessment data

• Etc.

• Has this changed as a result of COVID-19 (e.g. new data elements added or less data collected)?

5. What processes or systems are in place to measure and track families’ participation?

• How do you measure who received services, what services they received, and how much or what type of services were provided?

• Has this changed as a result of COVID-19?

6. How do you measure the success of [Program name]? How do you know [Program name] is working?

• What are your key indicators/outcome measures?

• Has this changed as a result of COVID-19?

7. How do you measure progress toward these indicators?

• Do you use this data to identify areas for improvement?

8. Does [Program name] collect data on participant’s progress toward their goals? If so, how is it tracked? Do you share it with families?

9. What processes or systems are in place to monitor the data in the systems?

• Are there any processes to check the completeness of data?

• Are there any processes to check the accuracy of data?

• Are there any processes to check data input by partners?

• Has any of this changed as a result of COVID-19?

10. What additional supports or capacity do you need to collect, analyze, or share data?

11. What areas would you like to collect, analyze, or share data on that you currently do not?

    • Has this changed as a result of COVID-19?

12. What are some successes associated with collecting, analyzing, and sharing data?

13. What were some the challenges you faced with collecting, analyzing, and sharing data?

14. Are there any unique ways you have used data to understand how COVID-19 has affected your work?

I. Funding

I’d like to learn a little more about how [Program name] is funded.

1. During our telephone call, staff shared that [Program name] receives [Head Start/TANF/ other federal grant funding/private funding, or funding from other sources] and combines these funding streams. Does [Program name] receive any other sources of funding?

• Are all program activities for all partners supported by these funding sources? If not, please describe the specific program activities and/or partners that are not funded.

• Are there any partners that are not supported by these funds? What about local partners/local implementation sites?

• Has this changed as a result of COVID-19?

2. How has combining these funding sources made it easier to coordinate services? How has it made it harder to coordinate services?

   • Has this changed as a result of COVID-19?

3. Has [Program name] encountered any barriers/challenges when trying to combine funding streams? How did you address these barriers?

   • Has this changed as a result of COVID-19?

4. Has anyone or anything in particular been helpful in addressing or removing barriers? What did they do?

5. What are the remaining barriers to combining funding streams?

6. Do you see any efficiency benefits to coordinating programs and services, for example, are there lower costs of providing a broader set of services to families? Or better outcomes than families would achieve if they received services in a different way? If so, why do you think that is? Does service coordination change how staff think about prevention and providing needs for families before a crisis happens? [PROBE: For example, are families seeking emergency health care less now that they participate in Early Head Start which tracks doctors’ visits?]

   • Has this changed as a result of COVID-19?

J. Coordination and alignment

You talked about the different services families receive in [Program name]. Now I’d like to talk about how those services work together to meet the needs of families and the process of integrating these services, starting with coordinating at higher levels of governance.

1. How have federal, state, and local agencies been involved in your efforts to coordinate services?

• Have any private organizations contributed to your efforts?

• Has this changed as a result of COVID-19?

2. Have any federal, state, and local policies make it difficult to align services?

• What strategies did you use to address these barriers?

• Has anyone or any organization been particularly helpful in addressing or removing federal and state barriers? What did they do?

• Since the start of your effort to coordinate services, have any changes been made to federal, state, or local policy to remove any barriers?

• Has this changed as a result of COVID-19?

3. What additional policies, procedures, or systems are needed to support [Program name]? [PROBE: for example, polices, requirements, regulations, or systems.]

   • Has this changed as a result of COVID-19?

4. [Interviewer note: Alignment of eligibility and enrollment is discussed earlier] I’d like to ask about how you and your partners aligned your programs so that you could provide a coordinated set of programs to families. Did you align:

• Policies?

• Procedures?

• Data?

5. What was the process to align each?

6. What lessons did you learn through your efforts to align these aspects of [Program name]?

7. What are the benefits and challenges to coordination among multiple partners?

   • Has this changed as a result of COVID-19?

K. Early Care and Education

[For ECE directors and staff]

1. How old are the children in your ECE program?

2. What are your hours of operation? Are you open year-round?

   • Has this changed as a result of COVID-19?

3. What percentage of families who participate in [Program name] use your ECE services?

   • Has this changed as a result of COVID-19?

4. How is the ECE program meeting the needs of families in [Program name] and in the community? Is the ECE program fully enrolled? Is there a waitlist? Are there available slots in the ECE program?

   • Has this changed as a result of COVID-19?

5. What are the staff requirements for educators in your ECE program?

   • Has this changed as a result of COVID-19?

6. Do ECE staff receive ECE professional development or training? Please describe this PD.

   • Has this changed as a result of COVID-19?

7. Does the ECE program participate in QRIS?

8. How are your ECE services funded? Does the ECE program receive Head Start/CCDF/TANF/ Other federal grant funding/ private funding, or funding from other sources?

• Do you layer funding streams? If so, how do you layer them?

• Has this changed as a result of COVID-19?

L. Best practices, barriers, challenges, and lessons learned

Finally, I’d like to talk a little more about what you have learned from coordinating services for families and children.

1. What has been the impact of [Program name] on:

• Your practice and your work?

• The parents/children/families you work with?

• The larger community?

• The efficiency of service delivery? [PROBE: In other words, what role has [Program name] played in how easy (or challenging) it is for families in your community to access and receive services and for [Program name] to provide services?]

• Has any of this changed as a result of COVID-19?

2. How has a coordinated program model allowed you to better meet the needs of your service population?

   • Has this changed as a result of COVID-19?

3. What lessons have you learned about providing coordinated services? Have there been any unexpected challenges?

   • Has this changed as a result of COVID-19?

4. If you could make any changes to [Program name], what would you change?

5. What advice would you give to other programs considering implementing coordinated services?

6. Is there anything we haven’t discussed that you would like to talk about?

Thank you for taking the time to share your thoughts and ideas with us today. This discussion has been very helpful in learning more about [Program name]. We appreciate your time.