Supporting Statement B- MCHB-HV-AIM Supervision Support- New - OIRA_revised.7.30.24 (Clean)

Supporting Statement B

Home Visiting Assessment of Implementation Quality Study: Understanding Supervisor Supports in Home Visiting

OMB Control No. 0906-XXXX

B. Collections of Information Employing Statistical Methods

1. Respondent Universe and Sampling Methods

Information will be collected from several different groups of respondents via surveys, interviews, and focus groups using a combination of convenience and purposive sampling methods. Table 1 summarizes the sampling approach by data collection activity and sample population. As shown, in Table 1, we will use a convenience sampling approach for the initial recruitment survey and a mix of convenience and purposive sampling for the interviews and focus groups. A convenience sampling approach allows us to easily and quickly recruit potential study participants who are interested in participating with little burden on their part. For this ICR, this approach will allow us to reach our goal to identify potential respondents for further data collection. A purposive sampling approach is a strategy to select participants based on individual characteristics (in this case, their role in a home visiting program). We anticipate that speaking to up to up to 50 home visiting program leaders, 50 supervisors, and 50 home visitors will allow us to hear from a diverse group of participants from across the country and achieve saturation.¹ Each sampling strategy is explained in more detail below, organized by data collection activity.

Table 1. Sampling Approach

Data Collection Activity	Sample	Sampling Approach
Recruitment Survey	Home Visiting Program Leaders	Convenience sampling
	Supervisors
	Home Visitors
Interviews	Home Visiting Program Leaders	Convenience and purposive sampling
Focus Groups	Supervisors	Convenience and purposive sampling
	Home visitors

Recruitment Survey

The MIECHV Program provides funding to 56 state and jurisdiction awardees² and 41 Tribal grantees.³ All 97 MIECHV leads will be invited to distribute a recruitment flyer and information about the survey to the approximately 730 MIECHV-funded home visiting programs to share with program leaders, supervisors, and home visitors. Because we will be using a convenience sampling approach, we will cut off the recruitment survey at 250 respondents as we believe a sample of this size from which to recruit will ensure a diverse sample and account for participant dropout.

Interviews and Focus Groups

Recruitment for interviews and focus groups will take place through the Recruitment Survey, which will function as a screener for eligibility to participate in an interview or a focus group and collect demographic information. If more people express interest in the study than we have space to accommodate, the study team will consider additional participant characteristics (e.g., race/ethnicity, years in their role and in the field, education) and program characteristics (e.g., model, location, number of home visitors, typical caseload) to ensure a diverse array of participants.

2. Procedures for Collection of Information

As described in the previous section, we do not propose to use statistical sampling approaches for this descriptive study. There will likely be variation in the number of participants; therefore, the anticipated sample size for each data collection activity will vary. In this section, we describe the procedures we propose for each of the data collection activities.

Recruitment Survey

The purpose of the recruitment survey will be to identify home visiting program leaders, supervisors, and home visitors who would be interested in participating in a focus group or interview. The survey will be administered electronically via REDCap and participants can complete the survey from a variety of devices (e.g., computer, phone with internet access, tablet).

Interviews and Focus Groups

The purpose of the interviews and focus groups is to learn more about the experiences of home visiting program staff with supervision. This data will allow us to identify whether findings from a previous quantitative analysis, which are based on older or non-representative secondary data, align with home visiting program staff’s current experiences in the post-COVID-19 era, describe if and how supervision support has changed in recent years, and identify any additional or different support supervisors need.

The survey will serve as our recruitment mechanism for interviews and focus groups, and the collected data will not be used for any other purpose. The study team will select interview and focus group participants from among interested respondents. If more people express interest than the study team can accommodate, the team will consider the participants’ characteristics (e.g., race/ethnicity, years in their role and in the field, education) and program characteristics (e.g., model, location, number of home visitors, typical caseload) to ensure a diverse array of participants. The study team will be intentional about including voices from Tribal home visiting programs. For focus groups, we will group interested participants according to availability and role (i.e., supervisors and home visitors will not be grouped together) and anticipate that each group will be comprised of participants representing a range of models, regions, and experience.

Interviews and focus groups will be conducted virtually and facilitated by at least two trained members of the study team. Participants can participate in interviews and focus groups from a variety of electronic devices (e.g., computer, phone, or tablet). Interviews and focus groups will be recorded, and one member of the study team will also take notes to ensure no data is inadvertently lost. After data collection is complete, the study team will begin analysis by deductively coding the transcripts with codes generated based on topics included in interview and focus group protocols and emerging findings from previous quantitative analyses conducted for this study. Then, the team will develop inductive codes in response to new and different themes that emerge from the data. The coding scheme will be refined and updated throughout the coding process as needed. At least 20 percent of transcripts will be double-coded to ensure intercoder reliability. After intercoder reliability is established, transcripts coded by a single coder will undergo quality checks from a methodological expert on the study team. Once coding is complete, the team will identify and summarize the themes that emerge and collaborate with workgroup members and expert advisors to help interpret and contextualize the findings.

3. Methods to Maximize Response Rates and Deal with Nonresponse

All data collection activities are completely voluntary. To maximize response rates, all research staff will be trained on the data collection procedures and on principles of respondent cooperation. Other ways the data collection team plans to maximize response rates include pre-testing these questions with appropriate participants for each respective protocol. At the time of data collection, all staff will provide detailed information about the study, provide an overview of the importance of the study, and provide information about receiving an incentive for participation. As described in A.10., all respondents will also be informed that their responses are confidential to the extent allowed by law. In addition, every effort will be made to establish a supportive and respectful relationship with participants, and participants will be reminded that they are free to refrain from answering questions or excuse themselves from participating at any time.

For the recruitment survey, we will ask state, jurisdiction and Tribal MIECHV Project Directors to distribute it to their home visiting programs or staff in their state, jurisdiction, or Tribe. If an insufficient number of people respond to the Recruitment Survey, we will send one reminder to the MIECHV Project Directors to encourage them to recirculate the recruitment survey.

The study team will reach out directly to participants who have expressed interest in completing an interview or focus group at two time points to facilitate scheduling, and work to find times that are convenient for participants. Once a focus group or interview is scheduled, the team will provide two reminders (one the day before the appointment, and one the day of the appointment) to participants to minimize no-shows. If participants are unable to attend their scheduled interview or focus group, the study team will work to re-schedule if possible.

Given our sampling approach, the study team will not employ any specific methods to address non-response. However, when presenting findings the team will clearly describe the study sample in order to indicate whose experiences are reflected in responses.

4. Tests of Procedures or Methods to be Undertaken

All of the data collection protocols have been pre-tested with the appropriate participants for each respective protocol. In April 2024, the study team consulted with four individuals with lived experience as a supervisor or home visitor to pilot either the interview or focus group protocol in a 60-minute video call. After responding to the questions in each section of the protocols, participants were asked to provide feedback on the clarity and flow of the questions as well as suggestions for improvement. Approval for pre-testing was not required because protocols were tested with fewer than 10 respondents. Given the descriptive nature of this project, the goal of the pre-testing was to ensure the questions will provide the study team with the type of information needed to address the research question. Feedback was used to revise questions that were unclear or difficult to answer, add clarification around questions and terminology used, and cut questions to ensure the estimated burden is not exceeded. For example, participants who piloted the home visitor protocol indicated it was challenging to respond to questions about their supervisors’ training, as they do not have much knowledge of the training content. In response, the team revised the follow up questions for home visitors who are not familiar with the content of their supervisors’ training to ask about the frequency with which their supervisors attend supervisor-specific training and whether supervisors attend training with home visitors.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

There were no individuals consulted on the statistical aspects of the design. The research team designed the data collection strategy and will collect and analyze data. The project received feedback on the study design from an interested party workgroup. This group includes families participating in home visiting, MIECHV awardees, home visiting model developers, and home visiting program administrators and staff. The study team also consulted with experts on supervision from academic and research institutions who have expertise in research methodology when developing the study design and data collection instruments.

HRSA has contracted with Child Trends and James Bell Associates to conduct this project. Child Trends and James Bell Associates are responsible for the collection and analysis of all information described in this Information Collection Request.

1 Hennink, M., & Kaiser, B. N. (2022). Sample sizes for saturation in qualitative research: A systematic review of empirical tests. Social science & medicine, 292, 114523.

2 Health Resources and Services Administration Maternal and Child Health. (2023). FY 2022 Maternal, Infant, and Early Childhood Home Visiting awards. https://mchb.hrsa.gov/programs-impact/programs/home-visiting/fy-2023-miechv-awards

3 Office of Early Childhood Development. (2023). Tribal home visiting grantees. https://www.acf.hhs.gov/ecd/tribal/tribal-home-visiting/thv-grantees