MUSE OMB Part A PASSBACK 1.17.19 FINAL CLEAN

Multi-Site Implementation
Evaluation of Tribal Home
Visiting
OMB Information Collection Request
New Collection
Supporting Statement
Part A: Justification
Non-Substantive Change Request February 2019
Initial Request Approved December 2018
Submitted July 2018
Submitted By:
Office of Planning, Research, and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
330 C Street, S.W.
Washington, D.C. 20201
Project Officer:
Aleta Meyer

Table of Contents
Information Collection Request Summary ------------------------------------------------------------- 1
A. JUSTIFICATION -------------------------------------------------------------------------------------- 2
A1. Necessity for the Data Collection -------------------------------------------------------------------- 2
A2. Purpose of Survey and Data Collection Procedures -------------------------------------------- 4
A3. Improved Information Technology to Reduce Burden ---------------------------------------- 10
A4. Efforts to Identify Duplication ---------------------------------------------------------------------- 11
A.5 Involvement of Small Organizations -------------------------------------------------------------- 11
A6. Consequences of Less Frequent Data Collection ------------------------------------------------ 11
A7. Special Circumstances -------------------------------------------------------------------------------- 12
A8. Federal Register Notice and Consultation-------------------------------------------------------- 12
A9. Incentives for Respondents -------------------------------------------------------------------------- 13
A10. Privacy of Respondents ----------------------------------------------------------------------------- 15
A11. Sensitive Questions ----------------------------------------------------------------------------------- 17
A12. Estimation of Information Collection Burden ------------------------------------------------- 17
A13. Cost Burden to Respondents or Record Keepers --------------------------------------------- 20
A14. Estimate of Cost to the Federal Government -------------------------------------------------- 20
A15. Change in Burden ------------------------------------------------------------------------------------ 20
A16. Plan and Time Schedule for Information Collection, Tabulation and Publication ---- 21

i

A17. Reasons Not to Display OMB Expiration Date ------------------------------------------------ 23
A18. Exceptions to Certification for Paperwork Reduction Act Submissions ----------------- 23
References ---------------------------------------------------------------------------------------------------- 24

ATTACHMENTS:
Attachment A.

Informed Consent Forms

Attachment B.

60 Day Federal Register Notice

Attachment C.

Agreement to Keep Participant Data Private

Attachment D.

Documentation of Initial IRB Approval and Amendment Approval for
Waiver of Parental Consent for Minor Caregivers (Ages 14-17)

Attachment K.

MIECHV Strategic Learning Agenda

ii

List of Exhibits
Exhibit A.1. Data Sources by Evaluation Aims and Questions ......................................... 9
Exhibit A.2 Proposed Incentives for Caregivers Participating in Caregiver Surveys and
Qualitative Interviews ......................................................................................................... 13
Exhibit A.3 MUSE Information Collection Burden Table .............................................. 19
Exhibit A.4 MUSE Study Time Schedule .......................................................................... 21
Exhibit A.5 MUSE Study Data Collection Schedule by Instrument ............................... 22

iii

Information Collection Request Summary
• Status of study:
o This is a new information collection as part of the Multi-Site Implementation
Evaluation of Tribal Home Visiting study.
• What is being evaluated:
o Multi-site evaluation of federally funded Tribal Maternal, Infant and Early
Childhood Home Visiting (MIECHV) programs.
• Type of study:
o

Implementation evaluation using a mixed-methods approach.

• Utility of the information collection:
o This study is the first multi-site, multi-model study that will systematically explore
how home visiting programs are operating across diverse tribal community contexts
and identify factors that lead to implementation successes. The study will address
a gap in the evidence base regarding the provision of home visiting services in tribal
and urban Indian communities.
o MUSE will provide information that will help the federal government design and
support federal home visiting initiatives in tribal communities and similar
populations.
o Tribal MIECHV programs will have access to study findings for use in decision
making about improving home visiting services for children and families.
o The information generated in this study is also expected to be used by technical
assistance providers, the scientific community, and other aligned professionals for
the purposes of refining technical assistance and other program supports and
prioritizing future research agendas.

1

A. JUSTIFICATION
A1. Necessity for the Data Collection
The Administration for Children and Families (ACF) at the U.S. Department of Health and Human
Services (HHS) seeks approval for a study of Tribal Maternal, Infant and Early Childhood Home
Visiting (MIECHV) programs. The Multi-Site Implementation Evaluation of Tribal Home
Visiting (MUSE) is the first multi-site, multi-model study that will systematically explore how
home visiting programs are operating across diverse tribal community contexts and identify factors
that lead to successful program implementation. The evaluation will provide information that will
help the federal government design and support federal home visiting initiatives in tribal
communities and similar populations. Evaluation findings will also assist programs with
improving home visiting services for children and families.
Through the proposed information collection, the evaluators will obtain information about the
planning processes employed by Tribal MIECHV programs, the services implemented by those
programs, the characteristics of staff and caregivers receiving services, and how staff and
caregivers experience home visiting services. The evaluation will collect information through
secondary data analysis, surveys of program staff and caregivers, qualitative interviews with staff
and caregivers and administrative program data.
The MIECHV program has been engaged in a broad portfolio of research, evaluation, and
performance measurement since its inception in 2010. The evidence generated by each of these
activities contributes unique perspectives to the understanding of the MIECHV program and to the
overall MIECHV Learning Agenda (see Attachment K). The MUSE study builds on lessons
learned from the Maternal and Infant Home Visiting Program Evaluation (MIHOPE)
Implementation Study. The study’s recent report1 (Duggan et al., 2018) is similarly focused on
addressing knowledge gaps related to implementation of home visiting programs. In designing
the MUSE study, the research team consulted with the MIHOPE team to create a design and
measurement strategy that represents the best of current approaches for studying implementation.
These consultations allowed for more efficient and effective data collection strategies and tools to
be integrated into the MUSE study design. Additionally, with a view towards a potential future
study of the impact of the Tribal MIECHV program as part of the MIECHV Learning Agenda, the
MUSE team is including measures of positive caregiver outcomes, including parenting selfefficacy and social support, to gain an understanding of how those measures operate with tribal
populations.
Examining a Distinctive Federal Approach to a Tribal Grant Program
MUSE is designed to provide useful information for the Tribal MIECHV program office to ensure
that the goals of building capacity to serve American Indian/Alaska Native (AIAN) families are
being met, to tailor program guidance, and increase the usefulness of support provided to grantees.
As part of the MIECHV Learning Agenda, this study is designed to generate findings that can
1

https://www.acf.hhs.gov/opre/resource/implementation-evidence-based-early-childhood-home-visiting-resultsmother-infant-home-visiting-program-evaluation

2

inform future federal investments by ACF. Questions exist around the use of intervention models
that were created and tested with the general population in tribal communities, whether these
models can be implemented as designed in different communities, and whether intentional support
for capacity building is necessary for translating evidence-based models into different community
contexts.
Through local, grantee-designed and -led single-site rigorous evaluations conducted between 2012
and 2016, Tribal MIECHV grantees assessed the effectiveness of their home visiting programs
across a variety of child and parent outcomes (Roberts et al, 2018). These grantee evaluations
conducted during the first Tribal MIECHV grant cycle addressed research questions that reflected
local community priorities and yielded important findings about individual home visiting
programs, but the applicability of these findings beyond the local contexts in which they occurred
is somewhat limited. More research is needed that examines key questions across diverse tribal
home visiting contexts. Small sample sizes, family recruitment and attrition, and staff turnover
limited some grantees’ abilities to draw conclusions about home visiting in their communities. A
multi-site study is a unique opportunity to pool data from tribal communities with small samples
sizes to answer important research questions. A multi-site study can systematically address
challenges related to family recruitment and attrition and staff turnover that complicated previous
evaluation efforts. Additionally, most grantee evaluations focused on understanding whether their
local programs achieved child and parent outcomes; local evaluations did not examine
implementation. More research on implementation is needed to interpret findings from outcome
studies in Tribal MIECHV communities. Unlike these prior studies that primarily focused on
outcomes, MUSE is designed to answer key questions about implementation of home visiting
across diverse tribal contexts. This purpose frames the scope of the MUSE study and what it will
and will not tell us about tribal home visiting. It will provide information about what kinds of
families are getting home visits, how programs are structured to provide those visits, how staff are
supported to be effective home visitors, and what happens within visits. It will not test the
effectiveness of home visiting models in changing family outcomes.
Informing Tribal Communities’ Efforts to Improve Services to Families
Grantees have repeatedly expressed their desire that the proposed study generate findings that can
help them better serve children and families in their communities. The MUSE study offers
grantees, specifically those with relatively small participant numbers, the opportunity to generate
findings that are both scientifically robust and meaningful to tribal settings and members. Our
community-engaged process for developing the study design ensures that the findings from MUSE
can be applied by tribal programs throughout the country, given that a primary goal of MUSE is
to explore supports and challenges to home visiting implementation specifically in tribal
communities. The mixed-method multi-site study design has the potential to generate new
knowledge about family recruitment and engagement; staffing; training, supervision and support
for staff; and processes for developing cultural modifications.
Legal or Administrative Requirements that Necessitate the Collection
There are no legal or administrative requirements that necessitate the collection. ACF is
undertaking the collection at the discretion of the agency.

3

A2. Purpose of Survey and Data Collection Procedures
Overview of Purpose and Approach
MUSE is a multi-site evaluation research study of the implementation of Tribal MIECHV
programs that has been designed and will be conducted within a participatory evaluation
framework. MUSE uses a mixed-method evaluation design to better understand home visiting
implementation across the Tribal MIECHV initiative from the perspectives of caregivers receiving
services and staff. The study incorporates two types of descriptive research as delineated by the
ACF Common Framework for Research and Evaluation2, foundational and exploratory. As
foundational research, MUSE seeks to comprehensively describe interventions, services,
programs, and policies being implemented across grantees, identify influential conceptual
frameworks and local theories of change, and characterize caregivers and their experience with
services. As exploratory research, the study will conduct preliminary examinations of relationships
between core constructs, drawn from human services and implementation science, to generate
evidence of potential connections between program inputs and outputs.
MUSE will use blended concurrent and sequential collection of qualitative and quantitative data
to optimize our ability to identify and respond to emergent findings in a pragmatic and feasible
way. The study approach will include secondary data analysis, qualitative interviews,
administrative data, longitudinal surveys of caregivers, home visit questionnaires, staff surveys,
and logs of program activities.
Grantees will have the option to make requests for data specific to their program to use for program
planning, grant funding, or other similar purposes, which the MUSE study team will accommodate
when capacity allows and when data reporting numbers are sufficient to protect participant
privacy. Representatives from participating grantees will be part of a dissemination committee,
providing input on dissemination priorities; collaborating on analysis, interpretation and
dissemination of findings; and approving all final dissemination materials. Participating grantees
will be made aware of and receive interim and final reports, highlighting the findings and
recommendations most relevant for this target audience.
Research Questions
The evaluation questions are organized around three primary study aims:
Aim 1: Identify and describe primary influences shaping tribal home visiting program
planning.
Aim 2: Identify and describe how tribal home visiting (THV) programs are being
implemented.
Aim 3: Explore supports and challenges to home visiting implementation in tribal
communities.

2

The Administration for Children & Families Office of Planning, Research, and Evaluation (March 2016). The Administration for Children &
Families Common Framework for Research and Evaluation. Retrieved from
https://www.acf.hhs.gov/sites/default/files/opre/acf_common_framework_for_research_and_evaluation_v02_a.pdf

4

Through Aim 1, the evaluation team will describe the tribal home visiting programs as planned by
grantees and examine the processes that influenced the design of local programs. The following
questions will be examined under Aim 1:
1. What is the local context that informs planning?
2. How are implementation science principles reflected in local program planning?
3. What is the degree of alignment between local context and implementation science
principles? How are differences addressed in planning?
4. How do model requirements inform program planning?
5. What adaptations, enhancements, and supplements to existing home visiting models are
planned? Why?
Through Aim 2, the evaluation team will describe the actual implementation of programs. The
following questions will be addressed through Aim 2:
6. How are THV programs staffed and what are the characteristics of those staff?
7. What services are provided to families?
8. What are caregivers’ experiences with services?
9. What happens during home visits?
10. What training, support and supervision do staff receive?
11. What are the characteristics of families served and do those characteristics change over
time?
Under Aim 3, the evaluation will explore associations between elements of THV to identify
influencing factors that both support and challenge program implementation. The fifth question
under Aim 3 will allow us to explore the relationship between program planning, addressed in Aim
1, and program implementation, addressed in Aim 2. The following evaluation questions will be
addressed under Aim 3:
12. What influences relationships between home visitors and families?
13. What influences what happens during home visits?
14. What influences the amount of home visiting families receive?
15. What influences staff self-efficacy and job satisfaction?
16. What influences tribal home visiting programs’ ability to implement their programs as
intended?
Study Design
MUSE addresses the aims in the section above using a rigorous, mixed methods approach. Mixedmethods allows the MUSE Team to integrate multiple theoretical approaches and worldviews into
the study design and application. This method increases study relevance, reduces limitations of
strictly qualitative (e.g. reduced generalizability) or quantitative studies (e.g. limitations associated
with statistical power) and enhances our ability to pursue a study that is responsive to stakeholders
and has a high level of cultural rigor. The theoretical orientation of this study is informed by
multiple domains including child development, implementation science, and community-engaged
research.
Quantitative and qualitative data will be collected over approximately two years. The use of
multiple data sources will ensure that constructs are queried at various time points in

5

implementation and with different modalities (i.e. quick reflection on a particular home visit and
more global feedback across months of service provision). MUSE will use a blended concurrent
and sequential collection of qualitative and quantitative data to optimize our ability to identify and
respond to emergent findings in a pragmatic and feasible way. This approach will enable strategic
expansion, triangulation and elaboration of data, thereby ensuring rich, dependable and highly
contextualized findings. Sequential, mixed method data collection will enable investigators to not
only pursue the aims separately, but also to elucidate relationships across aims. For example, the
research activities addressing Aim 1 will serve as a foundation with which findings from Aim 2
activities can be interpreted and analyzed. Such analysis will generate important results clarifying
the way in which planning processes at multiple levels (e.g. initiative planning by funder and local
implementation planning) influence actual implementation. Understanding the dynamics between
planning and implementation processes has the potential to inform practices for multiple
stakeholders including funders, model developers, and program implementers.
In addition to data collection, points of interface between qualitative and quantitative data will
occur during data analysis and interpretation, further enabling triangulation and comparison of
data across sources. This process will require strategic integration of data in dynamic data sets and
could include techniques associated with data merging, connecting and/or embedding depending
on the particular relationship or question.
Universe of Data Collection Efforts
The following is a summary of the data collection methods and instruments that will be used to
answer the MUSE research questions (RQ). See Exhibit A.1 for a matrix of research questions by
data sources.
Secondary Data Analysis (RQ 1, 2, 3, 4, 5, 16)
• Systematic review of 17 grantee implementation plans. These plans are existing documents
for which burden was approved through clearance process PRA#0970-0389 (exp.
8/31/2019). Therefore, there are no instruments or burden included for this effort within
this current package.
Caregiver Enrollment Form (RQ 7, 8, 9, 11, 12, 13, 14, 15, 16) [Instrument 1]
• Home Visitors will enter basic information about caregivers into the Caregiver Enrollment
Form after they have consented to participate in MUSE.
• Data from this form will be used to track data collection on caregivers who have agreed to
participate in MUSE, and to manage and ensure the quality of the data.
Caregiver Surveys (RQ 7, 8, 9, 11, 12, 13, 14, 16) [Instruments 2 & 3]
• Caregivers will be asked to complete a survey at baseline, 6 months, and 12 months after
study enrollment.
• The Caregiver Surveys will collect information on four key constructs: social support,
parenting emotion regulation, parenting self-efficacy and communal-efficacy, and
caregivers’ experiences with home visiting.
o Social support will be measured using an abbreviated version of the Social
Provisions Scale, plus 2 additional items added by MUSE to measure specific
support around parenting.

6

o Parenting emotion regulation will be measured using items from the Mindful
Attention and Awareness Scale, Interpersonal Mindfulness in Parenting Scale, and
Mindful Teaching Scale. The MUSE Team has adapted items from these scales to
be more appropriate for parents of young children and have added two items
specifically to assess parents’ ability to set aside their own stressors to attend to
their children.
o Parenting self-efficacy will be measured using an adapted version of the 7-item
Parenting Self-Efficacy subscale from the Parenting Sense of Competence scale. A
measure of Parenting Communal-Efficacy created by the MUSE Team will also be
used to assess the extent to which caregivers’ parenting efficacy is derived in part
from the supports of family and community.
o Caregivers’ experiences with home visiting will be measured using questions
created by the MUSE Team to help us better understand caregivers’ relationships
with their home visitors, their expectations of the program, and whether those
expectations have been met. The survey also includes questions adapted from a
survey developed by the Home Visiting Applied Research Collaborative.
Rapid Reflect Self-Completed Home Visit Questionnaires (RQ 7, 8, 9, 12, 13) [Instruments 4 &
5]
•
•
•

The MUSE Team has developed self-completed questionnaires for home visitors and for
caregivers.
The Caregiver Rapid Reflect asks about caregivers’ satisfaction with the home visit and
their home visitor.
The Home Visitor Rapid Reflect collects information on travel time to the home visit,
length and location of the home visit, people participating in the home visit, the content
covered in a home visit, the responsiveness of the home visitor to emerging needs and
interests, challenges encountered by the home visitor during the visit, and engagement of
the participating caregiver.

Staff Surveys (RQ 6, 7, 10, 15) [Instruments 6, 7, 8, & 9]
•

•

All program directors, program managers/coordinators, home visitors, and local program
evaluators will be asked to complete a one-time staff survey on their experience with the
tribal home visiting program. If a staff person plays more than one role within the program,
they will only receive one survey.
The staff surveys contain some common domains across roles that will allow us to explore
similarities and differences across role types (e.g., professional background, role within the
program, organizational culture and climate, job satisfaction, professional quality of life,
perceived program effectiveness, connection to community served and demographics) but
most survey domains are tailored by role.
o Program directors will be asked about prioritized outcomes, the program’s ability
to prepare home visitors, expectations of home visitors, self-efficacy in the program
director role, leadership style, and perceptions of training received related to the
program director role.
o Program managers/coordinators will be asked about alignment with model
prioritized outcomes, goal setting, expectations of home visitors, self-efficacy in
7

program management and supervisory role, leadership style, perceptions of training
received related to program management and supervision, and experience
providing supervision and home visit observations.
o Home visitors will be asked about the service environment, home visit planning
and resources, perceptions of the home visitor role, self-efficacy in the home visitor
role, perceptions of home visitor training, supervision and peer support received,
and reflective supervision.
o Local program evaluators will be asked about the importance of evaluation-related
skills, involvement with the program, amount of interaction with program staff,
data systems, performance measurement. CQI and role satisfaction.
Program Implementation Survey (RQ 6 & 7) [Instrument 10]
• Program managers/coordinators will be asked to complete a short survey on program
implementation activities. This survey can be completed jointly with other managers at the
grantee site if needed.
• This survey asks about services available in the community, sources of program funding,
program eligibility criteria, use of program incentives, staffing levels, caseloads, and home
visitor recruitment and hiring.
Qualitative Interviews (RQ 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16) [Instruments 11, 12, 13, & 14]
• Interviews will be conducted by study personnel during grantee site visits with individuals in
the five primary roles: program directors, program coordinators/managers, home visitors, local
program evaluators, and caregivers participating in home visiting.
• Qualitative interviews will be used to elucidate four primary areas: 1) real-world
implementation; 2) home visitor support and supervision; 3) home visitor-family relationship;
and 4) the home visit.
• The interviews will follow a semi-structured format allowing for some questions to be asked
consistently across grantees while also enabling grantee- and interviewee-specific questions
and question tailoring when necessary.
o The program director and program coordinator/manager interview protocols include
questions about staffing; services provided; program planning; model selection; model
fit; adaptations, enhancements and supplements; technical assistance; home visitor
support and supervision; why families enroll; successful parenting; program impact;
program manager role; role of community in program decisions.
o The home visitor interview protocol includes questions about home visitor support and
supervision, model fit, why families enroll, successful parenting, family experiences,
perceived program effectiveness, relationships with caregivers, what happens during a
typical home visit, and a description of a great home visit.
o The local program evaluator interview protocol includes questions about the role of the
evaluator, supports available, challenges experienced, data systems, data use,
interactions with program staff, model selection, program planning, technical
assistance, and the perceived impact of evaluation within the organization.
o The caregivers interview protocol includes questions about their expectations for home
visiting, home visitor-family relationship, what happens during a typical home visit,
description of a great home visit, screenings, visit preferences, support from home
visitor, skills gained, changes and benefits experienced.

8

Implementation Logs (RQ 6, 7, 10) [Instrument 15]
• Program managers/coordinators will be asked to submit Implementation Log data each
month.
• Implementation Logs will cover information about staff changes, training, individual and
group supervision, and family group activities.
Administrative Program Data (RQ 7, 11, 12, 13, 14) [Instrument 16]
• Grantees will be asked to designate a staff person(s) to submit Administrative Program
Data to MUSE
• The MUSE study will utilize existing individual-level data submitted by caregivers to their
local home visiting programs.
• The MUSE study will collect four types of Administrative Program Data: caregiver
demographics, screener data, home visit participation data, and group activity participation
data.
• Grantees will have the option to submit item-level substance use screener data and
summary score depression screener data.
• The MUSE Team will ask grantees to submit locally collected process data on the services
provided to families, most of which is already collected for home visiting model developers
or federal reporting requirements. These data include clients’ date of referral and referral
source, length of participation in THV, number and frequency of home visits, and
participation in family group events.
Exhibit A.1. Data Sources by Evaluation Aims and Questions

Evaluation Questions
Aim 1. Identify and describe primary influences shaping tribal home visiting program planning
1. What is the local context that informs
planning?
2. How are implementation science principles
reflected in local program planning?
3. What is the degree of alignment between
local context and implementation science
principles? How are differences addressed
in planning?
4. How do model requirements inform
program planning?
5. What adaptations, enhancements, and
supplements are planned? Why?
Aim 2. Identify and describe how tribal home visiting programs are being implemented

Implementation Logs

Staff Surveys & Program
Implementation Survey

Rapid Reflect
Questionnaires

Caregiver Surveys

Data Sources
Quantitative

Administrative Program
Data

In-Person Qualitative
Interviews

Secondary Data Analysis
of Implementation Plans

Qualitative

9

6.

How are THV programs staffed and what
are the characteristics of those staff?
7. What services are provided to families?
8. What are caregivers’ experiences with
services?
9. What happens within home visits?
10. What training, support and supervision do
staff receive?
11. What are the characteristics of families
served and do those characteristics change
over time?
Aim 3: Explore supports and challenges to home visiting implementation in tribal communities
12. What influences relationships between
home visitors and families?
13. What influences what happens during
home visit?
14. What influences the amount of home
visiting families receive?
15. What influences staff self-efficacy and job
satisfaction?
16. What influences tribal home visiting
programs’ ability to implement their
programs as intended?

MUSE data collection will occur for approximately 27 months. Once OMB approval is obtained,
participating grantees will begin enrolling caregivers into MUSE and collecting Caregiver
Surveys, and the Rapid Reflect Self-Completed Home Visit Questionnaire. These two caregiverfocused data collection activities will occur throughout the MUSE data collection period. The staff
surveys will be administered once, within the first month of the study. New staff who begin
working after the initial staff survey is conducted will be surveyed on a rolling basis throughout
the length of the study. Qualitative interviews will be conducted during one site visit made to each
participating grantee. These site visits will occur throughout the data collection period. The
Implementation Logs will be completed monthly for two years. Participating grantees will submit
administrative data every six months for a total of four times. Because administrative data will
include data on service delivery for the two-year data collection period, the final administrative
data submission will occur approximately 27 months after the data collection period begins.

A3. Improved Information Technology to Reduce Burden
Respondents will complete all surveys and logs on a computer or handheld device which requires
a lower time commitment compared to paper and pencil surveys. Caregivers receiving home
visiting will use a touch screen to complete surveys, which further reduces response time and is
more intuitive than using a keyboard. Reports and reminders generated by the MUSE Team will
be made available to home visiting staff to reference data collection schedules for participating
caregivers and keep track of surveys that need to be completed. Administrative data will be
submitted through a secure cloud-based portal with drag and drop capabilities. This mechanism
for submitting data entails lower burden than emailing files or sending them through secured mail.
For the qualitative interviews, the data collection team will travel to the home visiting program

10

office to reduce the travel burden on staff participating in interviews. The interviews will be audio
recorded with participant consent (see Attachment A).

A4. Efforts to Identify Duplication
The data requirements for this study have been carefully reviewed to determine whether the needed
information is already available. Efforts to identify duplication included a review of the current
literature and discussions with knowledgeable experts. Limited information is available about
home visiting implementation in tribal communities, and the few existing studies focus on a single
home visiting model. This study will be the first multi-site, multi-model study of home visiting
implementation in tribal communities. No existing data source can provide the data needed to
answer the study’s research questions.
MUSE will not generate data that is duplicative of information accessible to ACF; rather, the study
will analyze data that are already collected by grantees to satisfy various reporting requirements.
THV grantees are required to calculate and report aggregate demographic, service utilization and
performance measurement data to ACF annually. MUSE will also utilize existing information by
conducting a content analysis of grantee implementation plans and performance measurement data
submitted to ACF.
• Implementation plans and collection of demographic and service utilization data are
covered under OMB control number 0970-0389, expiration date 8/31/2019.3
• Collection of performance measurement data are covered under OMB control number
0970-0500, expiration date 8/31/2020.4 Home visiting models require programs to collect
and report data on caregivers and the services they receive.

A.5 Involvement of Small Organizations
Some of the organizations involved in this study are small, non-profit organizations. The research
team has minimized burden by keeping the interviews and surveys as short as possible, only asking
about information directly tied to the study’s aims and questions, and scheduling interviews onsite and at times convenient for the respondents.

A6. Consequences of Less Frequent Data Collection
When developing the design for the MUSE study, we carefully considered the respondent burden
associated with different research questions and methodologies. We elected to conduct staff
surveys once and assess staff characteristics at a single point in time rather than observe change
over time. Similarly, we opted to conduct one round of qualitative interviews instead of multiple
rounds of interviews to limit burden on staff and caregivers.

3

The Tribal MIECHV program will be submitting a renewal OMB package for demographic and service utilization
data in early 2019.
4
The Tribal MIECHV program will be submitting a renewal OMB package for the performance measurement data in
late 2019.

11

Caregiver Surveys are collected three times: at baseline, six months and 12 months. Other studies
have shown that twelve-month attrition rates can reach 50% or higher (Gomby, et al., 1999;
O’Brien, et. al., 2012), so including the six-month time point will allow us to gather follow-up data
from more respondents than if we only included a twelve-month follow-up, thereby increasing our
sample size and power to estimate effects. Less frequent data collection would result in fewer
paired baseline and follow-up surveys.
To reduce the burden on home visitors and caregivers, we have opted to collect data on what
happens during individual home visits (Rapid Reflect) for a twenty-two-month period on a sample
of visits instead of all home visits conducted during the data collection period. The Rapid Reflect
will be completed for each home visit completed by home visitors at the grantee organization one
week out of each month. This is an effective sampling rate of 23% of all home visits conducted.
Reducing the sampling of home visits further, by collecting the Rapid Reflect less frequently,
would limit the variation and representativeness of data across caregivers and home visits, and
limit the power to detect effects in statistical analyses.

A7. Special Circumstances
There are no special circumstances for this data collection.

A8. Federal Register Notice and Consultation
Federal Register Notice and Comments
In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13 and Office of
Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995)),
ACF published a notice in the Federal Register announcing the agency’s intention to request an
OMB review of this information collection activity. This notice was published on February 28,
2018, Volume 83, Number 40, pages 8681-8682, and provided a 60-day period for public
comment. This notice included information about all possible burden under this OMB number and
the future submissions for anything described but not yet submitted as final will be announced in
a 30-day Federal Register Notice only. A copy of the 60-day notice is included as Attachment B.
During the notice and comment period, three comments and two requests for the instruments were
received. Changes to the instruments were made in response to public comments received,
additional feedback provided by MUSE stakeholders, and needed refinements identified as the
MUSE Team further developed the detailed data collection protocols and technologies.
Consultation with Experts Outside of the Study
The MUSE Team conducted telephone consultations with experts in the fields of home visiting,
implementation science, and evaluation of tribal programs. The Team held multiple consultations
with researchers that conducted the MIHOPE implementation study. We consulted with these
experts on study design, measurement constructs, instrument development, sampling and potential
analyses. The MUSE Team also convened a Technical Workgroup (TWG) made up of a subset of

12

the aforementioned experts as well as Tribal MIECHV grantee staff, Federal Tribal MIECHV staff,
and Tribal MIECHV technical assistance providers. Expert consultants and the TWG provided
consultation on the conceptual model, proposed study design, preliminary analysis plans, data
collection instruments and working with grantees to collect program and caregiver data. Their
recommendations helped shape the final study design. The MUSE Team will continue to convene
the TWG throughout the MUSE study.

A9. Incentives for Respondents
The MUSE study places burden on caregivers receiving home visiting services by asking them to
participate in repeated data collection. A subset of caregivers participating in MUSE (caregivers
who enroll in home visiting after MUSE begins) will be invited to take the Caregiver Survey three
times and retaining these respondents over time is integral to the quality of the data collection. We
will ask these caregivers to take a 15-minute baseline survey, a 30-minute survey at six months,
and a 30-minute survey again at 12-months. Home visiting staff will be trained in protocols to
ensure that caregivers are given privacy to complete surveys, and all staff collecting these data will
be certified in human subjects’ research protections. In addition, surveys will be collected on
tablets using secure software that ensures caregiver responses cannot be accessed by home visiting
program staff (see Supporting Statement B, section B2 for additional information). We will offer
caregivers a gift card at each time point: $10 at baseline, $15 at six months, and $15 at 12 months.
We will conduct qualitative interviews with approximately three caregivers at each grantee. These
interviews will take approximately 1 hour. We will offer a $40 gift card to caregivers who
participate in an interview. Exhibit A.2 provides an overview of the incentives to be provided.
Exhibit A.2 Proposed Incentives for Caregivers Participating in Caregiver Surveys and
Qualitative Interviews
Data Collection Method
Caregiver Surveys
Baseline (at enrollment)
6-month follow-up
12-month follow-up
Qualitative Interviews of Caregivers
In-person interview
TOTAL

Incentive Amount Estimated Time to Complete
$10
$15
$15

15 minutes
30 minutes
30 minutes

$40
$80

1 hour

There are two primary reasons for providing incentives to caregivers participating in the MUSE
study and both are critical to the scientific integrity of the study: respecting cultural protocols in
the communities participating in this study and reducing non-response bias by obtaining and
maintaining a representative sample of caregivers over time.
Demonstrating Respect for Cultural Protocols
We are keenly aware of the specific cultural and contextual importance of incentives in research,
given our experience working with tribal communities. In previous studies, partners, including
tribal MIECHV grantees and tribal communities, conveyed the importance of providing incentives.
In our team’s experience within tribal contexts, the use of incentives for participation in research
13

is expected by both research participants and tribal leaders. This is especially true for studies
conducted by researchers external to the community.
Best practice guidelines developed by AIAN communities confirm our experience, explicitly
stating that incentivizing respondents for their time is an essential element of reciprocity between
the researcher and AIAN knowledge holders (Mihesuah 1993; Nielsen et al., 2007; Sobeck et al.,
2003; Davis, 1999). The National Congress of American Indians includes incentives as an
important component of community based participatory research (Sahota, 2010). Tribal research
review entities such as Tribal IRBs and Tribal Councils have codified these community norms and
best practices by requiring compensation for research participants as a condition of study approval
(Doughty, 2017).
Incentives reinforce the notion that the knowledge being shared by local participants is valued,
respected, and honored. Mutual respect and reciprocity are strong cultural traditions in many AIAN
cultures. Offering incentives to study participants is a way of offering recognition of the value of
the knowledge being shared with the study team and establishing trust. This is particularly critical
given the context of research in tribal communities and past histories of abuses in which
researchers went into communities, took away data and local knowledge, and offered nothing in
return (Pacheco, et al., 2013). Offering an incentive at the outset is an important step in establishing
trust, making research culturally valid and ultimately obtaining scientifically rigorous data (Tribal
Evaluation Workgroup, 2013). Incentives demonstrate respect for individual community
members’ contribution to research and attempt to ameliorate logistical challenges associated with
participating in research in many tribal communities (Doughty, 2017). In each of the 17
communities that intend to participate in MUSE, the appropriate research review entity for that
grantee must review and approve the MUSE research protocol in order for it to operate there. This
determination will undoubtedly factor in whether this study will be conducted in a respectful way,
honoring the knowledge caregivers provide, as well as incentive structures.
Increasing Response Rates
Knowing that they will receive an incentive for completing a survey or interview increases the
likelihood that caregivers will complete data collection activities. Previous studies demonstrated
that AIAN people living in urban areas were much less likely to participate in a study if it was led
by the federal government and less likely to participate without incentives (Buchwald et al., 2006).
Given the mistrust of federally sponsored research in tribal communities, an incentive may be
critical in obtaining an adequate response rate. A survey of more than 1000 tribal college students
presented vignettes of different types of research studies to gauge the likelihood of participation.
Noe et al (2007) found that immediate compensation approximately doubled the odds of
participation in focus group and intervention studies and lack of compensation reduced the odds
of participation by 20% across all types of studies.
Incentives will be instrumental in retaining representative respondents over time. Caregivers will
be asked to complete surveys at three time points over the course of a year and keeping
participating caregivers engaged longitudinally will be essential. Caregivers willing to complete
initial surveys are likely to grow weary of repeated surveys and need additional motivation with
repeated measurement. Our analysis plan includes assessment of caregiver characteristics over
time, and poor retention will compromise our ability to draw conclusions from those analyses.

14

To our knowledge, there are no published experimental studies that show the impact of incentives
on response rates or response bias in AIAN communities. Experimental studies where incentives
are provided to some research participants but not others would likely not be approved by tribes
and their research review boards due to expectations of the equitable treatment of community
members. Providing some community members with services or compensation, while denying
them to others is not an acceptable practice within tribal communities (Kilburn, 2018). While we
don’t have evidence of the effect of incentives on longitudinal response rates from experimental
studies, we do know that studies asking participants to respond to hypothetical scenarios suggest
that AIAN research participants are more likely to participate in data collection when offered an
incentive (Buchwald et al., 2006; Noe et al., 2007).
Our team has extensive experience conducting research in AIAN communities; previously, when
we offered incentives, we were successful in obtaining and keeping a diverse sample of
participants over time. The MUSE Principal Investigator and Co-Investigator were involved in the
American Indian and Alaska Native Head Start Family and Child Experiences Survey 2015
(AI/AN FACES 2015; OMB# 0970-0151) which was funded by ACF and carried out with 21 Head
Start programs run by federally-recognized American Indian tribes in the Office of Head Start
Region XI. This study utilized respondent incentives and obtained high response rates among
parents in this national study. Parents were provided $25 to complete a 30-minute parent survey
in the fall of 2015 and spring of 2016; 83% of eligible parents in the fall completed a survey, and
82% of eligible parents in the spring completed a survey. In other recent work, the MUSE Principal
Investigator and Co-Investigator conducted studies that offered incentives to AIAN respondents,
and those studies were successful at recruiting and retaining study participants. In an NIH-funded
study evaluating a substance use prevention program (Thiwáhe Gluwášakapi; Strengthening
Families) for young adolescents on a Northern Plains Reservation (R01DA035111; Whitesell, PI),
youth and their parents were provided $25 for a survey that took 45-60 minutes to complete. In
that study, 88% of parents and 89% of youth were retained to complete a 6-month follow-up
survey. In a study with families enrolled in Early Head Start in a tribal community in Oklahoma
funded by an ACF grant (Buffering Toxic Stress, 90YR0058; Sarche, PI), parents were provided
$25 to complete surveys at study enrollment, and again 3-months and 6-months later. Each survey
took approximately 45 minutes; 77% of participants were retained at the 3-month follow-up and
74% at 6-months. These data suggest that our plan to offer incentives to caregivers for completing
the three longitudinal caregiver surveys can play a role in successfully recruiting and retaining
caregivers for the MUSE study. These experiences inform our selection of the survey incentives.
For the one-hour interview, on the other hand, we draw on the literature associated with cognitive
interviews, where the respondent is being asked about impressions rather than providing factual
information. OMB has allowed $40 incentives for one-hour cognitive interviews.

A10. Privacy of Respondents
All study participants will undergo a combined informed consent and HIPAA Authorization
process. CAIANH is a HIPAA covered entity and is required by law to comply with HIPAA
regulations. All participants must provide their consent prior to enrolling in the study by
electronically signing the combined consent and HIPAA Authorization form (Attachment A).
Participants will be informed that their information will be kept private to the extent permitted by
15

law. MUSE Team members and grantee staff engaged in data collection will be fully trained in
human subjects’ protection and will sign an agreement committing to keep all participant
information private (Attachment C). All interviewers and data collectors will be knowledgeable
about privacy procedures and will be prepared to describe them in detail or to answer any related
questions respondents raise. Staff participants will be assured that their individual survey
responses will never be shared with other grantee staff, Federal funders, or anyone else outside
the research team.
This study was reviewed and approved by the Colorado Multiple Institutional Review Board
(COMIRB). See Attachment D for documentation of initial IRB approval. COMIRB operates
under Federalwide Assurance Number 00005070, and thereby adheres to the requirements in the
HHS Protection of Human Subjects regulations at 45 CFR Part 46. COMIRB approval for this
study is contingent upon receiving approval from OMB and the applicable tribal review entities.
Final COMIRB approval will be obtained prior to the beginning of any data collection activities
when OMB and tribal approvals are in place. Under the study protocol submitted to COMIRB,
the following procedures will be undertaken to protect data privacy:
•
•
•
•

All data are to be stored on a secure database server which is separate from the webfacing server – a best-practice for internet-based security.
All user access requires unique user accounts and passwords.
All user actions are recorded in a secure audit log.
The database server is routinely backed-up. All security patches and application updates
are applied immediately upon release by the developer.

All qualitative interview data will be recorded on encrypted digital recorders, uploaded onto a
secure data storage platform, transcribed with identifiers stripped, and all audio files
subsequently deleted. Paper data will be stored in a locked, secured cabinet at participating
grantee offices only until they are securely transmitted to the MUSE Team for electronic entry or
until they can be entered locally into the web-facing server. Once data from the paper form has
been either securely transmitted to the MUSE Team or entered electronically, the paper data will
be shredded.
The MUSE study will not collect any direct identifiers from caregiver participants. All caregiver
data will be submitted with a program ID instead of caregiver names. Grantees will also have the
option of submitting staff data with an ID instead of staff name. Individual participants will never
be identified in any study dissemination materials or activities. We will assure both caregivers and
staff that their responses will be reported only as part of aggregate statistics across all participants.
Names of participating grantees may be used in dissemination materials, but grantee-level data
will never be publicly reported. It will be made clear that all participants may withdraw their
consent at any time and/or refuse to participate in any study activity. Information will not be
maintained in a paper or electronic system from which they are actually or directly retrieved by an
individuals’ personal identifier.

16

A11. Sensitive Questions
MUSE will collect administrative data from grantees on topics that could be considered sensitive.
Grantees can choose to submit data from depression and substance use screeners. These data are
currently collected under Tribal Home Visiting Form 2 (OMB control number 0970-0500,
expiration date 8/31/2020)5 to satisfy federal performance measurement requirements. Grantees
collect these data as specified by their local data collection protocols and report them in the
aggregate to ACF. Given the sensitivity of these data, MUSE will allow grantees to opt into
providing participant-level data for both depression and substance use screeners.
MUSE will also collect data directly from home visitors and home visiting participants using
quantitative surveys and qualitative interviews. These surveys and interviews will include
questions that could be considered sensitive because they ask respondents to make critical
appraisals of agency staff (including supervisors) and of relationships between home visitors and
families served. Similarly, staff surveys ask grantee staff about job satisfaction, future career plans
including possible intent to leave, and working environment, which might be sensitive to some.
These questions are critical to answering MUSE study Aims 2 and 3. Responses will not be
associated with individuals or grantees in dissemination activities. During the informed consent
process, survey respondents and interview participants will be informed of their right to not answer
any question(s) or stop participation at any time as well as the processes in place to keep their
responses private.

A12. Estimation of Information Collection Burden
Burden Hours6
Exhibit A.3 shows estimated burden of the information collection, which will take place over
approximately 27 months. We are requesting a three-year clearance to account for any delays in
data collection.
•

•
•

Caregiver Enrollment Form: Form containing basic information about caregivers; filled
out by 93 home visitors across all 17 grantees once per caregiver who consents (estimated
to be 14 caregivers per home visitor) to be in the study after consent is obtained; average
length of 5 minutes.
Caregiver Survey – Baseline: Survey of 565 caregivers across all 17 grantees at
enrollment (baseline); average length of 15 minutes.
Caregiver Survey – 6 & 12 Month Follow-up: Survey of 380 caregivers across all 17
grantees 6 months and 12 months after they take the Baseline Survey; average length of 30
minutes per survey.

5

The Tribal MIECHV program will be submitting a renewal OMB package for the performance measurement data in
late 2019.
6
When calculating burden, we estimated a higher number of respondents per instrument than for our sample estimates.
The additional burden is requested in case grantees enroll more caregivers, complete more home visits, or retain
caregivers at a higher rate than we estimated based on available data.

17

•
•
•

•
•

•
•

Rapid Reflect Self-Completed Home Visit Questionnaire for Caregivers: Selfcompleted questionnaire completed by 1,136 caregivers across all 17 grantees after an
average of 12 selected home visits; average length of 5 minutes.
Rapid Reflect Self Completed Home Visit Questionnaire for Home Visitors: Selfcompleted questionnaire completed by 93 home visitors after an average of 60 selected
home visits; average length of 12 minutes.
Staff Surveys
o Home Visitor Survey: Survey of 817 home visitors across all 17 grantees done one
time only; average length of 70 minutes.
o Program Coordinator/ Manager Survey: Survey of 21 program
coordinators/managers across all 17 grantees done one time only; average length of
60 minutes.
o Program Director Survey: Survey of 21 program directors across all 17 grantees
done one time only; average length of 45 minutes.
o Local Program Evaluator Survey: Survey of 30 local program evaluators across
all 17 grantees done one time only; average length of 30 minutes.
o NOTE: If someone has more than one role, we have a process for only one survey
to be administered. See SSB for detail on process.
Program Implementation Survey: Survey of 34 staff in management roles completed in
teams (average of 2 people per team) for all 17 grantees done one time only; average length
of 15 minutes.
Qualitative Interviews
o Qualitative Interviews of Home Visitors: Interviews of 42 home visitors, up to 3
per grantee; average length of 120 minutes.
o Qualitative Interviews of Program Coordinators/ Managers and Program
Directors: Interviews of 1 program coordinator/manager and 1 program director at
each grantee; average length of 90 minutes.
o Qualitative Interviews of Local Program Evaluators: Group interviews of 30
local program evaluators, up to 3 at some grantees; average length of 90 minutes.
o Qualitative Interviews of Caregivers: Interviews of 51 caregivers, 3 per grantee;
average length of 60 minutes.
o NOTE: If someone has more than one role, we have a process for only one
interview to be conducted. See Supporting Statement B for more details.
Implementation Logs: Log of implementation activities completed by program
coordinators/managers on staffing changes, training, family group activities, and
supervision completed once a month; average length of 40 minutes.
Administrative Program Data: Electronic compilation and submission of Administrative
Program Data by local program evaluators every 6 months; average length of 24 hours per
submission, including running reports, compiling and reviewing data and submitting the
data file.

7

We estimate that fewer home visitors will complete the Home Visitor Survey (n=81) than the Caregiver Enrollment
Form (n=93) because home visitors will only receive a survey after they have been in their position for four months.
Home visitors will begin completing Caregiver Enrollment Forms as soon as they begin serving families.

18

Exhibit A.3 MUSE Information Collection Burden Table
Total Number
of
Respondents

Annual
Number of
Respondents

Number of
Responses Per
Respondent

Average
Burden
Hours Per
Response

Annual
Burden
Hours

Average
Hourly
Wage

Total
Annual Cost

Caregiver Enrollment
Form

93

31

14

.08

35

$19.80

$693

Caregiver Survey Baseline

565

188

1

.25

47

$10

$470

Caregiver Survey – 6
& 12 Month Followup
Rapid Reflect SelfCompleted Home
Visit Questionnaire
for Caregivers
Rapid Reflect Self
Completed Home
Visit Questionnaire
for Home Visitors
Home Visitor Survey

380

127

2

.50

127

$10

$1,270

1,136

5681

6

.08

273

$10

$2,730

93

471

66

.2

620

$19.80

$12,276

81

27

1

1.17

32

$19.80

$633.60

Program Coordinator/
Manager Survey

21

7

1

1

7

$34.07

$238.49

Program Director
Survey

21

7

1

.75

5

$34.07

$170.35

Local Program
Evaluator Survey

30

10

1

.5

5

$39.13

$195.65

Program
Implementation
Survey
Qualitative Interviews
of Home Visitors

34

11

1

.25

3

$34.07

$102.21

42

14

1

2

28

$19.80

$554.40

Qualitative Interviews
of Program
Coordinators/
Managers and
Program Directors
Qualitative Interviews
of Local Program
Evaluators
Qualitative Interviews
of Caregivers

34

11

1

1.5

17

$34.07

$579.19

30

10

1

1.5

15

$39.13

$586.95

51

17

1

1

17

$10

$170

Implementation Logs

17

91

24

.67

145

$34.07

$4,940.15

Administrative
Program Data

17

91

4

24

864

$39.13

$33,808.32

2,240

$406.94

$59,418.31

Instrument

Total:
1

The annual number of respondents is annualized over 2 years for instruments that are completed by respondents on an ongoing
basis.

19

Total Annual Cost
The estimated total annualized cost burden to respondents is based on the burden hours and
estimated hourly wage rates for each data collection instrument, as shown in the two right-most
columns of Exhibit A-1. These estimates are based on:
•

an assumed hourly wage of $34.07 for program directors and coordinator/managers, based
on mean hourly wage for “Social and Community Service Managers”, as reported in the
May 2016 U.S. Department of Labor, Bureau of Labor Statistics, Occupational
Employment and Wage Estimates, https://www.bls.gov/oes/current/oes_nat.htm.

•

an assumed hourly wage of $19.80 for home visitors, based on mean hourly wage for
“Community Health Workers,” as reported in the May 2016 U.S. Department of Labor,
Bureau of Labor Statistics, Occupational Employment and Wage Estimates,
https://www.bls.gov/oes/current/oes_nat.htm.

•

an assumed hourly wage of $39.13 for local program evaluators, based on mean hourly
wage for “Social Scientist or Related”, as reported in the May 2016 U.S. Department of
Labor, Bureau of Labor Statistics, Occupational Employment and Wage Estimates,
https://www.bls.gov/oes/current/oes_nat.htm.

•

an assumed hourly rate of $10.00 for caregivers. This equates to annual earnings of $20,800
for a worker employed full-time year-round. For reference, this assumed wage rate is more
than the federal minimum wage of $7.25 per hour.

A13. Cost Burden to Respondents or Record Keepers
The MUSE Team will provide an honorarium to grantees participating in MUSE in recognition of
the administrative burden associated with their participation in the study. Honoraria will range
between $1,560 and $3,110 per site, depending on the size of the grantee staff team and number
of caregivers enrolled in home visiting, will be split into four equal gifts to be distributed
throughout the data collection period with the grantee’s continued participation. The honoraria is
less than the anticipated cost of the staff time needed to facilitate MUSE data collection with
caregivers.

A14. Estimate of Cost to the Federal Government
The total cost for the data collection activities under this current request will be $1,964,349.60.
Annual costs to the Federal government will be $654,783 for this proposed data collection.

A15. Change in Burden
This is a new data collection.

20

A16. Plan and Time Schedule for Information Collection, Tabulation and
Publication
Analysis Plan
In line with our mixed-method approach, the analytic plan will involve an iterative process of
analyzing both qualitative and quantitative data.
Time Schedule and Publication
Data collection will begin once OMB approval is obtained. The evaluation contractor will present
findings from analysis of the information collected in a final report, expected in late 2022. OPRE
and James Bell Associates, Inc. will publicly disseminate this report. The evaluation contractor
will also produce interim reports for participating grantees. Dissemination efforts for MUSE will
include professional peer-reviewed journal publications, federal reports, reports back to
participating grantees, and professional conference presentations. Each dissemination product will
be developed to highlight the findings and recommendations most relevant for the target
audience(s) of that product (grantee, federal, scientific, technical assistance, and/or other key
stakeholders). A dissemination committee, made up of representatives from participating grantees,
will provide input on dissemination priorities; collaborate on analysis, interpretation and
dissemination of findings; and approve all final dissemination materials. Exhibit A.4 provides an
overview of the study timeline, and Exhibit A.5 outlines a more detailed data collection schedule.
Exhibit A.4 MUSE Study Time Schedule
Expected Time Period
Activity
During OMB review period
Preparation for data collection
0 – 27 months following OMB approval
(Approximately Fall 2018-Fall 2020)
12-24 months following OMB approval
(Approximately Fall 2019-Fall 2020)

Data Collection

24-36 months following OMB approval
(Approximately Fall 2020-Fall 2021)
36 months following OMB approval
(Approximately Fall 2021)

Final Data Analyses

Interim Data Analyses
Interim Reports

Final Report and Research Briefs

21

Exhibit A.5 MUSE Study Data Collection Schedule by Instrument
Time following OMB Approval
Months Months Months Months Months
Instrument
0-6
7-12
13-18
19-24
25-30
Caregiver Enrollment Form: Ongoing
throughout the first 18 months of the data
collection period
Caregiver Survey – Baseline: Ongoing
throughout the first 18 months of the data
collection period
Caregiver Survey – 6 & 12 Month
Follow-up: Ongoing throughout months
7-26 of the data collection period
Rapid Reflect Self-Completed Home
Visit Questionnaire for Caregivers:
Ongoing throughout months 2-24 of the
data collection period
Rapid Reflect Self Completed Home
Visit Questionnaire for Home Visitors:
Ongoing throughout months 2-24 of the
data collection period
Home Visitor Survey: Within the first
month of the data collection period for
all current staff, and as needed for staff
hired during the data collection period
Program Coordinator/ Manager Survey:
Within the first month of the data
collection period for all current staff, and
as needed for staff hired during the data
collection period
Program Director Survey: Within the
first month of the data collection period
for all current staff, and as needed for
staff hired during the data collection
period
Local Program Evaluator Survey: Within
the first month of the data collection
period for all current staff, and as needed
for staff hired during the data collection
period
Program Implementation Survey for
Managers: Within the first month of the
data collection period

22

Time following OMB Approval
Months Months Months Months Months
0-6
7-12
13-18
19-24
25-30

Instrument
Qualitative Interviews of Home Visitors:
Ongoing throughout the data collection
period
Qualitative Interviews of Program
Coordinators/ Managers and Program
Directors: Ongoing throughout the data
collection period
Qualitative Interviews of Local Program
Evaluators: Ongoing throughout the data
collection period
Qualitative Interviews of Caregivers:
Ongoing throughout the data collection
period
Implementation Logs: Ongoing
throughout the data collection period
Administrative Program Data: Once
every six months throughout the data
collection period
Shading Key
Ongoing Data Collection
Data Collected as Needed for Newly Hired Program Staff

A17. Reasons Not to Display OMB Expiration Date
All instruments will display the expiration date for OMB approval.

A18. Exceptions to Certification for Paperwork Reduction Act Submissions
No exceptions are necessary for this information collection.

23

References
Administration for Children and Families. (2012). Tribal Maternal, Infant, and Early Childhood Home
Visiting (TMIECHV) Grant Program under the Affordable Care Act (ACA). Funding Opportunity
HHS-2012-ACF-OCC-TH-0302. Washington, DC: U.S. Department of Health and Human
Services. Retrieved from http://www.acf.hhs.gov/grants/open/foa/files/HHS-2012-ACF-OCCTH-0302_0.pdf.
American Academy of Pediatrics, Council on Child and Adolescent Health. (1998). The role of homevisitation programs in improving health outcomes for children and families. Pediatrics.
1998;10:486–489.
Bilukha O, Hahn RA, Crosby A, et al. (2005). The effectiveness of early childhood home visitation in
preventing violence. Am J Prev Med. 2005;28(251):11–39.
Buchwald, D., V. Mendoza-Jenkins, C. Croy, H. McGough, M. Bezdek, and P. Spicer. 2006. Attitudes
of urban American Indians and Alaska Natives regarding participation in research. J Gen Intern
Med 21:648-51.
Castro, F., Barrera, M., & Steiker, L. (2010). Issues and challenges in the design of culturally adapted
evidence-based interventions. Annual Review of Clinical Psychology, 6, 213-239,
doi:10.1146/annurev-clinpsy-033109-132032.
Davis, S. M.; Reid, R. . (1999). Practicing participatory research in American Indian communities. The
American Journal of Clinical Nutrition 69:755-759.
Del Grosso, P., Kleinman, R., Mraz Esposito, A., Sama-Miller, E., & Paulsell, D. (2012). Assessing the
evidence of effectiveness of home visiting program models implemented in tribal communities.
Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and
Families, U.S. Department of Health and Human Services.
Doughty, M. (2017). Compensation for Study of Participation in Tribal Communities: A Research
Note. Indigenous Policy Journal, Vol 28 no (1), 2017.
Duggan, A., Portilla, X., Filene, F., Crowne, S., Hill, C., Lee, H., & Knox, V. (2018). Implementation
of Evidence-Based Early Childhood Home Visiting: Results from the Maternal and Infant Home
Visiting Program Evaluation. OPRE Report 2018-76A. Washington, DC: Office of Planning,
Research, and Evaluation, Administration for Children and Families, U.S. Department of Health
and Human Services.
Dunn LB, Gordon NE. (2005). Improving Informed Consent and Enhancing Recruitment for Research
by Understanding Economic Behavior. JAMA. 2005;293(5):609–612.
Dusenbury, L., Brannigan, R., Hansen, W., Walsh, J., & Falco, M. (2005). Quality of implementation:
Developing measures crucial to understanding the diffusion of preventive interventions. Health
Education Research, 20 (3), 308-313.

24

Emanuel, E. J. (2004), Ending Concerns About Undue Inducement. The Journal of Law, Medicine &
Ethics, 32: 100-105.
Gomby, D., Culross, P., & Behrman, R. (1999). Home Visiting: Recent Program Evaluations: Analysis
and Recommendations. The Future of Children, 9(1), 4-26.
Gottfredson, D., Cook, T., Gardner, F., Gorman-Smith, D., Howe, G., Sandler, I., & Zafft, K.
(2015). Standards of Evidence for Efficacy, Effectiveness, and Scale-Up Research in Prevention
Science: Next Generation. Prevention Science 2015 Oct;16(7):893-926. doi: 10.1007/s11121-0150555-x
Guyll, Spoth, and Redmond, (2003). The Effects of Incentives and Research Requirements on
Participation Rates for a Community-Based Preventive Intervention Research Study. The Journal
of Primary Prevention (2003) 24: 25.
Kemp, L. (2016). Adaptation and fidelity: A recipe analogy for achieving both in population scale
implementation. Prevention Science, 17 (4), 429-438.
Kilburn, M. R., Lyon, K., Anderson, C., Gutman, P. and Whitesell, N. R. (2018). Methodological
Considerations for Home Visiting Research in Tribal Communities. Infant Mental Health Journal,
39: 303-311.
Lo, B. (2010). Ethical Issues in Clinical Research: A practical guide. Philadelphia, PA: Lippincott
Williams & Wilkins.
Lyon, K., Geary, E., Sparr, M., Buckless, B., Salvador, M., & Morales, J. (2015). Tribal Maternal,
Infant, and Early Childhood Home Visiting: A report to Congress (OPRE Report #2015-88).
Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and
Families, U.S. Department of Health and Human Services.
Mihesuah, D. A. (1993). Suggested guidelines for institutions with scholars who conduct research on
American Indians. American Indian Culture and Research Journal 17:131-139.
Nielsen, M. O. and L. A. Gould. (2007). Non-Native scholars doing research in Native American
communities: A matter of respect. The Social Science Journal 44:420-433.
Noe, T. D., S. M. Manson, C. Croy, H. McGough, J. A. Henderson, and D. S. Buchwald. (2007). The
influence of community-based participatory research principles on the likelihood of participation
in health research in American Indian communities. Ethn Dis 17:S6-14.
O’Brien, R. A., Moritz, P., Luckey, D. W., McClatchey, M. W., Ingoldsby, E. M., & Olds, D. L. (2012).
Mixed Methods Analysis of Participant Attrition in the Nurse-Family Partnership. Prevention
Science: The Official Journal of the Society for Prevention Research, 13(3), 219–228.

25

Pacheco, C. M., Daley, S. M., Brown, T., Filippi, M., Greiner, K. A., & Daley, C. M. (2013). Moving
Forward: Breaking the Cycle of Mistrust Between American Indians and Researchers. American
Journal of Public Health, 103(12), 2152–2159.
Roberts, E., Morales, J., Salvador, M., Lyon, K., Geary, E., & Buckless, B. (2018). Grantee-led
evaluations in the Tribal Maternal, Infant, and Early Childhood Home Visiting Program: A
compilation of grantee evaluation plan profiles, OPRE Report # XXX, Washington, DC: Office of
Planning, Research and Evaluation, Administration for Children and Families, U.S. Department
of Health and Human Services.
Sahota, P.C. (2010). Community Based Participatory Research in American Indian and Alaska Native
Communities. National Congress of American Indians Policy Research Center.
Sobeck, J. L., E. E. Chapleski, and C. Fisher. (2003). Conducting Research with American Indians.
Journal of Ethnic & Cultural Diversity in Social Work 12:69-84.
Spillane, J., Reiser, B., & Reimber, T. (2002). Policy implementation and cognition: Reframing and
refocusing implementation research. Review of Educational Research, 72 (3), 387-431.
Stoltzfus, E., & Lynch, K. (2009). Home visitation for families with young children. Washington, DC:
Congressional Research Service.
Tribal Evaluation Workgroup. (2013, September). A roadmap for collaborative and effective
evaluation in tribal communities. Washington, DC: Children’s Bureau, Administration for
Children and Families, U.S. Department of Health and Human Services.
U.S. Department of Health and Human Services, Administration for Children and Families, Office of
Child Care. (no date). Guidance for submitting a needs assessment and plan for responding to
identified needs (Phase 2 implementation plan) Cohort 3 grantees. Retrieved from
http://www.acf.hhs.gov/sites/default/files/occ/tribal_hv_implementation_plan_guidance_cohort_
3_revised_final_508.pdf.
U.S. Department of Labor, Bureau of Labor Statistics (May 2016). Occupational Employment and
Wage Estimates, https://www.bls.gov/oes/current/oes_nat.htm.
Wandersman, A., Imm, P., Chinman, M., & Kaftarian, S. (2000). Getting to outcomes: A results-based
approach to accountability. Evaluation and Program Planning, 23(3), 389-395.

26