MIECHV Benchmarks Listening Sessions

MIECHV Reporting Requirements - Discussion Group Topic Guide

OMB #: 0970-0356

EXPIRATION: xx/xx/xxxx

Maternal, Infant and Early Childhood Home Visiting: reporting requirements Listening sessions

TOPIC GUIDE FOR Discussion GroupS

with home visiting experts and stakeholders

This topic guide will be used for group discussions with Maternal, Infant and Early Childhood Home Visiting (MIECHV) experts and stakeholders who can inform federal considerations about the current performance measurement reporting system and how it might be simplified to reduce burden and more accurately and reliably provide data for program performance, accountability and management purposes.

1. Warm-up/Introductions

• Purpose

HRSA and ACF are interested in gathering comments and feedback from grantees, measurement, quality improvement, and subject matter experts on the current MIECHV performance measurement reporting requirements. They are interested in hearing your recommendations for simplifying the current system and coming up with a set of standardized measures for particular constructs (when appropriate). In addition, HRSA and ACF would like to gather recommendations and cautions for how to implement any changes to the requirements. This effort will build on the work that the Pew Home Visiting Campaign undertook over the last year to define a simpler and more robust core set of standardized performance measures for home visiting, which I will discuss in more detail shortly.

Specifically, we have invited you here today to discuss: [FILL IN THE PERFORMANCE MEASUREMENT AREA(S)]

Please note that participation is voluntary and at any time during the listening session you may decide not to participate. We estimate that this session will last two hours. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it has a currently valid OMB control number. The OMB control number for this collection is 0970-0356

• Ground rules and expectations

Before we get started, I would like to set some ground rules to ensure we have a productive and useful conversation. First, please identify yourself when you speak. Second, please speak one at a time. Finally, please keep the conversation within the group and to not discuss the comments of others outside of the session. All of the information and feedback that you provide will be only be shared with the consultants, contractors working on this project and relevant federal staff. The information shared during the session will not be published but will be used to inform HRSA and ACF. A note taker will be taking detailed notes to make sure we capture your feedback accurately. However, your name will not be connected to your responses. If you have further thoughts or ideas, please contact me or a member of the HRSA/ACF team to follow up via email or phone. Does anyone have questions before we begin?

B. Review of the current performance measurement requirements

The current MIECHV performance measurement guidelines require grantees to measure multiple constructs under [FILL IN THE PERFORMANCE MEASUREMENT AREA(S)], such as [FILL IN PERFORMANCE MEASURES FROM TABLE 1].

MIECHV Performance Measurement Area	List of Performance Measures
Maternal and newborn health	Prenatal Care Parental use alcohol, tobacco, or illicit drugs Preconception Care Interbirth interval Screening for Maternal depression Breastfeeding Well-child visits Maternal and child health insurance status [Medical home or primary healthcare provider for both mothers and children (tribal grants only)]
Child injuries, child abuse, neglect, or maltreatment, and reduce emergency department visits	Visits for children to the emergency department from all causes Visits of mothers to the emergency department from all causes Information provided or training of adult participants on prevention of child injuries including topics such as safe sleeping, shaken baby syndrome or traumatic brain injury, child passenger safety, poisonings, fire safety (including scalds), water safety (e.g., drowning; unsafe levels of lead in tap water), and playground safety Incidence of child injuries requiring medical treatment Reported suspected maltreatment for children in the program (allegations that were screened in by the child protective service agency but not necessarily substantiated) Reported substantiated maltreatment (substantiated/indicated/alternative response victim) for children in the program First-time victims of maltreatment for children in the program
School readiness and achievement	Parent support for children's learning and development (e.g., having appropriate toys available, talking, and reading with their child) Parent knowledge of child development and of their child's developmental progress Parenting behaviors and parent-child relationship (e.g., discipline strategies, play interactions) Parent emotional well-being or parenting stress Child’s communication, language, and emergent literacy Child’s general cognitive skills Child’s positive approaches to learning including attention Child’s social behavior, emotion regulation, and emotional well-being Child’s physical health and development
Crime or domestic violence	Screening for domestic violence Of families identified for the presence of domestic violence, number of referrals made to relevant domestic violence services (e.g., shelters) Of families identified for the presence of domestic violence, number of families for which a safety plan was completed.
Family economic self-sufficiency	Household income (including earnings, cash benefits, and in-kind and non-cash benefits) Employment or education of participating adults Health insurance status of participating adults and children
Coordination and referrals for other community resources and supports	Number of families identified for necessary services Number of families that required services and received a referral to available community resources Number of completed referrals (i.e., the home visiting provider is able to track individual family referrals and assess their completion, e.g., by obtaining a report of the service provided) MOUs: Number of Memoranda of Understanding or other formal agreements with other social service agencies in the community Information sharing: Number of agencies with which the home visiting provider has a clear point of contact in the collaborating community agency that includes regular sharing of information between agencies.

C. Current experience and perceptions of performance measurement reporting requirements

• What is your role in collecting, analyzing, and reporting performance measurement data?

• How would you describe the quality and utility of the data as currently collected?

  • What is the utility of the performance measurement data for state- or community-level program management and improvement

  • How would you describe the comparability of the performance measurement data across grantees and to meet federal reporting requirements?

• How do the current requirements alignment with what is actually happening on the ground?

• How do the constructs align with areas that you think grantees have the most potential to see improvement in the next year, 3 years, 5 years?

• How do the performance measurement reporting requirements fit with other state, community, and model-specific data systems requirements?

  • How do they fit with model goals and reporting requirements

• What are the strengths of the current reporting requirements?

  • If changes are made to the requirements, which aspects should remain and why?

  • What have we learned that would guide moving forward and the next step of transformation

• What are some of the challenges of the current reporting requirements?

  • Which performance measures are particularly difficult to report on and why?

  • If changes are made to the requirements, which aspects should be modified and why?

D. Criteria for simplifying the performance measurement reporting requirements

Pew and others have been exploring home visiting reporting broadly and the potential for shared indicators across states. In terms of MIECHV, there are many criteria that one could use to reduce the number of measures under each performance measurement area. Pew, through their Home Visiting Data Initiative, has used an adapted version of the Results Based Accountability (RBA) framework to identify key measures to help examine the shared impact of home visiting, build on existing administrative data, and measure progress toward outcomes:

• Communication power: Can be understood and is meaningful to experts, policy makers, and the general public, has policy value/ salience, and represents consensus among diverse stakeholders.

• Proxy power: Represents a relevant topic, is proximal to work; based on evidence that home visiting can have impact, says something important about the desired outcome, and reflects a set of related issues.

• Data power: Is measurable now with reliable and readily available data, maximizes administrative data, and maximizes data collected by models/ programs.

We would like you to reflect on these criteria during our discussion today as we attempt to come up with a simplified list of measures under [FILL IN THE PERFORMANCE MEASUREMENT AREA(S) FROM TABLE 1].

E. Recommendations for simplifying the performance measurement reporting requirements based on Pew’s work

Pew and their data initiative group members have put in a great deal of work to come up with a concise list of indicators from the home visiting field broadly. Thus, we would like to use the Pew recommendations as a starting place. Under [FILL IN THE PERFORMANCE MEASUREMENT AREA FROM TABLE 2], Pew’s home visiting data initiative has proposed the following constructs: [FILL IN CORRESPONDING INDICATOR(S) FROM TABLE 2] defined as the [FILL IN THE CORRESPONDING DEFINITION FROM TABLE 2]. It should be noted that not all legislatively-mandated performance measurement areas are covered by this proposed list of indicators.

• Do the indicators that have come out of the Pew effort represent the key indicators under [FILL IN THE PERFORMANCE MEASUREMENT AREA(S) FROM TABLE 2]? Why or Why not?

  • Are there additional indicators that you would add to this list?

  • Which constructs should be dropped or added to best represent the work on the ground and reflect changes over time in implementation quality and outcomes?

• What do you see at the advantages and disadvantages of requiring grantees to measure the indicators that Pew has proposed?

  • Are these indicators feasible for grantees to measures?

• Do these indicators lend themselves to standardized measures? Which of these constructs should have standardized measures across grantees and why?

Table 2. Pew Home Visiting Data Initiative Proposed Measure as of December 2014

MIECHV Performance Measurement Area	Proposed List of Measures	Definitions
Maternal and newborn health	Interbirth interval Interconception/ Postpartum visits Breastfeeding Parental use tobacco Well-child visits Maternal depression and stress	Percent of women participating in home visiting who had an interbirth interval of 18 months or more Percent of women enrolled in home visiting at the time of birth who receive a timely postpartum visit (within two months following birth). Percent of mothers enrolled in home visiting during pregnancy who initiate breastfeeding for their infants. Percent of women participating in home visiting for at least six months who are currently smoking or using tobacco. Percent of children who participated in home visiting that received last recommended well child visit based on AAP schedule. Percent of women participating in home visiting who receive maternal depression screening with validated tool.
Child injuries, child abuse, neglect, or maltreatment, and reduce emergency visits	Child maltreatment	Percent of children who participated in home visiting with reported cases of child maltreatment
School readiness and achievement	Parental Capacity Child Development	Percent of parents participating in home visiting who engage in behaviors that nurture children Percent of children who participated in home visiting that received developmental screening and were referred when indicated.
Crime or domestic violence
Family economic self-sufficiency	Maternal education	Percent of women that entered home visiting without high school or GED completion who have completed high school or equivalent.
Coordination and referrals for other community resources and supports

F. Recommendations for simplifying the performance measurement reporting requirements based on the current reporting system

• Do the current required performance measures represent the key constructs under [FILL IN THE PERFORMANCE MEASUREMENT AREA(S) FROM TABLE 1]? Why or Why not?

  • Are there additional measures that you would add to this list related to the benchmark performance measurement areas?’

  • Which measures should be dropped or added from the current requirements to best represent the work on the ground and reflect changes over time in implementation quality and child and family outcomes?

  • Are there additional measures that you would add to this list related to the effective implementation of home visiting programs (e.g., related to family engagement)?

• If you had to pick two or three measures to retain from each current performance measurement area, which would you pick and why?

• How would any recommended changes improve the quality, utility (particular based on state or community needs and/or goals) and ease of reporting?

G. Considerations of how changes may relate to other state and national reporting requirements and efforts

• Are there efforts in your state (or community) or across states that provide a model for identifying a set of core standardized measures?

  • How would greater alignment across grantees benefit state- and national-level learning and the reliability and validity of reporting? How important is such alignment to you and the organizations you represent?

  • What do you see as the impact of potential changes on model-specific reporting?

H. Standardized Measurement

• What do you see as the utility or drawbacks of standardized measures in a performance measurement system?

• How might the utility of standardized measures be influence by state- or community-specific factors, such as state or community priorities, the use of multiple home visiting models, other reporting requirements tied to other sources of home visiting funds?

• Are there particular performance measures that lend themselves more or less to standardized measurement? Which constructs should have standardized measures across grantees and why?

• For the constructs that are not measured in a standardized way, what are the implications for looking across grantees to talk about the program as a whole?

  • What are the benefits and drawbacks to have a mix of standardized measures and non-standardized measures across grantees?

I. Recommendations for the process of making changes to the reporting requirements

• What are the implications of any proposed changes for grantees and their data systems?

• What are the implications for model developers and their data systems?

• What do you recommend for the amount of time needed to plan and implement any changes?

• What opportunities exist for shared learning across grantees and models about how to implement recommended changes?

J. Closing recommendations

• Other issues/topics related to streamlining reporting requirements

• Other recommendations that were not discussed

  • Additional measures or constructs that might be useful to measure even if not a requirement of the MIECHV grant.

1