Response to OMB Questions Draft-4.29.11

Responses to OMB Comments on Revised Package (4/11/11) Submitted byHealth Resources and Services Administration/Maternal and Child Health Bureau

An Evaluation of Core Components of the Federal Healthy Start Program

1. The Logic Model better illustrates the background assumptions that contextual factors influence implementation, which influences outcomes. However, there are a lot of one-sided and two-sided arrows in the graphic that don't seem to indicate influence or causation. Can HRSA explain what they are meant to represent?

(Also see page 4 of the Supporting Statement)

Overall, the goal of this logic model is to illustrate the potential influence of contextual factors in service and system component implementation and features, which then impacts individual and system-level outcomes, which ultimately impact population-level outcomes.

The double-sided arrows in the “Contextual Factors” section of the Logic Model are meant to represent the various characteristics from the individual level up to National and State policy levels that may interact with each other and may impact service and system implementation.

The larger arrows, from Contextual Factors to Implementation, indicate, as mentioned, the influence that contextual factors may have on how the Healthy Start projects then implement their services and systems components and the various features associated with each component.

The implementation of the services and systems may then impact outcomes in the short-, intermediate-, and long-term. Although distinct, the individual and systems short- and intermediate-term outcomes may interact or influence each other. For example, positive individual outcomes may lead participants to become move involved in the Healthy Start project at the systems level.

As short- and intermediate-term individual outcomes unfold, they inevitably will impact longer-term population outcomes.

At the bottom of the logic model are arrows indicating that funding, and in particular, duration of funding may impact a Healthy Start project’s ability to achieve short-, intermediate-, and long-term outcomes.

2. The intermediate and long term systems-based outcomes in the model are vague. For example, in the short term, "increased consumer participation" is a real and measurable outcome, but as a next-step intermediate outcome, it is not evident what "improved consumer voice" means or how it might be measured. Similarly, "improved efficiency of the service system" and "sustained community capacity to reduce disparities in health status in the target community" are not defined concepts. Can HRSA explain how these outcomes will be measured?

(Also see page 5, and 14-15 in the Supporting Statement)

The Logic Model for this evalaution sets out the inter-relationships of the service and systems components and potential outcomes. Improved maternal and child health outcomes are the primary focus for evaluating Healthy Start; it is not our intention to measure all areas of the Logic Model. Nevertheless, questions from the 2004 and 2011 Project Director Surveys will provide data to estimate many of the outcomes specified in the Model. Still, it is often difficult to collect “real” measures for intermediate and long-term systems-based outcomes in evaluations. Often, evaluations collect data that can be used to make inferences about achieved outcomes.

For constructs such as consumer participation, consumer voice, efficiency of the service system and sustained community capacity to reduce disparities in the target population, there are a number of questions throughout Sections 1, 5, 6, and 7 of the 2004 survey and Parts A (Section 1, Section 5); B (Section1, Section 2); C (Section1) and D (Section 1) of the 2011 survey that that will be used to define and measure these outcomes. For example, responses to questions about the use of former Healthy Start Participants as Program Staff and Peer Group Leaders for health education sessions, cultural competence of program staff , community participants serving as active members of the Consortium and Grantee reflections will be used to define consumer voice. Comparison of questions from the 2011 and 2004 surveys specifically related to the number, structure, purpose and active membership of Healthy Start Project Consortia will provide sufficient information to make an inference about the improvement of consumer voice.

Inferences about the "improved efficiency of the service system", or improved coordination of services, will be based on responses to questions in Part A Section 1 of the 2011 Project Director Survey that address the strategies used to raise community awareness of the Healthy Start project, recruit and retain participants, and processes for following up on completion of participant referrals for services.

With inferences about “sustained community capacity”, we expect that communities with active membership in Healthy Start Consortia and with former Healthy Start participants serving as program staff will create a local pool of individuals who are aware of the issues around infant mortality and the resources available to address those issues in their community. We can, therefore, infer that an expansion of this pool over time will lead to sustained community capacity to reduce disparities in health status in the target community. In addition, Project Directors’ responses to the Grantee Reflections sections in the 2004 and 2011 surveys will provide data to define and measure this outcome.

3. At one point (p. 13) the justification states that the analysis of the survey responses will involve comparisons of descriptive statistics about implementation "across grantees and across time." As this is a one-time evaluation, how will the agency compare percentages across time?

Comparisons across time will be based on data for the same variables or constructs from the 2004 and 2011 Project Director surveys. Note that the Project Directors Survey administered in 2004 reported on 2003 data and the 2011 Project Director Survey will report on 2009 data. Similar to the 2011 Project Director’s Survey, the 2004 Project Directors Survey, completed by 95 of the 96 grantees funded at that time, provided a “point-in-time snapshot” of the implementation of the Healthy Start program components. (See page 15 of Supporting Statement)

4. Every project director (or designee) is required to participate in an introductory webinar before the survey’s deployment. Is this obligation included in the burden estimate?

Participation in the introductory webinar can not be mandated such that attendance is contingent on survey completion. Every project director (or designee) will be asked to participate in an introductory webinar (strongly suggested). This suggested participation will be included in the burden estimate (see page 13 of Supporting Statement A).

5. Finally, the instrument must include a PRA statement, which ought to appear on the first screen of the web-based version.

A PRA statement will appear on the first screen of the web-based version of the survey instrument. Below is a sample statement that will be included on the first screen of the web survey:

The Paperwork Reduction Act requires approval of all federal government surveys by the Office of Management and Budget. This survey has been approved under this Act. The Office of Management and Budget control number and expiration date is available at your request. Additional information about this survey and its approval is available at your request.

3