Supporting Statement B

Supporting Statement B

Maternal Health Portfolio Evaluation

OMB Control No. 0906-XXXX-New

2. Collection of Information Employing Statistical Methods

1. Respondent Universe and Sampling Methods

No sampling will be used for the portfolio-wide evaluation of HRSA Maternal Health (MH) programs. HRSA staff who serve as Project Officers for the MH grant programs will participate in the evaluation as part of their jobs. All 15 MH grantees are expected to participate in the study. The partnership survey will be administered to all grantee partners that are identified by the grantees. The table below outlines the respondent groups, target response rates, and data collection strategies for each group.

Respondent Universe	Target Response Rates	Data Collection Strategies
Grantees	All grantees are expected to participate in the evaluation as part of their grant. A response rate of 100 percent is expected.	Semi-structured interviews (by phone or in-person) Web-based data collection tool
HRSA Project Officers	HRSA Project Officers oversee the MH grantees for the agency. A response rate of 100 percent is expected.	Semi-structured phone interviews
Partners	The respondent universe will include each grantee’s partners. Each grantee has between 5-40 partners. Partners include organizations who are part of the State MHI state taskforces, the RMOMS provider networks, AIM’s partner cohorts, AIM-CCI’s national and local maternal safety bundle workgroups, and Supporting MHI’s technical assistance center. A response rate of 80 percent is expected.	Web-based survey

Given the diversity among the MH grantees, and the small number of MH grantees, the evaluation team determined that sampling could not ensure a representative sample and would not likely improve the robustness of data gathered. Additionally, there are no reasonable criteria for purposefully sampling among the programs. Surveying the entire universe is needed to ensure representative information.

2. Procedures for the Collection of Information

There are four data collection tools that will be used for this study. Below we describe the procedures for collecting both qualitative and quantitative data for this study.

Semi-structured interviews with HRSA Project Officers. Project Officers will be invited to participate in the study via email (Attachment C1). The interviews will be conducted annually by telephone and will not exceed 90 minutes. Interviews will be consistently facilitated by the same senior evaluation staff each year to encourage continuity and a deep understanding of the program. Interviews will be recorded and transcript-style notes will be developed to capture as much context and detail as possible.

Semi-structured interviews with grantees. Interviews will be conducted with grantee staff on an annual basis. Interviews will not exceed 60 minutes with grantee staff. Interviews will be consistently facilitated by the same senior evaluation staff each year to encourage continuity and a deep understanding of the program. Interviews will primarily be conducted via telephone annually in order to document important factors over the full course of the program period, but one set of annual interviews will be conducted in-person during site visits with HRSA Project Officers. Each grantee will be visited one time by the evaluation team. Grantee staff will be invited to participate in interviews via email; invitation letters will be tailored for each grantee depending on whether the evaluation team will be on the site visit that year (Attachment C2). Interviews will be recorded and transcript-style notes will be developed to capture as much context and detail as possible.

Tailored web-based data collection tool. Data will be collected from grantees using tailored web-based data collection tools. The tool will be tailored for each grantee based on their program activities. This data collection will occur annually in order to collect data on the implementation and effectiveness of activities. Grantees will be sent an invitation to complete the data collection tool (Attachment C3) and asked to provide the data within one month. Training on the tool will be provided to grantees to ensure high quality data collection. Data will be reviewed for completeness and accuracy. In the event of missing or infeasible data, the grantee will be asked to clarify and make any necessary corrections to the data.

Partnership survey. The partnership survey will be conducted once during year 3 of the evaluation. Grantees and partners will receive an invitation to complete the survey (Attachment C4) and be asked to complete the survey within one month. Grantees will be asked to provide input on the survey period to ensure it does not interfere with partnership work plans and milestones. The survey takes 15 minutes to complete.

3. Methods to Maximize Response Rates and Deal with Nonresponse

Semi-structured interviews with HRSA Project Officers. The study team will contact the HRSA Project Officers via email to schedule interviews. If no response, the study team will send two follow up emails. If a Project Officer does not respond to the study team, the HRSA staff overseeing this study will follow up directly with their colleagues to request their participation.

Semi-structured interviews with grantees and Web-based data collection tool. The study team will invite the grantees to participate and send follow-up reminders. If grantees do not respond, HRSA will encourage participation as it is part of their requirements for receiving funding.

For interviews, the study team will contact the grantees via email to schedule interviews. Two follow up emails will be sent in the event of nonresponse by grantee staff. If grantees do not respond to the study team, the HRSA Project Officers overseeing each grant will follow up directly with the grantee staff to request their participation.

The web-based data collection tool is designed to minimize burden and increase response. Data will be collected through a user-friendly web-based platform and the tool will be tailored so that it only includes questions that apply to their programs.

If a grantee does not respond to our requests for data collection by the requested deadline, the study team will follow-up with the grantee to request their participation. If they still do not respond, HRSA will contact the grantee and request that they respond.

Because the survey will be tailored to each of the 15 grantees, there should be no missing responses on the survey. However, the study team will review the submitted data from each grantee and follow-up with the grantee if they did not complete any of the required survey items. The study team will offer technical assistance to grantees should grantee staff have questions about how to complete this tool

Partnership survey. The study team will email a link to the survey to all partners. Two reminder emails will be sent by the study team to respondents who have not yet completed the survey. The study team will ask grantees to follow up with their partners to encourage participation and completion of the survey.

4. Tests of Procedures or Methods to be Undertaken

The partnership survey is primarily comprised of a previously-validated survey and does not need to be pilot tested. The interview protocols and web-based data collection tool will not be pilot tested until after OMB approval, as only minor edits are expected.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Inquiries regarding the statistical aspects of the design should be directed to:

Theresa Chapple-McGruder

HRSA/MCHB/DHSPS

5600 Fishers Lane

Mail Stop 18N90A

Rockville, MD 20857

301-594-4421

The following individuals have consulted on the design:

Britta Anderson, Ph.D.

NORC at the University of Chicago

4350 East West Highway, 8th Floor

Bethesda, MD 20814

301-634-9356

Erin Ewald, M.Sc.

NORC at the University of Chicago

4350 East West Highway, 8th Floor

Bethesda, MD 20814

301-634-9359

LCDR Monique Richards

HRSA/MCHB/DHSPS

5600 Fishers Lane

Mail Stop 18N90A

Rockville, MD 20857

(301) 443-3883

The data will be collected and analyzed by an evaluator that will be contracted by HRSA.

3