Supporting Statement B

Supporting Statement B

Title V Maternal and Child Health (MCH) Block Grant Jurisdictional MCH Survey Instrument

OMB Control No. 0915-0379

B. Collection of Information Employing Statistical Methods

1. Respondent Universe and Sampling Methods

Respondent Universe

The respondent universe for the Pretest is women age 18 or older who live in one of the eight U.S. jurisdictions (Puerto Rico, U.S. Virgin Islands, Guam, Northern Mariana Islands, American Samoa, Palau, Marshall Islands, or Federated States of Micronesia) and who are mothers or guardians of at least one child aged 0-17 years living in the same household.

Sampling Methods

NORC anticipates completing 25 interviews per jurisdiction, for a total of 200 completed interviews. The sampling method will differ for the five jurisdictions with in-person data collection (Northern Mariana Islands, American Samoa, Palau, Marshall Islands, and Federated States of Micronesia) and the three jurisdictions with telephone data collection (Puerto Rico, U.S. Virgin Islands, and Guam). If respondent concern regarding using cell-phone minutes in Guam, where pre-paid cell-phones are more common than in Puerto Rico or the U.S. Virgin Islands, results in difficulty completing 25 interviews, then data collection in Guam will be converted to in-person mode.

The target sample for the In-Person mode is 125 completed interviews, divided evenly across the five in-person-mode jurisdictions (25 completes in each jurisdiction). Participants will be recruited through flyers posted in public areas (e.g., shopping malls, telephone poles, community boards, bus stops), as well as by word of mouth. Respondents for this component will not be selected through a random process, but rather will be selected for specific characteristics, such as their child’s health status or age. For the purposes of this study, a randomly drawn sample is not required to assess the reliability, validity, and clarity of the survey instruments. Interviewers will be instructed to recruit individuals in different locations and through different modes of introduction (e.g., through a hospital, door-to-door knocking, etc.). In each of the jurisdictions, for both phone and in-person data collection, NORC will attempt to obtain:

• At least five completes in each child age category: 0-5, 6-11, 12-17

• At least two Children with Special Health Care Needs completes

• At least one complete in each language offered in each of the jurisdictions.

Note that these are guidelines and actual respondent composition may vary depending on factors that occur in the field.

The target sample for the Telephone mode is 75 completed interviews, divided evenly across the three telephone-mode jurisdictions (25 completes in each jurisdiction). To obtain these completed interviews, NORC will utilize a random-digit-dial (RDD) sample of cell phones in each of these jurisdictions. A sample of approximately 17,577 cell-phone numbers will be drawn across the three jurisdictions, plus a 200% buffer for a total of 52,731 cell-phone numbers. The samples will be randomly selected from cell-phone sampling frames constructed by Marketing Systems Group (MSG) consisting of all cell-phone numbers in active telephone exchanges. The sample of phone numbers will yield approximately 764 completed screeners to identify respondents with children, from which the 75 completed interviews will be obtained.

The number of completed screeners needed (see Table 1) was determined based on assumptions for the eligibility rate and interview completion rate in each jurisdiction. See further description of assumptions used to inform this plan below.

Table 1: Mode of Survey Data Collection and Target Number of Respondents

Title V Jurisdictions	Mode of Data Collection	Telephone Sample Size/Sample Plus 200% Buffer	Anticipated Screener Completes Needed	Target Number of Respondents
Puerto Rico	Telephone	4,678/14,035	263	25
USVI	Telephone	6,334/19,003	309	25
Guam	Telephone	6,565/19,964	192	25
American Samoa	In-Person	n/a	156	25
Federated States of Micronesia	In-Person	n/a	156	25
Marshall Islands	In-Person	n/a	156	25
Northern Mariana Islands	In-Person	n/a	169	25
Palau	In-Person	n/a	156	25
Total		52,731	1,557	200

Further Description of Assumptions used in Table 1

For the Telephone mode (Guam, Puerto Rico, U.S. Virgin Islands) we are targeting 25 survey completes. The following formula was used to calculate the number of screener completes to reach this goal:

In this formula, “eligibility rate” is the proportion of screener completes that will be eligible (i.e., the cell phone must be used by an adult and the adult’s household must contain a child under age 18) and “interview completion rate” is the proportion of identified eligible respondents that will go on to complete the interview. Next, to convert screener completes into the number of sampled telephone numbers needed, we used the following formula:

In this formula, “resolution rate” is the proportion of dialed numbers we are able to resolve as either (1) active personal cell-phone number, (2) non-working number, or (3) business number; “APCN rate” is, out of the resolved numbers, the proportion that were resolved as active personal cell-phone numbers; and “screener completion rate” is, out of the identified active personal cell-phone numbers, the proportion that that go on to complete the screener.

This is a new data collection, so rates we informed by prior similar experience (see Table 2). Assumptions for the resolution rate and APCN rate are based on previous experience with a large, telephone-based, federal survey in Guam, Puerto Rico, U.S. Virgin Islands. The assumptions for the screener completion rate and interview completion rate were based on experience with a mode effects experiment of a similar children’s health survey. The assumptions for the eligibility rate were based on previous experience with this survey, in conjunction with the relative eligibility rates from a large, telephone-based, federal survey in Guam, Puerto Rico, U.S. Virgin Islands.

Table 2: Mode of Survey Data Collection and Target Number of Respondents

Cell-Phone Sample	Puerto Rico		U.S. Virgin Islands		Guam
	Factor	Count	Factor	Count	Factor	Count
Sample with 200% buffer	--	14,035	--	19,003	--	19,694
Required Sample of Telephone Numbers	--	4,678	--	6,334	--	6,565
Resolved Phone Numbers – Resolution Rate	45.0%	2,105	65.0%	4,117	45.0%	2,954
Households Identified – Active Personal Cell Phone Number Rate	25.0%	526	15.0%	618	13.0%	384
Households Screened – Screening Completion Rate	50.0%	263	50.0%	309	50.0%	192
Adult Cell-Phone Rate	95.0%		92.0%		93.0%
Household Has Child Under Age 18	25.0%		22.0%		35.0%
Overall eligible HHs – Eligibility Rate	23.8%	63	20.2%	63	32.6%	63
Completed interviews – Interview Completion Rate	40.0%	25	40.0%	25	40.0%	25

Because these rates are assumptions, it is a best practice to anticipate possible error and draw larger samples of telephone numbers in case there is a need to release more sample than planned. If all assumptions are correct, or underestimates, than this buffer sample will not have to be utilized.

For the In-Person mode, the number of survey completes and the formula used to calculate screener completes is the same as for telephone mode.

An interview completion rate of 40% was chosen based on experience with a mode effects experiment of a similar children’s health survey. For the in-person jurisdictions, it is likely that through the flyers and word of mouth recruitment we will identify eligible survey respondents faster, requiring fewer screener completes; therefore we assumed a somewhat higher eligibility rate for the In-Person mode (40%) than for the Telephone mode.

For both modes, in each jurisdiction, recruitment and screening will only last as long as is necessary to complete the 25 interviews.

2. Procedures for Collection of Information Collection

For the Telephone mode, data will be collected from respondents via a combination of Computer Assisted Telephone Interview (CATI) software and Paper and Pencil Interviews (PAPI).

The CATI screener asks respondents to verify that they are a member of the household age 18 or older; respondents who meet these eligibility criteria will be asked questions regarding the number of children in the household; the ages, gender, and special-health-care-needs status of each child; as well as the respondent’s level of comfort with English or Spanish (additional languages will be available for in-person data collection).

Once the screener is completed for an eligible household, the interviewer will administer the main questionnaire using PAPI. One child who is 17 years of age or under per household will be selected to be the subject of the main questionnaire; the interviewer will have leeway to choose the child to ensure a mix of interviews by child age and special-needs status. A topical survey will be administered for each selected child and will cover the following content areas: demographic information; child’s health and functional status; health insurance coverage; health care access and utilization; medical home; early childhood; middle childhood and adolescence; family functioning; parental health; and health insurance experience. Following the topical survey, the Jurisdiction-specific survey will be administered, asking questions specific to that jurisdiction.

With the respondent’s permission, screeners and interviews will be recorded to allow for review and validation of responses.

For the In-person mode, the topical survey, and the Jurisdiction-specific survey data will be collected from respondents using the PAPI method described in the telephone mode section. For the in-person mode, the screener will also be conducted on paper. For respondent recruitment, NORC will work with each jurisdiction to tailor a plan for recruiting respondents based on the sample guidelines described in section B1. Pretest data collection will occur in more populated areas that are less difficult to reach, if travel to villages or islands is not feasible. Based on information collected through informational interviews with experts in the jurisdictions, NORC has identified the initial approaches to recruitment (Table 3).

Table 3: Considerations for Recruiting Respondents

Jurisdiction	Considerations
American Samoa	Recruiting pilot test respondents through churches would be very effective. Different groups that exist in the villages (such as women’s groups) can also assist with recruiting pilot respondents. Recruitment at health centers is also an option.
Federated States of Micronesia	State hospitals and public health programs could also be helpful partners for recruitment.
Marshall Islands	In recruiting respondents for the pilot, it is possible to go door-to-door, although it may be easiest to recruit at the hospital. It would be best to include residents of the outer islands although travel to the islands would be difficult.
Northern Mariana Islands	Experts indicated that recruiting at grocery stores and clinics could work. As needed, we will reach out to jurisdictional leadership to find local leads to endorse the survey and to identify appropriate locations for interviewers to recruit respondents.
Palau	Plan to reach out to the bureau of public health to request advice in recruitment of respondents.

NORC will ensure that all needed supplies, including copies of the screener and surveys, will be shipped to each interviewer. NORC will provide each interviewer with a project laptop and scanner to allow for secure transmission of the screener and survey data. Once the data has been transmitted and verified as received by NORC’s Central Office, the interviewer will destroy the paper copies.

For both modes, all interviewers will be trained on informal cognitive probing regarding the comprehension, skip patterns, and wording of the survey instrument, with an emphasis on the jurisdiction-specific questions. Interviewers will be given a set of questions that help assess each respondent’s ability to answer the questions and a standardized form for recording this feedback. Once data collection is complete, NORC will hold a debriefing phone call with each interviewer to capture additional information regarding their experience administering the instrument. All interviewers will also have undergone human subjects protections training and have signed a confidentiality agreement prior to conducting interviews.

3. Methods to Maximize Participation Rates and Deal with Nonresponse

In designing the Jurisdictional MCH Survey Instrument, attention will be placed on the following design elements to help increase cooperation by prospective respondents:

• In developing and refining specific questions, the goal will be to create a logical, clear questionnaire with concrete question wording and simple grammar

• Questions will be grouped according to subject areas

The experience of the experts from CDC, jurisdictional leads, and other organizations indicates that the Jurisdictional MCH survey for American Samoa, Marshall Islands, and Palau will need to be fielded in English and at least one of the multiple local languages. In Guam, Northern Mariana Islands, and Federated States of Micronesia, the survey will be administered in English. In Puerto Rico and USVI, the survey will be fielded in both English and Spanish.

During review of the OMB package, the jurisdictions will have the opportunity to review and provide feedback on the translations before data collection begins. This may result in small changes in wording that enhance clarity, but do not change fundamental survey content.

Table 4: Languages

Title V Jurisdictions	Languages
Puerto Rico	English, Spanish
USVI	English, Spanish
Guam	English
American Samoa	English, Samoan
Federated States of Micronesia	English
Marshall Islands	English, Marshallese
Northern Mariana Islands	English
Palau	English, Palauan, and Tagalog

4. Test of Procedures or Methods to be Undertaken

This is a pretest to evaluate the screener and survey for comprehension, skip patterns, and accurate wording prior. Feedback from this pretest will be incorporated into the final version of the study questionnaires. The estimates of time burden presented in Part A of the Supporting Statement were generated from usability testing conducted with four English-speaking mothers with children ranging in age from 0-15 years old.

Items included in the screener and surveys were taken from validated, national surveys including:

• Behavioral Risk Factor Surveillance System (BRFSS)

• National Immunization Surveys (NIS)

• National Survey of Children's Health (NSCH)

• National Survey of Children with Special Health Care Needs (NS-CSHCN)

• Pregnancy Risk Assessment Monitoring System (PRAMS)

• Youth Risk Behavior Surveillance System (YRBSS)

5. Statistical Consultants

The contractor, NORC, will collect the information on behalf of MCHB. Contact information for NORC’s principal staff on the project is listed below:

Anna Wiencrot

Project Director

55 East Monroe Street, 30^th Floor

Chicago, IL 60603

Phone: (312) 357-3754

Email: [email protected]

Michael Stern PhD

Technical Expert

55 East Monroe Street, 31^st Floor

Chicago, IL 60603

Phone: (312) 357-3891

Email: [email protected]

7