OMB Part B WICPeerC_11-30-11

Cambridge, MA

Bethesda, MD

Chicago, IL

Durham, NC

Hadley, MA

Lexington, MA

WIC Breastfeeding Peer Counseling Study: Phase 2

Supporting Statement for Paperwork Reduction Act Submission

Part B: Statistical Methods

OMB# 0584-0548

Expiration Date:

July 31, 2011

Draft

May 13, 2011

Prepared for

Joseph Robare

U.S. Department of Agriculture

Food and Nutrition Service

3101 Park Center Drive

Alexandria, VA 22302

Prepared by

Abt Associates Inc.

Contents

PART B: STATISTICAL METHODS 2

B.1 RESPONDENT UNIVERSE AND SAMPLING METHODS 2

B1.1 Respondent Universe And Selection Of Local WIC Agencies 2

Eligibility and Requirements for Local WIC Agency Participation 3

Recruitment of Local WIC Agencies 5

B1.2 Demonstration Period 7

B1.3 Informed Consent And Enrollment Of WIC Peer Counseling Participants 8

B1.4 Random Assignment 11

Random Assignment and Maintaining Fidelity 14

B1.5 Sample Sizes 16

B1.5 Response Rates 20

B.2 PROCEDURES FOR THE COLLECTION OF INFORMATION 22

B2.1 Statistical Methodology For Stratification And Sample Selection 22

B2.2 Estimation Procedure 22

Descriptive Analyses 23

Process Data Analyses 24

Estimation of Impacts 25

B2.3 Degree Of Accuracy Needed For The Purpose Described In The Justification 29

B2.4 Unusual Problems Requiring Specialized Sampling Procedures 29

B2.5 Use Of Periodic Data Collection Cycles To Reduce Burden 29

B.3 METHODS TO MAXIMIZE RESPONSE RATES AND DEAL WITH NON-RESPONSE 29

B3.1 Maximizing Response Rates Among LWA Staff 29

B3.2 Maximizing Response Rates Of WIC Peer Counseling Participants 29

B3.3 Methods To Account For Non-Response 31

Overall Non-Response 31

Differential Non-Response Between the Treatment and Control Groups 32

B.4 TESTS OF PROCEDURES OR METHODS TO BE UNDERTAKEN 32

B4.1 Surveys Of WIC Participants 32

B4.2 Interviews With LWA Staff 33

B.5 INDIVIDUALS COLLECTING AND ANALYZING DATA AND CONSULTED ON STATISTICAL ASPECTS 33

REFERENCES 34

Part B: Statistical Methods

Part B of the justification for this information collection activity, Phase 2 of the WIC Breastfeeding Peer Counseling Study, addresses the five points outlined in Part B of the OMB guidelines. We begin with a brief overview of the study; more detail is contained in Part A: Justification.

B.1 Respondent Universe and Sampling Methods

Describe (including a numerical estimate) the potential respondent universe and any sam­pling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corre­sponding sample are to be provided in tabular form for the uni­verse as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection

In this section, we describe the procedures used to recruit and select local WIC agencies and individual WIC peer counseling participants into the study, including:

1. The respondent universe and selection of local WIC agencies

2. Procedures for the Demonstration Period

3. Informed consent and enrollment of WIC breastfeeding peer counseling (WIC-BPC) participants into the study;

4. Random assignment of WIC-BPC participants into treatment or control groups

5. Sample sizes; and

6. Expected response rates

B1.1 RESPONDENT UNIVERSE AND SELECTION OF LOCAL WIC AGENCIES

The respondent universe is the subset of the 1,810 Local WIC Agencies (LWAs) in the 50 United States and the District of Columbia that have an active Loving Support Peer Counseling Program (n = 675) and the pregnant WIC participants served by these 675 agencies (approximately 512,000 per month).¹ Volunteer LWAs with Loving Support Peer Counseling Programs will be invited to respond to a Funding Opportunity Request (FOA) issued by the evaluation contractor (see Appendix F4) to participate in the study, with the costs of participation defrayed by grant funding. Up to eight volunteer LWAs will be selected to participate in the study, with the exact number depending on the nature and capacity of the LWAs that apply. Specifically, the selected sites must be able collectively to enroll a sufficient sample of first-time expectant WIC participants to support the study design (the WIC participant sample is described below under B1.5).

Eligibility and Requirements for Local WIC Agency Participation

LWAs will be eligible to participate in the study if they have an operational and stable Loving Support Peer Counseling Program—but one that is not already implementing the enhanced approach this evaluation is designed to test. That is, to ensure that the control condition is distinct from the intervention, only LWAs that do not currently provide the two components of the intervention to WIC participants receiving breastfeeding peer counseling are eligible to participate in the study. In addition, LWAs will be prioritized to the degree to which they include the following characteristics:

• They serve areas with high population density—such as, but not restricted to, urban areas—where distances traveled are less likely to impose a barrier to implementing in-person, post-partum contacts between WIC peer counseling participants and peer counselors.

• They serve populations that have historically tended to have low breastfeeding rates; and

• Their Loving Support Peer Counseling Programs are large enough to enroll a sufficient number of first-time expectant mothers (determined by the number of participating LWAs relative to the total sample size needed) in the study and provide the enhanced Loving Support Peer Counseling services to approximately half of them.

LWAs will be selected based on responses to the FOA that describe feasible plans to ensure that:

• Peer counselors can implement the intervention successfully;

• The caseload of each peer counselors participating in the study will include roughly 50 percent women in the treatment group and 50 percent in the control group; and

• WIC-BPC participants in the control group do not receive the intervention.

LWAs selected on the basis of their response to the FOA will first be required to demonstrate in a Demonstration Period that they can implement the intervention. That is, before WIC-BPC participants are enrolled in the study and randomly assigned, LWAs will first implement peer counselor contacts with women in the hospital and during the first 10 days post-partum. LWAs that can complete these contacts with a target number of WIC-BPC participants during the two-month Demonstration Period will enter into full study implementation and begin enrolling first-time expectant mothers into the study.²

The purpose of the study is to test the efficacy of the intervention in LWAs that have the strongest likelihood of implementing the intervention with fidelity (not to test its impact on a nationally representative sample of LWAs). Because these enhancements include an in-person meeting with a peer counselor during the first 10 days post-partum, experts consulted during the design of the study recommended that participating LWAs be located in areas with relatively high population density (such as urban areas) to minimize the distances that WIC participants and peer counselors would need to travel to this in-person meeting.

In addition, to maximize the potential impact of the intervention, experts recommended that the study focus on populations with historically lower breastfeeding rates. These populations include non-Hispanic Black, non-Hispanic white, and those in the East South Central, and East North Central and West North Central states (Li et al., 2005). However, to ensure face validity, effort will be made to include in the study at least one volunteer LWA from each of four US Census regions—the Northeast, South, Midwest, and West; these account respectively for 13, 41, 19, and 27 percent of pregnant WIC participants. In addition, experts recommended that the study focus on LWAs the greatest likelihood of successfully delivering the enhanced Loving Support Peer Counseling services to the treatment group. Local WIC agencies will be recruited for the study using a promotional campaign and a Funding Opportunity Announcement that clearly describe these study requirements.

Recruitment of Local WIC Agencies

As mentioned in the study overview (see Part A), up to eight LWAs will participate in the study. Only LWAs that do not currently provide the two enhancements to the Loving Support Peer Counseling program are eligible to participate in the study. To support the sample size needed for the study; these participating LWAs must be able, collectively, to enroll 1,800 WIC-BPC participants, 900 to be assigned to each condition (treatment and control).³ Assuming that the study enrollment period lasts six months and there are eight LWAs in the study, each LWA must enroll an average of 225 women in the study, which equals 38 first-time expectant mothers per LWA per month.⁴ In addition, participating LWAs must have the capacity in their peer counseling program to provide the enhanced Loving Support Peer Counseling services to half of enrolled WIC-BPC participants in the first week post-partum – while maintaining their provision of the standard peer counseling services to women assigned to the control condition. Recruitment of LWAs will require the following:

Study Announcement and Informational Webinars:

The evaluation contractors will send State and local WIC agencies a letter (Appendix F1) announcing the upcoming study (pending OMB clearance) with information about eligibility criteria and requirements for LWA participation. This letter will include information about attending one of three optional information webinars that the evaluation contractor will host for interested State and local WIC agencies (the evaluation contractor will establish a URL for webinars after OMB clearance is obtained).

WIC Participant Characteristics (2010) data: Extant WIC PC data will be used to compile a list of the 250 largest LWAs in the country for whom contact information is available.⁵ In addition to asking State WIC agencies to disseminate the FOA to their local WIC agencies, the FOA will be sent directly to these 250 largest LWAs (although there is no agency size requirement to apply for the study). It is anticipated that the larger LWAs may have the strongest chances for a successful implementation of the intervention. According to WIC PC 2008, these 250 agencies range in size from approximately 36,000 to 1,000 pregnant WIC participants (USDA, 2010).

Funding Opportunity Announcement (FOA): The evaluation contractor will disseminate an Funding Opportunity Announcement (FOA; see Appendix F2) to State and local WIC agencies that includes a description of the study, eligibility criteria and requirements for participation, size of the grant amounts available (conditional on the number of WIC-BPC participants enrolled) and the criteria that will be used to prioritize and select LWAs to participate. Applicants will be asked to provide some general information about their existing Loving Support Peer Counseling Program, their proposed approach to implementing the specified enhancements, and the number of WIC-BPC participants they plan to enroll in the study over a six month period. It is expected that the response to the FOA will be between 8-10 pages in length.

Responses to the FOA will be evaluated based on LWAs’ eligibility for participation and ability to meet the requirements for the study. The evaluation contractor will review responses from all eligible applicant LWAs and select those that best meet the evaluation criteria listed in the FOA (Appendix F2) and select a group of eight LWAs to conduct a Demonstration Period of the intervention.

LWAs selected on the basis of their responses to the FOA will be asked to sign Memoranda of Understanding (MOUs) (a sample MOU is included in Appendix F3) that outline the roles and responsibilities of the LWAs and of the evaluation team. This MOU will include an agreement by the LWA that peer counselors will not provide the enhancements (i.e., the intervention) to those WIC Participants assigned to the control condition. Each LWA will be asked to designate a point of contact and the evaluation team will assign a study liaison to each LWA. The study liaison will work with the LWA’s point-of-contact throughout the study (including the Demonstration Period period) to provide technical assistance and to coordinate and support enrollment and random assignment and scheduling of site visits.

To offset the costs of undertaking the intervention and participating in the study, LWAs will receive a grant pro-rated to the number of WIC-BPC participants that they enroll in the study. The evaluation contractor will give grantees approximately $175 for each WIC peer counseling participant in the treatment group to whom peer counselors successfully deliver the enhanced Loving Support Peer Counseling services.⁶

B1.2 DEMONSTRATION PERIOD

LWAs selected from the pool of applicants will conduct a two-month Demonstration Period of the intervention during which peer counselors will attempt to complete hospital and post-partum contacts with a specified target number of WIC-BPC participants who give birth during this period.⁷ Upon the signing the MOU, each LWA will receive grant funding to offset the costs of the Demonstration Period period, ($175 for each WIC peer counseling participant in the Treatment condition to whom peer counselors successfully deliver the enhanced Loving Support Peer Counseling services). During the Demonstration Period, LWA staff will submit a biweekly Demonstration Period Progress Form (Appendix C1) with information about the “take-up” rate for the intervention (i.e., the number of women offered the intervention’s enhanced services and the number of these women who accepted the offer), and selected information from peer counselor contact logs to verify the number of women who actually received the intervention’s enhanced services, relative to the LWA’s target number. In addition, site visitors will interview LWA staff (the Peer Counseling Coordinator, with assistance as needed from the LWA Director, Breastfeeding Coordinator, and an executive assistant or other individual in charge of maintaining LWA budget and administrative records) during a one-day on-site visit to the agency. This site visitor will also conduct training for LWA staff on enrollment and random assignment procedures anticipated for the impact study.

LWAs with a successful Demonstration Period will enter the impact study period. If an LWA is unsuccessful at meeting its target, the research team will ask the LWA to identify what the barriers were and to provide plans for addressing them in order to be able to implement the intervention with fidelity. FNS, with advice from the evaluation team, will assess whether to terminate the LWA’s participation in the study or to extend the Demonstration Period period for an additional two months and offer technical assistance with the implementation of the intervention. If it is determined that the LWA should no longer be in the study, then the study will stop at that LWA and we will work with more successful LWAs to see if it is possible to increase their samples, either by increasing the length of the intervention period or the number of WIC participants per month who are enrolled in the study. Unspent funds from the site where the study ended will be redistributed to these LWAs.

B1.3 INFORMED CONSENT AND ENROLLMENT OF WIC PEER COUNSELING PARTICIPANTS

Clearance is being sought for 1,800 WIC peer counseling participants (WIC-BPC participants) who are first-time expectant mothers to participate in the study. First-time expectant mothers are the targeted population because the peer counseling program has the potential to have the greatest impact on this group, rather than mothers who have had previous children, who may be inclined to use the same infant feeding practices they used for their previous children.⁸ Before describing the rationale for this sample size, we give a brief overview of the informed consent and enrollment procedures.

The study will enroll WIC-BPC participants throughout a six-month enrollment period. Staff at each of the participating LWAs will invite pregnant WIC-BPC participants to enroll in their breastfeeding peer counseling program. Next, from among those WIC participants who enroll in the peer counseling program, LWA staff will invite first-time expectant mothers who are aged 18 or older. A minimum age of 18 years is necessary to ensure that study participants are capable of providing informed consent without prior parental permission (85 percent of pregnant WIC participants are aged 18 or older; see USDA, 2010, Exhibit 2.3).

All age-eligible first-time expectant mothers who sign up for peer counseling will be invited to participate in the study.⁹ However, because the number of months’ pregnant at time of enrollment determines the number of months’ delay until any woman in the treatment group will receive the enhanced Loving Support peer counseling services (i.e., at delivery and during the first week post-partum), women will be invited to participate in the study only once they have reached their fifth, sixth, or seventh month of pregnancy.¹⁰ This invitation to the study will occur during a WIC participant’s in-person visit to a WIC clinic within the LWA. This group of women will include:

• Women who first certify to receive WIC benefits during their second (5th or 6th month) or third (7th month) trimester; and

• Women who had first certified to receive WIC benefits during their first four months of pregnancy but who are returning to the WIC clinic for additional vouchers, nutritional classes, or other WIC services.¹¹

For women assigned to the treatment condition, the top panel of Exhibit B2 illustrates when random assignment (RA), birth (B), delivery of the Enhanced Loving Support Peer Counseling Program (ELS) peer counseling services, and the follow-up survey would occur for women who enrolled in Study Month 1. For example, a woman in her first month of pregnancy in Study Month 1 would give birth and receive the intervention in Study Month 9; her follow-up survey would be completed in Study Month 11 or 12. Notice that women who are 9-months pregnant during Study Month 1 cannot be randomly assigned before they give birth (B); as a result, these women cannot be enrolled in the study. Likewise, it would be difficult to complete random assignment prior to birth for women who are 8-months pregnant at the time they might otherwise enroll in the study.

The bottom panel of Exhibit B2 shows what happens if the study were to enroll women who were one, two, three, or four months pregnant at Study Month 6: the duration of the study increases: women 1-month pregnant in Study Month 6 would not give birth until Study Month 14 – extending the study several months to allow for delivery of peer counseling and follow-up data collection.

As a result, the timing of enrollment in the study relative to the number of months a woman has been pregnant is a crucial consideration. Although it is possible during Study Months 1, 2, 3 and 4 to enroll women who were 1 to 7 months pregnant and still complete all necessary study activities within one-year, starting in Study Month 5 participating local WIC agencies would have to restrict enrollment to women who were at least 4 months (and no more than 7 months) pregnant; in Study Month 6 this restriction would narrow again to women 5-7 months pregnant. Because this procedure is overly burdensome for LWAs, the study will time enrollment into the study so that women are invited to participate when they reach their fifth month of pregnancy.

All women invited to the study will receive a study brochure (Appendix F5) that describes the purpose of the study, the random assignment process, what she will be asked to do as a study participant (i.e., participate in the baseline and follow-up surveys), and an explanation of the two types of Loving Support Peer Counseling services she would receive depending on her study assignment.

At the time of enrollment, each WIC-BPC participant will be assigned a peer counselor. For each WIC-BPC participant enrolled, LWA staff will complete a Study Enrollment Form. If a WIC-BPC participant declines to participate in the study, she will be asked to complete a brief Decline Form that will allow the study to collect limited data needed for any necessary non-response bias analyses. It is expected that all peer counselors who participate in the study will have caseloads that include women in the treatment group, the control group, and women who are not enrolled in the study.

To obtain informed consent for all study activities, each WIC-BPC participant invited to the study by the LWA will be asked to read and sign a form giving their consent to participate in the study (Appendix F6). This consent form (Appendix F6) specifically describes the random assignment procedure to be used in the study and will be reviewed by the evaluation contractor’s Institutional Review Board to ensure that it meets the requirements for obtaining informed consent from human participants in research. In addition, verbal consent from each respondent will also be requested prior to the Baseline and Follow-up Surveys.

B1.4 RANDOM ASSIGNMENT

Random assignment will occur on a rolling basis throughout the six-month enrollment period and will be conducted within each participating LWA. Upon completion of the baseline survey, or the passing of 30 days from the date of enrollment (whichever comes first), each study participant will be randomly assigned to either the treatment (the enhanced Loving Support Peer Counseling Program) or control (the LWA’s standard Loving Support Peer Counseling Program) condition.¹² Following the study design, 50 percent will be assigned to the treatment and control conditions each. The assignment status for each participant will be communicated back to the LWA so that the peer counseling services appropriate to her assignment can be implemented.

Exhibit B2: Occurrence of study-relevant events for participants in the Treatment Group who are 1 to 9 months pregnant at enrollment (E). E = Enrollment, RA = Random Assignment, B = Birth, ELS = Enhanced Loving Support peer counseling delivered, F = Follow-up survey

Study Month

# of months pregnant at enrollment (E)

1

2

3

4

5

6

7

8

9

10

11

12

1 months pregnant

E

RA

B, ELS

F

2 months pregnant

E

RA

B, ELS

F

3 months pregnant

E

RA

B, ELS

F

4 months pregnant

E

RA

B, ELS

F

5 months pregnant

E

RA

B, ELS

F

6 months pregnant

E

RA

B, ELS

F

7 months pregnant

E

RA

B, ELS

F

8 months pregnant

E

B

9 months pregnant

E, B



First month of study enrollment period

Final month of study enrollment period



1

2

3

4

5

6

7

8

9

10

11

12

13

14

1 months pregnant

E

RA

B, ELS

2 months pregnant

E

RA

B, ELS

3 months pregnant

E

RA

B, ELS

F

4 months pregnant

E

RA

B, ELS

F

5 months pregnant

E

RA

B, ELS

F

6 months pregnant

E

RA

B, ELS

F

7 months pregnant

E

RA

B, ELS

F

8 months pregnant

E

B

9 months pregnant

E, B

The evaluation contractor, Abt Associates, will use the probability sampling procedures in SAS SURVEY SELECT to make the random assignment. The advantage of this approach is that the SAS program log serves as clear documentation of the correct implementation of the random assignment process. A similar approach can also be used if the eligible households are listed in a spreadsheet.

Implementing Random Assignment

To determine the effects of this intervention –namely, the enhanced Loving Support Peer Counseling Program—on breastfeeding outcomes, we considered three approaches to implementing random assignment:

• Randomly assign local WIC agencies to implement either the enhanced Loving Support Peer Counseling Program or their “business-as-usual” Loving Support Peer Counseling Program; or

• Randomly assign Peer Counselors to deliver either the enhanced Loving Support Peer Counseling services or their agency’s “business-as-usual” Loving Support Peer Counseling services; or

• Randomly assign WIC peer counseling participants to receive either the enhanced Loving Support Peer Counseling services or their local WIC agency’s “business-as-usual” Loving Support Peer Counseling services.

We determined that the third approach, randomly assigning participants, was the only feasible option for a number of reasons. We determined that randomly assigning LWAs was not feasible. First, this project’s Phase 1 implementation study found that there was wide variation in the ways in which the Loving Support Peer Counseling Programs is implemented in terms of modes of contact, training of peer counselors, and levels of intensity. Given the level of variation in the “control” condition, the number of LWAs needed to implement a group randomized design was well beyond the resources available for the study. To detect an MDD of 6 percentage points using this design, we would need to increase the number of participating LWAs from eight to 40.

We also determined that randomly assigning peer counselors is not feasible because it violates the essence of “peer counseling.” Because a central tenet of the peer counseling program is to match Loving Support Peer Counselors to WIC participants based on shared language, race, ethnicity, national origin, age, common experiences, and the judgment of the LWA staff, randomly assigning peer counselors would very likely result in an unbalanced treatment and control group of WIC participants. For example, if an agency had one peer counselor who spoke Vietnamese, all of her caseload would either be in the treatment group or the control group. Randomly assigning peer counselors who will subsequently be matched to WIC participants also is highly likely to result in differences in unobservable characteristics between the treatment and control groups. For example, LWA staff could (inadvertently) steer WIC participants whom they judge “least likely to breastfeed” to peer counselors in the treatment group (or vice versa, those “most likely to breastfeed” to the control group), based on unobservable judgments about which peer counselor might best work with which WIC participants. The risk of this type of violation of the conditions necessary for the validity of the study is unacceptably high.    

For these reasons, we rejected the first two approaches to random assignment and adopted the third approach, namely, random assignment of individual WIC participants. Despite otherwise warranted concern about the risks of control group contamination, the particular features of the proposed intervention and the planned study procedures minimize this risk.

The intervention to be tested consists of a change in the mode and timing of peer counseling service delivery. Specifically, the intervention adds two enhancements to existing Loving Support Peer Counseling Programs that currently rely primarily on telephone contact and do not already include in-person early post-partum meetings with new mothers. This intervention does not change the content of breastfeeding support or assistance that peer counselors would offer in its absence. Instead, it requires contact with the new mother at two distinct times – once immediately after delivery and a second time within the first 10 days after delivery – and it requires that the second contact be conducted in-person. LWAs selected to participate in the study will not be offering these two components already. As a result, peer counselors will have to make additional effort to deliver the intervention; there is little risk that peer counselors will inadvertently provide the control group with the enhancements to the Loving Support Peer Counseling services.

Ensuring that the Treatment Condition is Distinct from the Control Condition

In participating LWAs, WIC Participants in the study who are assigned to the Control Group will receive the agency’s existing Loving Support Peer Counseling program. That is, depending on the components of their agency’s pre-existing Loving Support Peer Counseling program (i.e., prior to implementation of the intervention for WIC Participants in the Treatment Group) peer counselors may offer WIC participants assigned to the control condition one or more of the following services :

• One or more telephone contacts with her peer counselor after she has left the hospital post-delivery;

• Mailings from her peer counselor providing educational information about the benefits of breastfeeding and resources available for new mothers to receive support for breastfeeding;

• In-person meeting with her peer counselor at any time after the first 10 days post-partum;

• Referral to a lactation consultant, if requested;

• Information about support group meetings offered by her WIC clinic for new mothers who are breastfeeding

LWAs with existing programs that already require peer counselors to deliver the two enhancements that are part of the intervention are ineligible for the study. Eligible LWAs that participate in the study will be expected to offer these enhancements to the treatment group but not to the control group.

A critical distinction between the control group and the treatment group is whether or not the peer counselor will initiate and offer delivery of the two peer counseling enhancements. For WIC participants in the Treatment Group, the peer counselor will actively offer each enhancement to every member of the Treatment Group; for WIC Participants in the Control Group, the peer counselor will not offer these enhancements – that is, she will not attempt to contact members of the control group when they are in the hospital for delivery, and she will not make any attempt to meet in-person with members of the control group during the first 10-days post-partum.

The evaluation has been designed to ensure that control group members do not receive the intervention, but do receive the types of assistance that participating LWAs would typically offer. Peer counselors will be trained on how to handle requests from members of the control group to receive services that are intended only for treatment group members. In particular, if a WIC Participant in the control condition calls her peer counselor from the hospital, the peer counselor will accept the incoming call if she is available at that time (or if not available at the time of the call, the peer counselor will call the WIC Participant back) and use the call to schedule a later appointment to conduct telephone-based peer counseling once the WIC Participant has been discharged from the hospital (if deemed necessary, the peer counselor may refer the WIC Participant to a lactation consultant or other appropriate health care professional at that time).¹³ Similarly, if a WIC Participant in the control condition requests an in-person meeting with her peer counselor during her first 10-days post-partum, the peer counselor will refer the WIC Participant to a lactation consultant and schedule a meeting with the WIC Participant on or after her 11^th day post-partum.

Maintaining Fidelity

The following study procedures will enable the study to maintain fidelity of random assignment:

1. Eligibility for selection into the study is limited to those LWAs in which the enhancements will be distinct from the “business as usual” Loving Support Peer Counseling Program. Any LWA in which the peer counseling program already incorporates contact WIC Participants when they are in the hospital for delivery or in-person meetings with WIC Participants within the first 10-days post-partum is ineligible for the study;

2. Participating LWAs selected for the study will be required to sign a Memorandum of Understanding in which they agree not to offer the two enhancements to WIC Participants randomly assigned to the Control Condition or to WIC Participants who decline to enroll in the study and to regularly monitor peer counselors to ensure compliance with random assignment;

3. Participating LWAs will be required to assign a peer counselor to a WIC participant enrolled in the study before each WIC participant’s random assignment status is determined.

4. Prior to random assignment, LWA staff will also assign to each WIC participant in the study an alternate peer counselor who will be able cover for the primary peer counselor should the primary be unavailable later on. For example, if a WIC participant is randomly assigned to the treatment group and her primary peer counselor is unable to complete an in-person contact during her first post-partum week, the alternate peer counselor – designated prior to random assignment—will deliver this component of the enhanced Loving Support Peer Counseling services. Assigning an alternate prior to random assignment ensures that LWAs will not be able to select a particular peer counselor to work with women in the treatment group. Designating a backup peer counselor prior to random assignment ensures, for example, that sites will not select a “more experienced” alternate peer counselor to deliver enhanced Loving Support Peer Counseling services to a WIC participant in the treatment group.

5. LWA staff will regularly review peer counselors’ contact logs and the evaluation contractor will regularly review the corresponding Contact Log Abstraction Form (Appendix D) to ensure that WIC participants in the control condition are not receiving the enhancements. Should an instances of non-compliance with random assignment (inadvertent or not) occur, they will be discussed with the LWA as soon as it occurs in order to prevent future violations of random assignment. In addition, during the Follow-up Survey, all WIC participants are asked about the peer counseling services they received, including at delivery (i.e., while in the hospital) and during the first 10 days post-partum. These data will allow the research team an additional, independent means to detect any contamination.

B1.5 SAMPLE SIZES

We propose to randomly assign 900 WIC-BPC participants to each of two groups, treatment and control, yielding a total initial sample of 1,800. Assuming an 85 percent response rate to the participant surveys, this initial sample will yields 765 first-time expectant mothers in each group for a total analytic sample of 1,530. This sample size will yield an MDD of 6 percentage points, assuming the following:

1. a balanced design (n_C=n_T) in which half of the sample is assigned to each group;

2. desired statistical power is 80 percent;

3. a significance criterion () of .05 will be used to test one-tailed hypotheses;

4. expected breastfeeding exclusivity and breastfeeding intensity rates in the control group sample of 50 percent;

5. the amount of variation in the outcomes explained by covariates will be 10 percent (R² = .10).

This expected response rate assumes attrition of 10 percent between consent and the WIC Participant Baseline Survey (a response rate of 90 percent) and additional attrition of 10 percent between the WIC Participant Baseline Survey and Follow-up Survey (a response rate of 90 percent), providing an overall response rate of 81 percent (.90 x .90). However, because the study design calls for conducting a Followup Survey with WIC Participants who did not complete a Baseline Survey we anticipated recovering 4 to 5 percent of the sample that had not participated at Baseline, leading to a higher anticipated response rate (the purpose of the Baseline Survey is to improve precision of the impact estimate, but missing data on the Baseline Survey would not interfere with impact estimate because this is a random assignment study).

Although peer counselors maintain contact with women during the interval between the Baseline and Follow-up survey (women in both the Treatment and Control conditions receive regular contact from their peer counselor), upon further consideration of the two to three-month lag between birth of the infant and the administration of the Follow-Up Survey, there is some risk that mothers may lose interest in the study, in peer counseling, or could move out of the service delivery area of their local WIC agency. If the effective response rate is as low as 70 percent, the initial sample of 1,800 would yield an analytic sample of 1,260 (630 in each group) for an MDD of 6.6 percentage points (rounded to 7 percentage points) under the same assumptions (i)-(v) above.

Below we justify these assumptions, show the various sample sizes considered (Exhibits B3 and B4) based on power analysis and our rationale for selecting the final sample size.

Balanced Design

Half of the WIC participants enrolled will be assigned to the control condition and half to the treatment condition, creating a balanced design that is balanced.

Minimum Detectable Difference

For this study, the desired minimum detectable difference in the percentage of women breastfeeding exclusively or breastfeeding intensively (i.e., breastfed only or mostly in the last 24 hours) is between 6 and 7 percentage points. This level was selected based on both previous research on the effects of peer counseling on breastfeeding outcomes and the need to balance resources for implementing the study with the need to detect policy-relevant differences.

Previous research on the effects of peer counseling on breastfeeding outcomes have found differences just over 6 percent for rates of breastfeeding initiation (Gross, et al., 2009) and for rates of (any) breastfeeding at 3- and 6-months (Olsen et al., 2010). One study reported a 19 percent difference in rates of exclusive breastfeeding at 3-months post-partum (Anderson, et al., 2005), but another study with a similar population found no difference in exclusive breastfeeding rates at 3-months (Chapman et al., 2004).

Other things being equal, the smaller the MDD, the larger the sample size that is required. As a practical matter, the MDD should be set at a level that is meaningful to policymakers. Setting an unnecessarily small MDD substantially raises the cost of the evaluation without providing important information for making policy decisions. However, setting the MDD too high can lead to the unsatisfying outcome that impacts of policy importance are not likely to be detected. Exhibit B3 shows the MDDs associated with various sample sizes, assuming that none of the variation in the outcome measures are accounted for by covariates included in the model.

Exhibit B3: Sample Size Requirements for a Balanced Design for Selected Minimum Detectable Effect Sizes for One- and Two-Tailed Significance Criteria Equal to .05 (Power=80%) and No Regression Covariates

MDD	Sample Size in Each Group (nC=nT)
	One-tailed (1=.05)	Two-tailed (2=.05)
0.05	1,249	1,573
0.06	871	1,092
0.065	732	929
0.07	567	720
0.075	550	698

Significance Criterion

This study will use a one-tailed significance criterion of ₁ =.05. There are two issues to consider regarding the choice of a significance criterion. In the present context, the significance criterion () is the likelihood of finding a significant difference in outcomes between the treatment and control groups that is due to random chance. A significance criterion of  = .05 is typically used in studies of this kind. The second issue is whether to use a one-tailed or two-tailed test. Since there is no plausible reason to expect that in-person peer counseling will reduce the rates of breastfeeding exclusivity or intensity, the use of a one-tailed test (which tests the hypothesis that the intervention will only increase breastfeeding exclusivity or intensity) will be used in this evaluation. Our sample size calculations will use ₁ =.05. While a two-tailed test is more conservative than a one-tailed test, it requires a substantially larger sample size than a one-tailed test (Exhibit B3).

Statistical Power

Statistical power is the probability of finding a statistically significant difference between the treatment and control groups in the study sample if, in fact, such a difference exists in the population. The standard for research of this type is statistical power of 80 percent, which we use in determining the sample size requirements of this evaluation.

Expected Outcomes in the Control Group

Research suggests that the breastfeeding exclusivity and intensity among WIC participants in the control group could range from approximately 20 to 50 percent. Anderson et al. (2005) reported that 27 percent of infants born to women in a predominantly Latina, low-income community who were randomly assigned to peer counseling were exclusively breastfeed at 3-months post-partum. Preliminary results of FNS’s recent Evaluation of the Birth Month Breastfeeding Changes to the WIC Food Packages (OMB control no. 0584-0551, expiration date: 2/29/2012) reported that 18 to 20 percent of mothers with infants who were, on average, 4 to 5 weeks old, had fed breastmilk only during the previous 24 hours. Kistin, Abramson, & Dublin (2004) reported that 40 percent of infants born to low-income, urban women in a control group that did not receive peer counseling were breastfed exclusively.

Note that health issues affecting ability to breastfeed will be independent of WIC participants’ assignment to treatment or control. Many infant or maternal health issues affecting breastfeeding among WIC participants likely derive from WIC participants’ (women and infants) low-income status and not necessarily their selection into WIC participation. Further, research suggests that prenatal participation in WIC improves birth outcomes such as low birth weight, a risk factor for not breastfeeding (see Oliveira and Frazao, 2009).¹⁴ And in fact, some of the most common reasons why low-income women do not breastfeed include lack of knowledge about the health benefits of breastfeeding. Peer counseling that provides education and support by knowledgeable peers can overcome these barriers.

The power analyses conducted here show the expected outcomes in the control group of WIC participants (who are subject to the same health issues as the treatment group) using estimates of 20 and 50 percent rate of exclusive breastfeeding in the control group. We chose to base sample size calculations on a conservative estimate of 50 percent, in order to ensure that the sample was of sufficient size to provide meaningful minimium detectable differences. The power analyses and desired sample sizes should not be affected should there be a higher prevalence of health issues among the WIC population that affect breastfeeding outcomes.

Proportion of the Variance in the Outcome Measure that is Explained by Covariates

This evaluation will use logistic regression models to estimate the impact of the intervention on breastfeeding exclusivity and intensity (see Estimation Procedures, in B.2, below). The covariates used in these models might account for some of the differences that exist between the treatment and control groups in factors that might influence breastfeeding exclusivity and intensity. Other things being equal, the higher the proportion of the variation in our outcome measures that is accounted for by the co-variables, the smaller the sample size that is required. Exhibit B3 shows that a sample of 732 would be needed in each group if none of the variation in the dependent measures is accounted for by co-variates included in the model to achieve an MDD of 6.5 percentage points. Exhibit B4 shows the sample sizes needed for MDDs between 6 and 7 percentage points if covariates account for ten percent of the variation in the outcomes.

Exhibit B4 shows the sample sizes required with a balanced design (n_C=n_T) for breastfeeding exclusivity or intensity rates in the control group sample of 20 percent or 50 percent and for various proportions of the variance in the outcome measure that are explained by the co-variables.

While prior research suggests that about 20 percent of the control group sample may be expected to breastfeed exclusively,¹⁵ the more conservative assumption is that this rate will be 50 percent. To be conservative, we assume that the covariates will account for only 10 percent of the variance.¹⁶ As shown below, to achieve an MDD between 6.and 7 percentage points, the analytic sample size required ranges between 567 and 765 per group (1,202 and 1,530 total).

As discussed above, assuming a response rate of 85 percent and an initial sample size of 1,800 WIC participants (900 per group); the resulting analytic sample of 1,530 (765 first time expectant mothers in each group) would yield an MDD of 6 percentage points. Alternatively, if the response rate is 70 percent, this initial sample of 1,800 WIC participants will yield an analytic sample of 1,260 (630 per group) and an MDD of 6.6 percentage points.

Exhibit B4: Sample Size Requirements to achieve MDDs between 6 and 7 percentage points with a Balanced Design

Regression R2	Sample Size in Each Group (nC=nT) needed for MDD = 6 percentage points
	PC=.20 Rate of Exclusive Breastfeeding	PC=.50 Rate of Exclusive Breastfeeding
0.00	554	871
0.05	525	820
0.10	505	765
0.15	470	738
0.20	423	693
Regression R2	Sample Size in Each Group (nC=nT) needed for MDD = 6.5 percentage points
	PC=.20 Rate of Exclusive Breastfeeding	PC=.50 Rate of Exclusive Breastfeeding
0.00	469	732
0.05	446	695
0.10	421	658
0.15	399	622
0.20	376	585
Regression R2	Sample Size in Each Group (nC=nT) needed for MDD = 6.8 percentage points
	PC=.20 Rate of Exclusive Breastfeeding	PC=.50 Rate of Exclusive Breastfeeding
0.00	428	669
0.05	407	635
0.10	385	601
0.15	365	569
0.20	343	535
Regression R2	Sample Size in Each Group (nC=nT) needed for MDD = 7.0 percentage points
	PC=.20 Rate of Exclusive Breastfeeding	PC=.50 Rate of Exclusive Breastfeeding
0.00	404	631
0.05	384	599
0.10	363	567
0.15	344	537
0.20	324	505

B1.5 RESPONSE RATES

Our expectation, based on similar studies, is that we will achieve a 100 percent response rate for the staff at LWAs once the agency has prepared a successful response to the FOA. One of the requirements of study participation (which will be explained in the FOA) is that the LWA staff agree to participate in all evaluation activities, including the sharing of data (e.g., such as that collected using study enrollment forms, monthly progress reporting forms) with the evaluation contractor and participating in the interviews conducted as part of the process study.

For WIC-BPC participants, we anticipate that 85 percent of the initial sample will complete both the baseline and follow-up surveys.¹⁷ This expectation is based on the fact that study participants will provide up-to-date contact information when they enroll in the study and will be in regular contact with the LWA.

The Evaluation Contractor (Abt Associates and their subsidiary AbtSRBI) has achieved high response rates with low-income populations similar to the WIC Participant population to be surveyed in this study, and in studies with a design similar to this one (i.e., a randomized control trial or quasi-experimental study where baseline data and follow-up data were necessary to compare two groups). Examples include:

• Assessing Quality of Life Issues in FEMA’s Alternative Housing Pilot Program (AHPP) completed in 2007 for the US Department of Housing and Urban Development. A Household Outcomes Survey was conducted with individuals receiving an AHPP housing unit in four Gulf states with large numbers of residents diplaced by Hurricane Katrina (study design differed by state and was either a randomized control trial or a pre-post design). Among individual leaseholders in AHPP housing, the response rate was 83 percent.

• Impact Evaluation of Upward Bound's Increased Emphasis on Higher-Risk Students (2006) for the US Department of Education. In this randomized control trial, at-risk high school students (living in a low income household was one of several factors defining “at-risk”) were assigned to either a Treatment and Control group and administered a Baseline Survey. Response rate for this survey was 94 percent (the study was subsequently canceled so no follow-up data collection was conducted).

• Impact Evaluation of the U.S. Department of Education's Student Mentoring Program (2005) for the US Department of Education. Students were randomly assigned to a mentoring program or a waiting list. Baseline surveys were collected from 1,300 students across 21 different sites with a response rate of 97 percent; a 93 percent response rate was achieved on the follow-up survey.

• Moving to Opportunity, completed in 2002 for the US Department of Housing and Urban Development. As part of a randomized control trial demonstration project, collected participant data from in 4,248 individual heads of households for a response rate of 89 percent. Participants in the experimental group received vouchers for use only in low-poverty neighborhoods along with agency counseling. Members of a comparison group received Section 8 vouchers for housing in the locations of their choice but did not receive counseling. Members of the control group continued to live in public housing or project-based assisted housing and received no vouchers.

Like individuals who participated in the above studies, participants in Phase 2 of the WIC Peer Counseling study will be low-income individuals who have applied for a benefit or resource available from a social service agency and who have volunteered to participate in a random assignment study. Moreover, women who agree to participate in the study will also have volunteered to participate in breastfeeding peer counseling. Thus, this population is likely to be interested in the survey topic, a significant factor in stimulating survey participation.¹⁸ In addition, researchers will have recent contact information for study participants provided by the participant themselves to the LWA staff who recruit them into the study (i.e., the evaluation contractor will receive contact information at most 7 days after a participant enrolls in the study, and attempts to conduct the Baseline Survey would commence within 8 to 10 days of enrollment). Furthermore, study participants, who have actively sought to certify for WIC benefits, must return to the LWA approximately every 90 days to receive additional food vouchers. As a result, LWA staff will be able to update any changes to participants’ telephone numbers or home addresses. In addition, peer counselors’ will indicate on their contact logs if a WIC-BPC participant’s contact information has changed and this updated information will be sent to the evaluation contractor before attempts to complete the follow-up survey with that participant begin.

B.2 Procedures for the Collection of Information

Procedures for the collection of information addressed below include:

1. Statistical methodology for stratification and sample selection;

2. Estimation procedure;

3. Degree of accuracy needed for the purpose described in the justification;

4. Unusual problems requiring specialized sampling procedures; and

5. Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

B2.1 STATISTICAL METHODOLOGY FOR STRATIFICATION AND SAMPLE SELECTION

A statistical power analysis to determine the appropriate sample size for this study was described in Section B.1, as were the methods for selecting the Local WIC agencies. The study design will not use stratification or sample selection methods other than what was described above.

B2.2 ESTIMATION PROCEDURE

In this section we describe our approach to estimating the impact of the intervention on two outcomes: breastfeeding exclusivity and intensity. These outcomes were used previously in the OMB-approved data collection for the Evaluation of the Birth Month Breastfeeding Changes to the WIC Food Packages (OMB control no. 0584-0551, expiration date: 2/29/2012 ). The construction of each of these measures is discussed below. First, however, we begin with a discussion of descriptive analyses and a process analysis that will examine the implementation of in-person counseling.

Descriptive Analyses

Prior to estimating multivariate statistical models for the impact of the program, we will present several tables to provide an overview of the distribution of our sample across participating LWAs, and baseline characteristics of the sample characteristics. These descriptive analyses help us to understand the study sample and the local WIC agencies, and they allow us to verify that the randomization was properly implemented.

We will tabulate the distribution of baseline demographic characteristics of the study sample and how they vary across the LWAs, and how these characteristics vary for treatment and control groups within each site. Characteristics of interest include race/ethnicity, educational level, household size and income, marital status, as well as baseline attitudes toward and knowledge of breastfeeding.

In addition, using data from peer counselor contact logs and the WIC-BPC participant follow-up surveys, we will report measures related to participation in peer counseling. For the treatment group we will report measures related to participation in the enhanced Loving Support Peer Counseling Program (e.g., percent of WIC-BPC participants who received a peer counseling contact while in the hospital; percent who received in-person visit during the first 10 days post-partum). For the control group, we will report measures of the frequency and timing of peer counseling contacts, including any contacts that would indicate contamination of the control group (i.e., cross-overs), such as receipt of an in-person peer counseling visit while in the hospital or during the first 10 days post-partum.

In addition, descriptive statistics on the similarity of peer counselors to the WIC-BPC participants they serve will be examined to explore the degree to which peer counselors were “matched” to WIC-BPC participants based on racial, ethnic, and educational background, as well as on languages spoken. It is possible that WIC-BPC participants’ receptivity to breastfeeding counseling will be affected by the degree to which they perceive their peer counselor as similar to themselves. Although the impact estimates will not include any data on the degree of similarity between Peer counselors and WIC participants, these data will provide FNS with useful information about the extent to which peer counselors are “matched” to WIC-BPC participants along these lines. No tests of the impact of the intervention on any subgroups are planned, as the sample sizes are too small to produce reliable estimates. Data from the Peer Counselor Background Questionnaire will also be used to create a covariate measure for inclusion in the impact analyses discussed below.

Implementation Data Analyses

The implementation analysis will describe how staff at each of the local WIC agencies implemented the intervention. Because the number of LWAs is quite limited and not intended to be nationally representative, variation in the characteristics of the population served or control condition (“business as usual”) across the sites can affect the results of the impact analysis. The implementation analysis discussed below will help provide an understanding of any such differences and is essential to interpreting the meaning and implications of site-to-site differences in the impact of in-person peer counseling.

Exhibit A2.1 in Part A shows the key research questions for the process study. Interview guides are designed to elicit information to address these research questions (see Part A, Exhibit A2.2). Following the completion of site visits, each site visitor will prepare a site report for each site. Site visitors and analysis team members will read each others’ reports and convene an analytic focus group to identify and discuss similarities and differences in LWAs’ implementation of the study procedures (e.g., enrollment, random assignment) and delivery of both the “business–as-usual” and enhanced Loving Support peer counseling services. Key areas of discussion will include major factors at each LWA that facilitated or hindered the agency’s efforts to promote breastfeeding and to deliver their standard and enhanced Loving Support Peer Counseling services.

Individual site reports will be integrated with site-level descriptive statistics, and a cross-site analysis capturing key elements of the variation in implementation practices will be prepared. Tables showing cross-site similarities and differences will be prepared. This implementation analysis will help inform interpretation of impact study findings and will provide guidance for any future attempts to implement the intervention.

Estimation of Impacts

Next we describe the construction of outcome measures for breastfeeding exclusivity and intensity and the regression models we will use to estimate the impact of the intervention on these outcomes.

Outcome Measures

To construct the outcome measures we will ask the following question during the follow-up survey conducted eight to 12 weeks after each WIC-BPC participant has given birth.

Which of the following best describes the kind of milk you fed your baby in the last 24 hours?

• Breast milk only

• Mostly breast milk with some formula

• Breast milk and formula about equally

• Mostly formula with some breast milk

• Formula only

To measure breastfeeding exclusivity we will convert their response to a binary (i.e., dichotomous) measure that will equal 1 if they fed breastmilk only in the last 24 hours, and 0 otherwise.

To measure breastfeeding intensity, we will assign an ordinal value of 1 to 5 to each of the above responses, where higher values represent a greater proportion of breastmilk to formula. That is, we will let 1 = formula only (no breastfeeding), 2 = mostly formula with some breastmilk, …, and 5 = breastmilk only.

This measure will be validated with information from other survey items on the average number of times per day that the baby was fed breastmilk over the last week, and the total ounces of formula that the baby drank during the previous day. This information, in conjunction with the baby’s weight, will

be used to corroborate the mothers’ responses to the above direct question describing breastfeeding and formula feeding patterns over the last 24 hours. In the Evaluation of the Birth Month Breastfeeding Changes to the WIC Food Packages this validation procedure showed that breastfeeding intensity was inversely correlated with the total ounces of formula infants consumed the previous day.

Estimation of Impacts

Because the outcome variable is either binary (exclusivity) or ordinal (intensity) we will use logistic regression analysis to estimate the impact of the treatment on exclusivity and intensity. The statistical model used in the analysis is expressed as:

Where

Y_ij is the outcome measure of breastfeeding exclusivity or intensity for the i^th mother in j^th LWA,

T_ij is an indicator variable denoting whether the i^th mother in the j^th LWA is in the treatment group (trt= 1) or in the control group (trt= 0),

X_ij is a covariate measure of the i^th mother in the jth LWA,

I_lwa,j is an indicator variable denoting whether the i^th mother is in j^th LWA (1), 0 otherwise.

ε_ij is a random error term.

Two models will be estimated; a dichotomous logistic regression model for exclusivity and an ordinal logistic regression model for intensity.

Model 1: Y_ij = logit(EXCLUSIVITY_ij), where

EXCLUSIVITY_ij = 1 for i^th mother in the jth LWA if they breastfeed exclusively, 0 otherwise.

Model 2: Y_ij = logit(INTENSITY_ij), where

INTENSITY_ij = breastfeeding intensity for i^th mother in j^th LWA.

The coefficient represents the estimated marginal impact of the treatment on Y_ij. The logit transformation will then be used to convert these coefficients into the change in the probability that a mother will breastfeed exclusively (Model 1) or the change in the intensity of breastfeeding (Model 2).¹⁹. A positive coefficient for the exclusivity outcome tells you that exclusivity is more likely for the treatment group. A positive coefficient for the intensity outcome tells you that being in the treatment group increases the likelihood of being in a higher intensity category.

Covariates constructed from data collected using the Baseline Survey (Appendix A1) will be used to improve the R-squared (which allows for a smaller sample size). In addition, one item from the Follow-up Survey and data from the Peer Counselor Background Questionnaire will be used to construct a covariate for inclusion in impact models . The Peer Counselor Background Questionnaire will provide data on various dimensions used in matching Peer Counselors to WIC Participants, such as:

• Language;

• Race/ethnicity;

• Highest educational level;

• Household income;

• Number of children living in household; etc.

By combining these data with similar data on WIC Participants (collected during the Follow-up Survey), for each Peer Counselor-WIC Participant pair, a congruence score for each demographic characteristics will be calculated (e.g., if peer counselor and WIC participant share a common language then Congruence_language = 1, else =0). Item B7a on the WIC Participant Follow-up Survey (Appendix A2) will be used to construct a composite measure of WIC Participants’ perception of their peer counselor. Item B7a consists of 3 subitems from Turban and Jones’ (1988) scale of perceived similarity²⁰ a scale that has been used to predict the success of mentor/protégé relationships with Cronbach’s alpha coefficent of.85.²¹ For each WIC Participant, a single “Perceived similarity” score will be calculated as the average rating across the three subitems. The congruence factor with the highest correlation to WIC Participants’ perceived similarity of their peer counselor will be included in the impact model.

The results will be presented using regression-adjusted tabulation, because the determination of whether a difference between treatment and comparison group means is significant is more meaningful if we look at the means after adjusting for any differences in the characteristics of the participants that could also affect the variable of interest. For these questions we will thus first run regressions of the variables of interest on mothers’ demographic and socioeconomic characteristics, and then display the adjusted treatment vs. control group comparison in simple tables or charts.²²

Effects of the Treatment on the Treated

The above discussion refers to the impact of being in the treatment group, that is, the impact of being offered the enhanced peer counseling model (i.e., peer counseling while in the hospital (post-delivery) and in-person during the first week post-partum). However, some women in the treatment group will not avail themselves of these offered services (the intervention). Such members of the treatment group in effect do not receive the treatment. In view of this, the analysis will also estimate the impact of the treatment on those WIC participants who actually receive the enhancements to the Loving Support program. That is, in addition to estimating the overall effects of making the intervention available, we will also estimate the impact of the treatment on those women who actually received the intervention—treatment-on-treated.

Methods to Account for Missing Data Due to Attrition

In addition to estimating the effects of the treatment on the treated, the evaluation will also investigate the effects of missing data (e.g., due to study attrition or non-response) and apply procedures to mitigate these effects. Although missing data can reduce statistical power to detect differences between the treatment and control groups, the more serious concern in randomized control trials is the potential for bias of the impact estimate. If the causes of missing data differ for the treatment and control groups, for example, or if the data are more likely to be missing for some subgroups than for others, then impact estimates can be biased. Procedures for dealing with missing data will vary depending on:

• the overall proportion of missing data;

• the differential proportion of missing data between the treatment and control groups; and

• whether the missing data occur for covariate (Baseline Survey) or outcome measures (Follow-up Survey)

Generally, the risk of moderately high levels of overall attrition can be offset by relatively low differential attrition and vice versa. Thus, we will examine both overall attrition and the difference in attrition between the treatment and control conditions to determine whether or not steps are needed to address possible bias in the impact estimates. If the pattern of attrition suggests possible bias in the impact estimate’s magnitude and/or standard error, we will apply methods advocated by Puma et al., 2009. For missing covariate data we will apply either a dummy variable adjustment or a multiple imputation approach (Puma et al., 2009). ²³ For missing outcome data, we will use one of two methods: we will (1) apply a weighting approach (described below Section B.3) or (2) specify an impact model with terms that interact each of the covariates with the treatment indicator. In general, we will apply methods for dealing with missing data that seek to minimize possible bias on the magnitude of the impact estimate or its standard error (Puma et al., 2009).

B2.3 DEGREE OF ACCURACY NEEDED FOR THE PURPOSE DESCRIBED IN THE JUSTIFICATION

A statistical power analysis was conducted to determine the appropriate sample size for this study and was described above in our response to Section B.1.

B2.4 UNUSUAL PROBLEMS REQUIRING SPECIALIZED SAMPLING PROCEDURES

No specialized sampling procedures are involved.

B2.5 USE OF PERIODIC DATA COLLECTION CYCLES TO REDUCE BURDEN

This is a one-time study.

B.3 Methods to Maximize Response Rates and Deal with Non-Response

B3.1 MAXIMIZING RESPONSE RATES AMONG LWA STAFF

The procedures to be used to ensure a high rate of response for the staff from State and local WIC agencies in the study are largely non-statistical in nature and focus on methods to ensure the cooperation of WIC staff at the State and local levels. Our expectation, based on similar studies, is that we will achieve a 100 percent response rate for the staff at LWAs once the agency has prepared a successful response to the FOA. One of the requirements of study participation (which will be explained in the FOA) is that the LWA staff agree to participate in all evaluation activities, including the sharing of data (e.g., such as that collected using study enrollment forms, monthly progress reporting forms) with the evaluation contractor and participating in the interviews conducted as part of the process study.

Because the agencies will have agreed to participate in the study, will have received resources and will have signed a memorandum of understanding outlining roles and responsibilities, to include participation in interviews, declining to participate would be greatly unexpected. The evaluation contractor has conducted similar interviews with very high response rates among LWAs that were recruited to participate in Phase 1 interviews. In contrast, agencies that will participate in Phase 2 will have volunteered. LWA staff interview protocols proposed for Phase 2 were adapted from similar protocols used in Phase 1 in 40 local WIC agencies and with 16 LWAs in the Evaluation of the Birth Month Breastfeeding Changes to the WIC Food Packages (OMB control no. 0584-0551). In both studies, the response rates were 100%. None of the staff members interviewed at any of these 16 LWAs declined to participate, withdrew from the interview or declined to answer any item.

B3.2 MAXIMIZING RESPONSE RATES OF WIC PEER COUNSELING PARTICIPANTS

Methods will be used to ensure high response rates by WIC-BPC participants to the baseline and follow-up surveys. We anticipate achieving at least an 85 percent response rate by WIC-BPC participants. The study will take advantage of the fact that study participants will provide up-to-date contact information when they enroll in the study and the fact that study participants will be in regular contact with the LWA. Furthermore, study participants, who have actively sought to certify for WIC benefits, must return to the LWA approximately every 90 days to receive additional food vouchers. As a result, LWA staff will be able to update any changes to participants’ telephone numbers or home addresses. In addition, peer counselors’ will indicate on their contact logs if a WIC-BPC participant’s contact information has changed and this updated information will be sent to the evaluation contractor before attempts to complete the follow-up survey with that participant begin.

The use of CATI technology to conduct the telephone interviews maximizes response rates via by taking advantage of efficient and automated sample tracking. The evaluation contractor maintains a staff of seasoned telephone center supervisors and interviewers skilled in conducting surveys and trained in procedures to reduce non-response. All interviewers complete an interviewer Basic Training module prior to study-specific training modules. This Basic Training focuses on interviewing skills, confidentiality, and use of the CATI system. Special training programs are conducted for Spanish language interviewers and for refusal converters.

Project training materials for WIC participant surveys will include: an interviewer manual that includes an overview of the WIC program, of the study and its methodology, the anticipated demographic characteristics of study participants. Interviewer manuals will describe all survey procedures and will include question-by-question specifications (Q-by-Qs) for each survey. These will contain explanations, definitions of terms, and instructions for administering appropriate question-specific probes. These Q-by-Qs will be incorporated into the training. During the study, the Q-by-Qs are incorporated directly into the CATI system and available to interviewers on demand. Finally, the training covers study-specific vocabulary needed to administer the survey, and strategies for achieving a high response rate with the survey sample (i.e., first-time mothers or expectant mothers).

Monetary incentives in the form of $20 incentives for each of the surveys will also help maximize response rates. The justification for these incentives is included in Part A.

B3.3 METHODS TO ACCOUNT FOR NON-RESPONSE

Overall Non-Response

To address non-response that might otherwise bias the study’s estimates, the study plans to use standard weighting adjustments. Despite the study’s plan for ensuring high response rates, study response rates will not be 100 percent, and the study will need to take appropriate steps for addressing non-response bias due to missing data. For example, because the Baseline Survey must be completed within four weeks after the date of enrollment (so that random assignment can take place in a timely fashion). The Follow-up Survey targets first-time mothers with two- to three-month infants, who are adjusting to a new schedule and new responsibilities. Although mothers with newborn infants may be easier to reach by telephone than other populations, as they are more likely to be home, they may be more difficult to keep on the telephone for the duration of the survey. We have made the follow-up survey as short as possible, but some non-response may occur.

Therefore, the study will weight the data to account for differential non-response across the treatment and control groups of WIC-BPC participants. In particular, the study will use the characteristics of the WIC-BPC participants in the sample, such as age, place of birth, race/ethnicity, and pre-test scores, to estimate a response propensity for each sample member. Study participants will be stratified into a small number of groups based on their response propensities. Within each stratum, the study will compute a response rate, compute a weighting adjustment factor that equals the inverse of the response rate, and reweight the data by multiplying the initial weight by the adjustment factor. These weights will be used in the estimation described in our response to Question B.2.

The collection of data from WIC participants who decline to enroll in the study will also be attempted using a brief Decline Form. The LWA staff who invites a WIC participant to enroll in the study will ask each woman who declines to complete this very short form. The instrument collects age, place of birth, languages spoken, highest educational level, income, race, ethnicity, and prior receipt of TANF, Medicaid, SNAP, etc. Characteristics of the non-participants will be compared to those of study participants; t-tests and chi-square statistics will be used to test the null hypothesis that participants and non-participants do not differ on these characteristics.

Differential Non-Response Between the Treatment and Control Groups

There is no reason to expect differential response rates between the treatment and control groups in this study. The baseline survey takes place prior to random assignment, so it is equally likely that a non-responder would be assigned to the treatment or control conditions. For the follow-up survey, potential respondents will have been randomly assigned to a treatment or control condition; members of both groups will have received breastfeeding peer counseling and will have an infant approximately 8- to 12-weeks old at the time the survey is attempted.

B.4 Tests of Procedures or Methods to Be Undertaken

B4.1 SURVEYS OF WIC PARTICIPANTS

These data collection instruments are largely based on similar survey instruments used successfully in FNS’s Evaluation of the Birth Month Breastfeeding Changes to the WIC Food Packages (OMB control no. 0584-0551, expiration date: 2/29/2012). All items used to construct outcome measures, for example, are identical to items from WIC participant surveys in the Evaluation of the Birth Month Breastfeeding Changes to the WIC Food Packages. As a result, we will not conduct additional pilot testing of the WIC Participant Baseline or Follow-up Surveys (during the Demonstration Period, no data collection from WIC participants will take place).

B4.2 INTERVIEWS WITH LWA STAFF

These data collection instruments are adapted from similar interview protocols used in Phase 1 of the WIC Breastfeeding Peer Counseling Study with staff from 40 LWAs in a nationally representative sample.

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Abt Associates is responsible for all data collection and analysis for this study. Individuals consulted for statistical aspects of the study include the evaluation contractor, Abt Associates, and Abt’s subcontracted consultant, Fred Glantz from Kokopelli Associates. He can be reached at (505) 983-0785 or [email protected]. In addition, staff from FNS’ Office of Research, Nutrition and Analysis have reviewed the study design and instruments. Joe Robare is the FNS contact for this effort. He can be reached at joseph.robare@fns.usda.gov.

References

Anderson, A.K., Damio, G., Young, S., Chapman, D.J., and Perez-Escamilla, R. (2005). A randomized trial assessing the efficacy of peer counseling on exclusive breastfeeding in a predominantly Latina low-income community. Archives of Pediatrics and Adolescent Medicine, 159, 836-841.

Bentley, M.E., Caulfield, L.E., Gross, S.M., Bronner, Y., Jensen, J., Kessler, L.A., and Paige, D.M. (1999). Sources of Influence on Intention to Breastfeed Among African-American Women at Entry to WIC. Journal of Human Lactation, 15, 27-34.

Britton, C., McCormick, F.M., Renfrew, M.J., Wade, A., & King, S.E. (2007). Support for breastfeeding mothers. Cochrane Database of Systematic Reviews 2007, Issue 1. Art. No.: CD001141. DOI: 10.1002/14651858.CD001141.pub3.

Chapman DJ, Damio G, Young S, Perez-Escamilla R. (2004). Effectiveness of breastfeeding

peer counseling in a low-income, predominantly Latina population: a randomized

trial. Archives of Pediatrics and Adolescent Medicine, 158, 897-902.

Gross, S.M., Resnick, A.K., Cross-Barnet, C., Nanda, J.P., Augustyn, M., and Paige, D.M. (2009). The differential impact of WIC perr counseling programs on breastfeeding initiation across the state of Maryland. Journal of Human Lactation, 25, 435-445.

Kistin, N., Abramson, R., & Dublin, P. (2004). Effects of peer counselors on breastfeeding initiation, exclusivity, and duration among low-income urban women. Journal of Human Lactation, 10, 11-15.

Li, R. Darling, N., Maurice, E., Barker, L., & Grummer-Strawn, L. (2005). Breastfeeding rates in the United States by charactertistics of the child, mother or family: The 2002 National Immunization Survey. Pediatrics, 115(1), e31-e37.

Olsen, B.H., Haider, S.J., Vangjel, L., Bolton, T.A. and Gold, J.G. (2010). A quasi-experimental evaluation of a breastfeeding support program for low income women in Michigan. Maternal and Child Health, 14, 86-93.

Puma, M.J., Olsen, R.B., Bell, S.H., and Price, C. (2009). What to do when data are missing in

group randomized controlled trials (NCEE 2009-0049). Washington, DC: National

Center for Education Evaluation and Regional Assistance, Institute of Education Sciences,

U.S. Department of Education.

U.S. Department of Agriculture, Food and Nutrition Service, Office of Research and Analysis (2010). WIC Participant and Program Characteristics 2008, WIC-08-PC, by Patty Connor, Susan Bartlett, Michele Mendelson, Katherine Condon, James Sutcliffe, et al. Project Officer, Fred Lesnett. Alexandria, VA. http://www.fns.usda.gov/ora/menu/Published/WIC/FILES/pc2008.pdf, retrieved March 8, 2011.

1 These 675 LWAs with Loving Support Peer Counseling programs serve approximately 56 percent of all pregnant WIC participants (n ≈ 915,000) per month. Number of LWAs: See U.S. Department of Agriculture, Food and Nutrition Service, Office of Research and Analysis, WIC Breastfeeding Peer Counseling Study, Final Implementation Report, WIC-10-BPC, by Ann Collins, Catherine Dun Rappaport, and Nancy Burstein. Project Officer: Tracy K. Palmer, MPH. Alexandria, VA: June 2010. Number of pregnant WIC participants per month in the 50 U.S. States and District of Columbia: Special tabulation from WICagencies2008ytd.xls (http://www.fns.usda.gov/pd/wicmain.htm).

2 No data collection from WIC participants will occur during this two-month Demonstration Period (i.e., the Baseline and Follow-up Surveys will not be conducted, nor will WIC participants receiving peer counseling be enrolled into the study, or subjected to random assignment). The sole purpose of the Demonstration Period is for each LWA to demonstrate feasibility of implementing the intervention – the impact of which will be tested during the subsequent Study Period in those LWAs that successfully implement it.

3 Justification for this sample size is provided in B1.5, below.

4 We anticipate that the maximum number of LWAs will be eight but the study could be undertaken by a smaller number. The final number will depend on the number and size of the applicant LWAs that meet the study’s requirements. It would be possible, in theory, to include a mix of large and small LWAs so long as the average number of women enrolled across the participating LWAs was 225.

5 WIC Participant and Program Characteristics are collected via a reporting system developed by FNS that routinizes compilation of participant information from State WIC agencies. The system is an automated transfer of an agreed-upon set of data elements that are routinely downloaded by State WIC agencies. These data are downloaded from State WIC agencies’ existing automated client and management information systems used to certify applicant eligibility for WIC benefits and to issue food vouchers and checks. The evaluation contractor (Abt Associates) has access to these data already.

6 The per-participant grant amount is approximately $88 per participant enrolled in the study (half assigned to treatment, half to control), but will be disbursed based on the number of women assigned to the treatment group to whom peer counselors successfully deliver the enhanced peer counseling services—that is, approximately $175 per successful delivery of the enhancements. .

7 The target number for each LWA will be based on the agency size (in terms of the average number of pregnant WIC participants served per month) in cooperation with the research team. For example, if an agency asserts that it can enroll 225 women into the efficacy trial (or 38 per month); the LWA’s target number of WIC participants to receive the enhanced peer counseling services would be 17 per month. The LWA would need to deliver the enhanced peer counseling services to at least 85 percent of those women who agreed to an in-person meeting.

8 For example, some research suggests that pregnant women who formula fed their other children are much less likely to breastfeed subsequent children (e.g., Bentley et al., 1999).

9 If the population of all age-eligible first-time expectant mothers exceeds the LWA’s capacity to deliver the intervention to half the number enrolled in the study, the evaluation contractor will work with the LWA to implement a recruitment algorithm to ensure that only a subset of eligible WIC participants are recruited and that they are recruited in a systematic way

10 Women who first certify for WIC benefits in their 9^th month of pregnancy are ineligible for the study because there is insufficient time to collect baseline data and complete random assignment before they give birth. For the same reason, it is unlikely that the study can enroll women who first certify for WIC benefits in their 8^th month of pregnancy but interviews with LWA staff during the Demonstration Period will provide more information on the feasibility of including this latter group in the study.

11 For example, women who first certify for WIC in their 4^th month of pregnancy can be enrolled in the study in their 7^th month of pregnancy, when they return to their LWA 90-days after certification to obtain additional vouchers or meet with a nutritionist, etc.

12 It is necessary to complete random assignment quickly enough to allow LWAs to deliver peer counseling services based on assignment status and this requires limits on the length of time allotted to collect baseline survey data. Methods to ensure a high response rate during this 30-day period are described under Part B.

13 Under circumstances where any WIC Participant—regardless of assignment to Treatment or Control Group—is in imminent danger, has a medical or psychiatric emergency, or reports domestic abuse or child abuse or neglect, peer counselors will be required to follow their LWA’s policy for referral and reporting to the appropriate personnel.

14 Oliveira and Frazão (2009). The WIC Program: Background, Trends, and Economic Issues, 2009 Edition, Economic Research Report No. 73, U.S. Department of Agriculture, Economic Research Service, April 2009.

15 Britton, McCormick, Renfrew, Wade & King (2007).

16 Recent (unpublished) results from FNS’s Evaluation of the Breastfeeding Changes to the WIC Food Packages suggests that covariates will account for 9 to 11 percent of the variation in breastfeeding outcomes.

17 Moreover, we would still include in impact analyses the small proportion of WIC-BPC participants who lacked a completed baseline survey, used to collect data for covariates, but had a complete follow-up survey, used to collect data for the outcome measures. The follow-up survey includes items to collect demographic characteristics should the respondent not have a completed baseline survey.

18 Groves RM, Presser S, Dipko S. The role of topic interest in survey participation decisions. Public Opinion Quarterly. 2004;68(1):2–31

19 The logit equals L_ij= Ln[P_ij/(1-P_ij)]. Thus is equal to the change in the log of the odds of breastfeeding exclusively that result from receiving in-person peer counseling which gives us

∆[P_ij /(1-P_ij)]= e^Xij where X_ij=β₀ + β₁ + β₂(meanX_1j). Solving for ∆P_ij , we have

∆P_ij = e^X/(1+e ^X).

20 Turban, D. B., & Jones, A. P. (1988). Supervisor-subordinate similarity: types, effects and mechanisms. Journal of Applied Psychology, 73, 228−234.

21 See Ensher, EA & Murphy, SE (1997). Effects of race, gender, perceived similarity, and contact on mentor relationships. Journal of Vocational Behavior, 50, 460-481; and Polander, E & Schneider, T (August, 2010). The importance of perceived similarity within faculty-faculty mentoring dyads. Paper presented at the 118th Annual Convention of the American Psychological Association, San Diego, CA.

22 Note that because of the low sample size we will not be conducting any sub-group analyses, since the MDDs would be relatively large. For example, for subgroups that comprise half of the study sample the MDDs would be nearly 9 percentage points.

23 Because we will not be examining the relationship between covariates and the outcomes, dummy variable adjustment does not result in any bias on the impact estimate; furthermore, randomization should result in covariates being independent of the treatment indicator, an assumption we will test by comparing the baseline characteristics of the treatment and control groups.