Appendix K: NASS Comments

Appendix K

NASS Comments

This page has been left blank for double-sided copying.

NASS Review of OMB xxxx-xxxx WIC,

WIC Local Agency Breastfeeding Policy and Practices Inventory

General Comments:

This docket is another well written research agenda from Abt Associates. If executed as advertised, the results will meet the stated objectives, particularly a census of policies and practices employed by state and local agencies to encourage breastfeeding, a census of means and methods to assess breastfeeding outcomes, and some preliminary estimates of population parameters across a variety of domains. It should also be influential in the construction of a future tracking and monitoring system for breastfeeding policies and practices and this monitoring system will standardize the measured outcomes across agencies. The eventual standardized measures will allow for more deliberate policy and practices evaluation.

Specific comments are given below:

A.9. Decision to provide payment or gift…..

NASS tried this “summary” report incentive with its respondents who were reporting financial data. It became contentious because of comparisons across operations and was eventually discontinued. This may not be a problem for the type of survey reviewed for this docket, but comparing statistics across respondents does have the potential for agitation. Agitation is also possible, but to a lesser extent I think, from comparing policies and practices across agencies.

[Mathematica:] We identified and addressed three potential issues with providing agency summary reports (ASRs) as an incentive. First, agencies may be concerned that they could be identifiable. We will aggregate agency responses within the State or FNS region in which they are located. Second, staff at agencies with poorer breastfeeding outcomes or fewer policies or practices may not enjoy learning that other agencies in their area are faring better. However, they might improve their participants’ breastfeeding outcomes by using the information in the ASR to identify promising policies and practices to implement. We might also mitigate the “shock factor” of an agency learning that it falls in the bottom of the distribution by presenting more aggregate-level information, such as ranges, rather than means or percentages. Third, some agencies may question the reports if they believe their relative performance to be better than our results show. To address this, we will consider offering an optional printout of agency responses at the end of the survey to serve as documentation to compare against the ASR. We will be mindful of the potential that ASRs may be contentious.

A.14. Annualized cost to Federal Government

$748,239 to survey and tabulate reports for 2,090 known respondents is expensive by NASS standards.

[Mathematica:] This is the original contract value agreed upon by FNS and Mathematica.

A.16. Outlines for Tabs and Pubs

Table A.16.1 is a natural, descriptive starting point for this data. The user of these results is cautioned that statistics summarized in Table A.16.1 require careful consideration of the fact that outcomes are potentially measured by each agency in a non-standardized fashion. Some agencies, it appears, will report measures originating outside the reporting agency itself. Abt and Associates have included several instrument questions which will provide for some standardization, nonetheless, FNS should note that Table A.16.1 is accurately described by Abt as a summarization of the variation in outcomes across agencies. I would add “that use non-standard practices and procedures to assess the outcomes”.

[Mathematica:] This is an important point and we understand the concern. Using the breastfeeding initiation and duration estimates from the WIC PC data will somewhat standardize the data for those two outcomes, to the extent that agencies supply information in a consistent manner. We do not yet know, however, the extent to which practices and procedures to assess outcomes are standardized or non-standardized across agencies. We will know more about this after we field the survey using responses to survey items about agency question wording and outcome definitions. We believe it is best to wait until that time to decide in conjunction with FNS what qualifiers to place on the table titles and footnotes.

Another caution about Table A.16.1 and its cross-tables is clarity for users who may be lulled into imagining the statistics in this table and the cross-tables are parameter estimates for some given population of woman, infants and/or children instead of an actual, defined population of WIC agencies. Statistical inference and/or hypothesis testing is not an objective of the study, nor is it suggested, but I found myself having to use constant vigilance against subconscious inference making and hypothesis testing for an imagined population of woman, infants and children instead of the actual population of agencies.

[Mathematica:] We will remind readers in the report that statistics are agency-level statistics such as “average breastfeeding initiation rate across agencies” and not individual-level such as “breastfeeding initiation rate of WIC participants.”

The percentage tables associated with subsection Breastfeeding Policies and Practices are more straightforward in terms of ease of interpretation and user clarity.

Table A.16.2 receives the same cautions as I give for Table A.16.1.

[Mathematica:] Our response to the comment for Table 16.1 applies here as well.

The expected table for correlations between breastfeeding rates and policies and practices by agency characteristics and by local population characteristic shares the same caution as Tables A.16.1 and A.16.2.

[Mathematica:] Our response to the comment for Table 16.1 applies here as well.

Ditto for Table A.16.3. This is another good cross-tabulation; it will be interesting from several perspectives.

[Mathematica:] Our response to the comment for Table 16.1 applies here as well.

I’m assuming some appropriate, known, external weighting will be applied before Tables A.16.1, 2, and 3 are produced at the state, region, and national level. I did not notice comments describing tabulations to the state, region, and national level.

[Mathematica:] Because this is a census of agencies, compared to a sample, weights will adjust solely for agency nonresponse. We plan to construct a single set of weights for State WIC agencies and a separate set of weights for local WIC agencies. We plan to present results at the national, FNS region, and State level, but in each case we will use either the State or local WIC agency weight and use the State variable to define the subgroup. To produce a national level estimate of the agency percentage of infants that are breastfed exclusively, we will estimate a weighted average of the percentages across all State WIC agencies using the State-level weights. Similarly, to produce a regional estimate, we will estimate a weighted average of the percentages across all State WIC agencies in a given region using the State-level weights. Tables that show State-level statistics will use local WIC agency data and local WIC agency weights. Thus, to define estimates in a specific geography, we will restrict the set of agencies to those in the State or region of interest and use the State WIC agency weight or the local WIC agency weight.

B.1. Respondent Universe and Sampling Methods

Based on the materials included in this docket and from a brief examination of the organizational structure of FNS and its relationship with state and local agencies, this collection should be straightforward, i.e., accepted by and responded to by the agencies. The agency universe is already constructed. Response is required by the Healthy Hunger-Free Kids Act for all types of agencies involved (non-profit, profit, government, etc.). If the data collection instrument is properly advertised as mandatory, systematic non-response should not be a problem that requires intricate investigation utilizing ACS or WIC PC data.

[Mathematica:] We agree.

B.3. Methods to Maximize Response Rates and to Deal with Nonresponse

See my comments in B.1.

[Mathematica:] We agree.