Supporting Statement: Part A
Evaluation of Childhood
Obesity Prevention and Control Initiative:
New York City Health Bucks Program
December 28, 2009
Submitted by
Co-Technical Monitors:
Jan Jernigan and Gayle Payne
Division of Nutrition, Physical Activity and Obesity
National Center for Chronic Disease Prevention and Health Promotion
Centers for Disease Prevention and Control
4770 Buford Highway, NE, MS K-03
Atlanta, GA
Telephone (770) 488-5224
Contents
List of Appendices ii
A. Justification 1
A-1. Circumstances Making the Collection of Information Necessary 1
A-2. Purpose and Use of Information Collection 4
A-3. Use of Improved Information Technology and Burden Reduction 6
A-4. Efforts to Identify Duplication and Use of Similar Information 7
A-5. Impact on Small Businesses or Other Small Entities 8
A-6. Consequences of Collecting the Information Less Frequently 8
A-7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5 8
A-8. Comments in Response to the Federal Register Notice 8
A-9. Explanation of Any Payment or Gift to Respondents 9
A-10. Assurance of Confidentiality Provided to Respondents 9
A-11. Justification for Sensitive Questions 13
A-12. Estimates of Annualized Burden Hours and Costs 13
A-13. Estimates of Other Total Annual Cost to Respondents or Record Keepers 17
A-14. Annualized Cost to the Government 17
A-15. Explanation for Program Changes or Adjustments 18
A-16. Plans for Tabulation and Publication and Project Time Schedule 18
A-17. Reason(s) Display of OMB Expiration Date is Inappropriate 20
A-18. Exceptions to Certification for Paperwork Reduction Act Submissions 20
References 21
Appendix A: Authorizing Legislation
Appendix B: Federal Register Notice
Appendix C: Local Community Organization Survey Instrument
Appendix D-1: Farmers’ Market Manager Survey Instrument
Appendix D-2: Fact Sheet for Distribution to Farmers’ Market Managers
Appendix E-1: Farmers’ Market Vendor Survey Instrument
Appendix E-2: Fact Sheet for Distribution to Farmers’ Market Vendors
Appendix F-1: Focus Group Moderator Guide – Farmers’ Market Vendors
Appendix F-2: Focus Group Consent Form – Farmers’ Market Vendors
Appendix G-1: Farmers’ Market Consumer Survey Instrument
Appendix G-2: Fact Sheet for Distribution to Farmers’ Market Consumers
Appendix H-1: Focus Group Moderator Guide – Farmers’ Market Consumers
Appendix H-2: Focus Group Consent Form – Farmers’ Market Consumers
Appendix I: NYC Health Bucks Neighborhood Resident Survey Instrument
Appendix J: IRB Approval Letter
Appendix K: List of Individuals Consulted During Project Development
A-1. Circumstances Making the Collection of Information Necessary
Childhood obesity is a major public health concern. One out of every five children is affected by overweight or obesity in the United States, making it the most prevalent nutritional disease of this population (Mokdad et al. 2004). Reducing the prevalence of obesity among children and adolescents in the U.S. below 5% is a key national health objective for the Healthy People 2010 agenda (DHHS 2000). However, the most recent data from the National Health and Nutrition Examination Survey indicate a flat trend in obesity rates among US children and adolescents since the year 2000, with 15.5% of children and adolescents above the 95th percentile of BMI for age in 2005-2006 (Ogden et al. 2008). Obesity and overweight are particularly prevalent among children and adolescents of lower socioeconomic status, although there is some evidence that this pattern varies by ethnicity and that the gap may be narrowing over time (Gordon-Larsen et al. 2003; Wang & Zhang 2006). Additionally, minority groups are at greater risk than their white counterparts (Mirza et al. 2004; Wang 2000; Ogden et al. 2006).
Increased consumption of fruits and vegetables has been found to reduce long-term obesity risk (He et al. 2004), as well as risk of heart disease and some cancers (Steinmetz & Potter 1996; Riboli & Norat 2003; Dauchet et al. 2006). Experimental interventions involving the addition of fruits and vegetables to the diet have demonstrated short-term effectiveness in reducing body weight in some cases, particularly when paired with reduction of dietary fat and/or overall caloric intake, although it is difficult in these studies to identify the influence of increased fruit and vegetable intake separately from that of overall dietary change (Rolls et al. 2004; Jebb 2005; Carlton-Tohill 2007).
Nevertheless, relatively few children and adolescents consume the USDA recommended minimum standard of five servings a day of fruits and vegetables. Only 14 percent of boys and 10 percent of girls aged 4-8 ate five or more servings of fruits and vegetables per day in 1999-2000, compared with 18 percent of boys and 20 percent of girls aged 9-13, and 37 percent of boys and 28 percent of girls aged 14 to 18 (Guenther et al. 2006). Krebs-Smith et al. (1996) found that less than one in five children consume the recommended five servings, and moreover that French fries comprised nearly a quarter of vegetables consumed by youths.
In response to this growing public health crisis, the Division of Nutrition, Physical Activity, and Obesity (DNPAO) at the Centers for Disease Control and Prevention (CDC), is working to reduce obesity and its related health conditions via a multi-pronged approach including active identification of promising local programs and policies designed to prevent childhood obesity. Priority is being given to those programs and policies targeting improved eating habits and physical activity levels among children in low-income communities (CDC 2009).
In keeping with this broad set of objectives, CDC has contracted with Abt Associates, Inc. to perform an evaluation of the NYC Health Bucks initiative, a program administered by the New York City Department of Health and Mental Hygiene (NYC DOHMH) through three District Public Health Offices (DPHOs) which distributes $2 coupons for the purchase of fresh fruits and vegetables at farmers’ markets in three underserved New York City neighborhoods. In partnership with local community groups, DPHOs distribute these $2 coupons – or “Health Bucks” – to residents in the South Bronx, North and Central Brooklyn, and East and Central Harlem. At farmers’ markets in these neighborhoods accepting Food Stamp/Supplemental Nutrition Assistance Program (SNAP) benefits via the Electronic Benefit Transfer (EBT) system, there is an added incentive to consumers. For every $5 in EBT purchases, an additional $2 Health Buck coupon is provided, either to be spent then or at a later time.
The evaluation will consist of two main components: a process evaluation, to identify barriers and facilitators to implementation of this financial incentive program in order to improve on the existing program model and to inform states wishing to implement similar interventions, and an outcome evaluation, to evaluate the effectiveness of NYC Health Bucks in increasing fresh fruit and vegetable access, purchase, and consumption in targeted low-income populations.
The New York City Health Bucks initiative is an ideal candidate for program evaluation in support of ongoing CDC objectives. As noted by the GAO in its recent report on options for delivering financial incentives to SNAP participants for purchasing targeted foods (2008), there is currently a lack of reported research on the effectiveness of incentive programs of this type, despite their increasing frequency and popularity. A more formalized, rigorous evaluation of the NYC Health Bucks program will therefore play a key role in informing the broader policy and research communities.
CDC authority to conduct this information collection is provided by the Public Health Service Act (Appendix A).
Privacy Impact Assessment: This is a new information collection, and has not therefore been previously assessed for privacy impact. In the remainder of this section, we provide an overview of the proposed data collection system, an outline of the types and sources of information to be collected, and a description of the website data collection component.
Overview of the Data Collection System: Data will be collected from five broad groups of respondents as part of this information collection:
1. local community organizations involved in distributing Health Bucks to individuals;
2. farmers’ market managers operating New York City farmers’ markets;
3. farmers’ market vendors selling at New York City farmers’ markets;
4. farmers’ market consumers at New York City farmers’ markets; and
5. residents of neighborhoods in which the NYC Health Bucks program operates.
Survey data will be collected from each of these five groups to inform process and outcome evaluation goals. Data collection procedures for surveys have been tailored to minimize burden while taking into consideration differences in feasibility, access, and information needs for each group. Local community organizations will complete a web-based questionnaire at the conclusion of the farmers’ market season. Farmers’ market managers will complete a written survey administered by mail during the farmers’ market season, with in-person follow up by trained interviewers on site at farmers’ markets for managers who do not respond to the initial mailing. Farmers’ market vendors will complete a written survey administered by trained interviewers on site at farmers’ markets. Trained interviewers will also conduct written point-of-purchase intercept surveys with farmers’ market consumers on site at farmers’ markets. Finally, a random-digit-dial telephone interview will be conducted with residents of neighborhoods in which the NYC Health Bucks program operates.
Additionally, two groups of respondents, farmers’ market consumers and vendors, will participate in a set of focus groups designed to explore in greater detail preliminary survey findings, for a total of seven separate data collection activities.
Items of Information to be Collected: Instruments and consent documents for all surveys and focus groups to be conducted as part of this information collection have been provided as Appendixes C through I. Local community organizations will primarily provide survey information about their experiences with the NYC Health Bucks program and effects on organizational operations in support of the process evaluation and program improvement. Farmers’ market managers and vendors will also be surveyed about their experiences with the program to facilitate the process evaluation, and will provide general information in support of the outcome evaluation about factors influencing propensity to operate farmers’ markets in underserved New York City communities. Surveys of farmers’ market consumers will similarly support both the process and outcome evaluations, asking both about experiences with the NYC Health Bucks program and use of SNAP benefits at farmers’ markets, as well as targeted program outcomes such as perceived access to and consumption of fresh fruits and vegetables. Focus groups with farmers’ market consumers and vendors will explore preliminary survey findings in greater depth. Finally, telephone surveys of residents of neighborhoods in which the NYC Health Bucks program operates will yield important context about the reach and penetration of the program, as well as providing a point of comparison between farmers’ market consumers and typical residents in these neighborhoods.
Information in identifiable form (IIF) will be collected by the contractor as part of survey data collection procedures from local community organizations and farmers’ market vendors and managers, and for scheduling purposes as part of focus groups from farmers’ market vendors and consumers. Additionally, as part of standard random-digit-dial (RDD) telephone survey procedures, telephone numbers will be temporarily stored by the data collection contractor in electronic form for residents of NYC Health Bucks neighborhoods responding to the survey. However, no IIF will be transmitted to the CDC as part of this study. Table A-1-A summarizes collection of IIF for this study. See Section A-2 for additional details related to collection of IIF for each data collection activity.
The farmers’ market consumer survey and the survey of NYC Health Bucks neighborhood residents request some information that might be considered sensitive by a portion of respondents. These surveys include questions on personal demographics, income ranges, and participation in food assistance programs, which are not considered highly sensitive. These survey questions are necessary in order to accurately describe participants in the NYC Health Bucks program and to assess whether the program is reaching its intended target population. In addition, participants may disclose information related to food assistance program eligibility and participation as part of farmers’ market consumer focus groups. See Section A-11 for additional details related to collection of sensitive information for each data collection activity.
Identification of Website(s) and Website Content Directed at Children Under 13 Years of Age: As described above, local community organizations will submit survey responses via a secure website as part of this information collection. The website will be developed and hosted by the prime contractor’s wholly-owned data collection subsidiary, Abt SRBI. The website will contain no content directed at children under thirteen years of age.
Table A-1.A. Collection of Information in Identifiable Form (IIF) and Sensitive Information |
||||
Type of Respondents |
Data Collection Activity |
IIF Collected |
Purpose of IIF |
Sensitive Information Collected (not considered highly sensitive) |
Local Community Organizations |
Local Community Organization Survey |
|
|
None |
Farmers’ Market Managers |
Farmers’ Market Managers Survey |
|
|
None |
Farmers’ Market Vendors |
Farmers’ Market Vendor Survey |
|
|
None |
Farmers’ Market Vendor Focus Group |
|
|
None |
|
Farmers' Market Consumers |
Consumer Point-of-Purchase Survey |
None |
N/A |
|
Consumer Focus Group |
|
|
|
|
NYC Health Bucks Neighborhood Residents |
Neighborhood Resident Survey |
|
|
|
A-2. Purpose and Use of Information Collection
This will be a one-time data collection. The purpose of the information collection is to support a rigorous process and outcome evaluation of NYC Health Bucks. Evaluation results will meet a pressing need for assessment of promising community-based strategies for combating rising rates of obesity. In particular, the goals of this evaluation study are:
To assess the ability of NYC Health Bucks to improve nutrition behaviors and access to fresh fruits and vegetables within its target population;
To identify factors that serve as barriers and facilitators to program implementation and expected outcomes, as well as address the issue of any health disparities;
To provide feedback to the NYC DOHMH for the purposes of program improvement; and
To share program results with other states and localities that may be interested in implementing a similar program.
Results of the evaluation will be disseminated in the form of a summary final report, a manuscript submitted for publication in a peer-reviewed journal, and an implementation toolkit providing practical guidance on how to implement and evaluate similar programs.
Privacy Impact Assessment:
As stated above, this information collection will be used to inform a rigorous process and outcome evaluation of the NYC Health Bucks program. Because the program is intended to influence outcomes at both the community and individual level, and because Health Bucks distribution occurs through two different mechanisms (through local community organizations and on-site at farmers’ markets as an incentive to SNAP participants), it is necessary to gather information from a number of key respondent groups. Local community organizations, farmers’ market managers, and farmers’ market vendors will provide information on distribution and reimbursement of Health Bucks, and managers and vendors will additionally provide information on how the program influences their propensity to operate in underserved New York City neighborhoods. Farmers’ market consumers will describe their experiences with the program and with farmers’ markets in general, and NYC Health Bucks neighborhood residents will provide valuable context for comparison with farmers’ market consumers and information on the reach and influence of the program as a whole. Information must be collected from each of these groups in order to form a complete picture of the program. Data will be used by the contracting organization, Abt Associates, Inc., to evaluate the NYC Health Bucks program for CDC, to support program tracking and planning by the NYC DOHMH, and to disseminate results to other interested parties via a final report, a journal manuscript, and an implementation toolkit.
As described in Section A-1, IIF will be collected as part of survey data collection from local community organizations, farmers’ market managers, farmers’ market vendors, and residents of neighborhoods in which NYC Health Bucks operates. No IIF will be collected as part of the farmers’ market consumer survey. IIF will be accessible only to staff at Abt and its wholly-owned data collection subsidiary, Abt SRBI, who are directly involved in data collection activities. No IIF will be shared with CDC or the FMFNY. Names of organizations responding to the local community organization survey will be shared with the NYC DOHMH for tracking and planning purposes only. Responses will not affect the relationship of organizations or individuals with CDC, NYC DOHMH, or FMFNY.
Potentially sensitive information on income, personal demographics, and food assistance program participation is being collected from respondents as part of the farmers’ market consumer and NYC Health Bucks neighborhood resident surveys, and potentially sensitive information on food assistance program eligibility and participation may be shared by participants during farmers’ market consumer focus groups. This information is not considered highly sensitive, and is necessary for accurately describing these groups of respondents and for assessing the reach and impact of the program on its target population. Inclusion of personal demographic information on the consumer and neighborhood resident surveys is also necessary for assessing non-response bias. The questions employed on these surveys are either borrowed or adapted from standardized measures and have been used extensively in prior studies with no evidence of harm. As part of standard consent procedures, respondents will be explicitly informed that they have the right to refuse to answer any question they may deem sensitive. Protections will be in place to safeguard respondent privacy to the maximum extent allowed by law.
A-3. Use of Improved Information Technology and Burden Reduction
The proposed data collection will use a variety of information technologies and other techniques to reduce respondent burden. Specific qualitative and quantitative data collection methods (described in greater detail below) were chosen for each set of respondents in order to balance considerations of burden and appropriateness for the targeted group. In all cases, we will avoid collection of information that is available from other sources (e.g. administrative records), and instruments will cover only domains that are directly relevant to the study research questions.
Focus Groups: In-person focus groups with farmers’ market vendors and consumers targeted by the NYC Health Bucks intervention will provide information not readily captured by quantitative surveys or online focus groups, particularly since web access may be limited for these respondents. Each two-hour focus group will be comprised of eight participants. This number is sufficiently large to allow the expression of a diversity of viewpoints, yet sufficiently small so that participants may respond to each others’ comments, discuss topics in-depth, and allow the moderator to explore emerging themes. Each focus group will be scheduled at a mutually convenient time for all participants at a convenient, central location in New York City easily accessible by public transportation.
Surveys: As a general matter, in order to reduce respondent burden associated with completion of surveys as part of this information collection, experts in survey methodology and individuals with extensive knowledge about NYC Health Bucks will review and streamline all instruments to avoid unnecessary questions and redundancy. Where feasible, we will additionally employ information technology such as online survey administration and computer-assisted telephone interview procedures in order to further minimize respondent burden.
Local community organization surveys will be completed online to reduce respondent burden; because the application process for NYC Health Bucks is completed online, we are confident that the internet is accessible to this group of respondents.
In contrast, because farmers’ market managers and vendors work on site at farmers’ markets, their internet access may be limited; online administration of surveys will not therefore be feasible for these two groups. Similarly, internet access may be limited for consumers shopping at farmers’ markets in low-income New York City neighborhoods. Even more importantly, efficient identification of farmers’ market consumers requires on-site intercepts. Farmers’ market consumer surveys will therefore be conducted in-person at farmers’ markets.
Telephone surveys of neighborhood residents in areas in which NYC Health Bucks operates will be conducted using computer-assisted telephone interview (CATI) technology. CATI is an efficient interviewing mode that reduces respondent burden and improves the quality of the data collected. The CATI system will include logic checks and skip-pattern controls to ensure that respondents receive the appropriate questions and that the interview process goes smoothly. These programmed checks also identify inconsistent responses, allowing the interviewer to resolve discrepancies during the interview.
A-4. Efforts to Identify Duplication and Use of Similar Information
A significant effort to avoid duplication has been made by conducting an extensive literature review and interviews with key experts and program stakeholders early in the formative phase of this evaluation project.
We initially performed a review of scientific articles around the general topic of financial incentives for purchasing targeted foods and concluded that no existing sources are available to provide the data necessary to answer the study’s research questions. In a recent report on options for delivering financial incentives to SNAP participants for purchasing targeted foods, the General Accounting Office confirmed a lack of reported research on the effectiveness of incentive programs of this type (GAO 2008). Our literature review did identify several smaller studies that demonstrated positive effects of coupon programs in farmers’ markets on fruit and vegetable consumption in low-income populations (Anderson et al. 2001, Herman et al. 2006). Additionally, a currently ongoing experimental study of the WIC Farmers’ Market Nutrition Program (FMNP) is being conducted for the CDC by Emory University researchers in Atlanta (see below), although results of this study are not yet available. However, these studies were relatively limited in scope and provided little information about program implementation, a major focus of the process evaluation component of the proposed data collection.
In addition to the literature review described above, during the Summer and Fall of 2009, we (1) conducted site visits to three farmers’ markets in which NYC Health Bucks operates, and (2) held informal interviews – either in-person or via telephone - with nationally recognized food and nutrition experts and NYC Health Bucks program stakeholders. Key informants included:
Sabrina Baronberg, Deputy Director and NYC Health Bucks Coordinator, NYC DOHMH
Alyson Abrami, Farmers’ Market Nutrition Education Coordinator (Stellar Markets), NYC DOHMH
Kimberly Bylander, Brooklyn District Public Health Office (DPHO)
Darrin Taylor, Bronx District Public Health Office (DPHO)
Diane Eggert, NYC Health Bucks and EBT Program Coordinator, Farmers’ Market Federation of New York (FMFNY)
Alexis Stevens, Greenmarket Farmers Markets, Council on the Environment of New York City (CENYC)
Elvira Rella, Program Coordinator, Urban Health Plan
Tamara Dawson, Program Coordinator, Bedford-Stuyvesant Campaign Against Hunger
Kathleen Adams and Julie Gazmararian, Researchers, WIC Farmers’ Markets’ Cost and Influence on Fruit and Vegetable Consumption Study, Rollins School of Public Health, Emory University
Although the NYC DOHMH and FMFNY have conducted some internal evaluation activities, their efforts necessarily focused on operational details meant to assist program staff and administrators in improving the program, including participation and redemption rates, with additional qualitative information gleaned from pre-season and post-season surveys conducted with organizations responsible for issuing Health Bucks in the community. Limited internal time and resources are available, however, to augment these efforts in the context of a broader evaluation designed to assess the effectiveness of the NYC Health Bucks program in influencing nutrition behaviors and outcomes in targeted populations, and to formally assess barriers and facilitators to program implementation for the purposes of informing other localities wishing to initiate similar programs. The various data collection instruments for this project have therefore been carefully constructed to build upon those activities already in place as part of a more comprehensive examination of processes and outcomes across communities served.
Based on these findings, it remains clear that existing evidence on financial incentive programs and their effectiveness at promoting healthy food choices remains limited, despite substantial interest in the public health sector. The planned information collection thus represents a unique effort to fulfill this need.
A-5. Impact on Small Businesses or Other Small Entities
As part of this information collection, data will be obtained from farmers’ market vendors, farmers’ market managers, and from participating local community organizations, all of whom may qualify as small entities as defined by OMB. Efforts made to ensure minimal burden for each of these groups are outlined in greater detail in Section B.2 (”Procedures for the Collection of Information”).
A-6. Consequences of Collecting the Information Less Frequently
This evaluation effort is a one-time information collection.
A-7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
This request fully complies with the guidelines of 5 CFR 1320.5.
A-8. Comments in Response to the Federal Register Notice
A 60-day Notice was published in the Federal Register on December 17, 2009 (Vol. 74, No. 241, pp. 66970-66971; see Appendix B). There were no public comments.
B. A list of consultants on the development of the project is included as Appendix K.
A-9. Explanation of Any Payment or Gift to Respondents
Participants in all focus groups will receive a $35 cash incentive to compensate them for their time and for transportation to and from the focus group facility. In our experience, this size incentive will minimize the labor and cost involved in recruiting participants, keep attrition low, and yet it is not so large as to be an inappropriate influence on voluntary participation.
Consumers interviewed at farmers’ markets will receive a round-trip MetroCard (with a value of $4.50) as an incentive to participate. No payments or incentives will be provided for participation in the local community organization survey, farmers’ market manager survey, farmers’ market vendor survey, or the telephone survey of neighborhood residents.
A-10. Assurance of Confidentiality Provided to Respondents
This information collection will be conducted on behalf of CDC by a contractor, Abt Associates, Inc. For each component of data collection, procedures will be in place to safeguard respondent privacy to the fullest extent of the law. Institutional Review Board (IRB) approval has been approved for this study; an approval letter from the contractor IRB has been supplied as Appendix J. All study procedures will conform to ethical practices for collecting data from human participants based on the Federal policy for Protection of Human Subjects in Research (45 CFR Part 46).
All study staff involved in this information collection will receive a comprehensive training prior to conducting study-related activities. Training will address study purpose and requirements, study procedures, and issues of security and privacy. Standards for the surveys performed for the Federal government will be communicated to staff, highlighting the importance of the interviewers' responsibilities.
Privacy Impact Assessment Information
A. Privacy Act Determination
This submission has been reviewed by staff in the CDC Information Collection Review Office, who determined that the Privacy Act is applicable to the telephone survey of NYC Health Bucks neighborhood residents. Two of the seven proposed data collection activities (the telephone survey of NYC Health Bucks neighborhood residents and the farmers’ market consumer focus group), include collection of both IIF and data that may be considered sensitive by some portion of respondents (Table A-1-A). However, because IIF collected during the farmers’ market consumer focus group will not be linked to individual response data, the Privacy Act applies only to the telephone survey of NYC Health Bucks neighborhood residents. Data for the telephone survey data collection activity will be collected and protected in accordance with Privacy Act system notice 09-20-0136, “Epidemiologic Studies and Surveillance of Disease Problems.”
The telephone survey of NYC Health Bucks neighborhood residents includes questions about income ranges, personal demographics, and participation in food assistance programs that may be considered sensitive by some portion of respondents, although these questions are not considered highly sensitive. As part of standard RDD procedures for this survey, IIF in the form of telephone numbers will be collected and maintained by the contractor in electronic form on a temporary basis for tracking and quality control purposes until data collection has been completed. Although these telephone numbers will be stored separately from response data on a secure server and will be labeled only by respondent ID number, because the data can be re-linked to response data including sensitive information by the contractor, the Privacy Act applies to this data collection activity.
However, the Privacy Act does not apply to the remaining six data collection activities to be conducted as part of this study. No personally sensitive information is being collected as part of surveys of local community organizations, farmers’ market managers, or farmers’ market vendors, or during farmers’ market vendor focus groups. Although information on income range, personal demographics, and participation in food assistance programs is being collected as part of the farmers’ market consumer survey, no IIF is being collected as part of this data collection activity. Finally, although participants in farmers’ market consumer focus groups may reveal information about eligibility for and participation in food assistance programs that may be considered sensitive by some respondents, names and telephone numbers for focus group participants will be collected by the contractor only for scheduling purposes, and will not be linked to individual response data.
Data collection will be conducted by Abt Associates, the contractor hired by CDC for this project, and its wholly-owned data collection subsidiary Abt SRBI. No individual response data will be transmitted to or maintained by CDC as part of this study.
B. Information Security
The data collection contractor study staff will safeguard respondent privacy regarding all aspects of a respondent’s participation in this study. Any individual information gathered in the course of the study will be discussed only with individuals directly involved with this study. All data will be identified only by an ID number and will be kept in a locked location (hard copy data) or password protected (electronic data). Hard copies of survey instruments will be kept in a locked file in the research offices of the data collection contractor and will only be accessible to project staff.
Three years following completion of the study all data (both written and electronic) will be destroyed. This will include signed consent forms, hard copy instruments, and electronic databases. Data resulting from this evaluation will be reported in the aggregate and no identifiable participant data will be reported. Expected products may include internal reports, final reports for CDC, toolkits, and manuscripts for publication.
Below we describe in greater detail procedures that will safeguard privacy for focus groups and surveys to be conducted as part of this data collection.
Focus Groups: Focus group response data will be collected by the contractor’s study staff. Sessions will be audiotaped. Transcripts derived from the audiotapes will be read and analyzed only by study staff, and the audiotapes will be destroyed after transcription. Each respondent will be assigned a unique identifier that will be used to track and store data. The only IIF that will be obtained by the data collection contractor are the participants’ names and phone numbers for scheduling purposes; this information will not be linked to the response data or shared with CDC or NYC DOHMH. In addition, results will be analyzed and reported in the aggregate so that it will not be possible for an individual to be identified. First names only will be used during the group discussion, and only unique identifiers assigned for the purposes of this study as described above will be included in transcripts.
Surveys: During data collection for the neighborhood resident survey, IIF in the form of respondent telephone numbers will be temporarily stored in the contractor’s CATI system in isolation from other individual response data, in order to facilitate callbacks and track interview completion. Telephone numbers will be labeled only with a unique respondent identification number and will be deleted from the system immediately upon completion of data collection. However, while this data collection activity is in process, the respondent identification number could potentially be used to link telephone numbers to individual response data. We expect the telephone survey to last no longer than eight weeks; telephone numbers will therefore be stored in our CATI system for a maximum than 60 days. All files used in the CATI system will be stored on a secure server, and access to those files will be restricted to Abt SRBI staff working on the project. Telephone numbers will not be included in any data sets delivered to Abt or CDC staff.
IIF will be collected from local community organizations and farmers’ market vendors and managers. Local community organizations will complete online surveys via a secure website. The name of the organization and the individual completing the survey will be pre-populated in the online survey along with the e-mail address and telephone number for each organization using information from pre-season applications provided to the Abt team by the NYC DOHMH. The purpose of collecting this identifiable data is to link survey responses for a community organization from pre- to post-farmers’ market season. Once the survey responses are linked, the identifying data will be stored separately from the survey responses on a secure server.
Mailing addresses of farmers’ market managers will be provided to the Abt team by the NYC Health Bucks implementation contractor, the Farmers’ Market Federation of New York (FMFNY) and will be used to send out the initial farmers’ market manager survey mailing. To ensure appropriate follow-up with managers on site at farmers’ markets as necessary, managers will be asked to provide their name and the name and sponsoring organization of the farmers’ market they manage on their written surveys. Similarly, vendor ID numbers and farm/business names will be collected to ensure only one response per vendor/farmer is received. However, pages containing identifiable information from these two surveys will be separated from survey responses and stored in a secure, locked location following completion. All surveys will be mailed in business reply envelopes to the Abt study team; the NYC DOHMH and CDC will not be able to link survey responses with respondent names for these two surveys.
Data from hard copy surveys will be entered into a password-protected electronic database by trained study staff. Information collected via the phone and online surveys will be electronically secure. Only staff directly involved in data collection will have access to the survey program files and collected data. Access to the web survey of community organizations is restricted, meaning it can only be accessed by identified respondents who click their customized link in an invitation email, or other parties (for testing purposes) who are given "test" links to review the survey as respondents. The survey will not be accessible to anyone who has not been provided with such a link. Cookies will not be used as part of this survey. The surveys will be administered using Secure Socket Layer (SSL) encryption. By default, users connect with the highest level of encryption enabled in their browser. In addition to data encryption, web interviewing software only transmits small amounts of data at a time, usually at the input screen level (e.g., each time a respondent hits the NEXT button). Data are transmitted without the associated questions, so unlinked responses will not be informative in the unlikely event that encryption is broken.
All Abt and Abt SRBI servers, workstations, and notebook computers are protected from viruses, spyware, and worms. The devices are continuously updated with the latest definition file and scanned regularly from a central server. Devices that are flagged as being infected are removed from the network and cleaned or quarantined before reattaching.
C. Consent Procedures
The consent procedures described in this section have received IRB review and approval.
Focus Groups: Prior to each focus group, each respondent will be asked to sign a consent form to participate. The consent form will explain the study procedures for assuring that the answers provided by the respondent will remain secure. The consent form explains that (a) participation in the focus group is voluntary and there are no penalties for refusing to participate at any time during the focus group; (b) the respondent can refuse to discuss any topic; (c) data will be stored in de-identified files and (d) no names will be used in any evaluation reports. At the time of the focus groups, respondents will be asked to sign two copies of the consent form. One will be retained by the focus group moderator (a member of the Abt study team), and the second copy will be given to the respondent for his/her files. Copies of focus group consent forms are included in Appendixes F-2 and H-2.
Surveys: For the self-administered surveys (local community organization survey, farmers’ market manager survey, farmers’ market vendor survey), information will be distributed along with the survey to explain the purpose of the study and stipulations regarding respondent privacy; respondents will demonstrate their consent by voluntarily completing and returning their surveys. For the local community organization survey, a printable online consent document at the beginning of the survey will provide this information. For the manager and vendor surveys, hard-copy fact sheets will be distributed along with the surveys. Copies of the manager and vendor fact sheets are provided in Appendix D-2 and E-2, respectively.
Verbal consent will be obtained from NYC Health Bucks neighborhood residents participating in our telephone survey, and from consumers interviewed as part of the point-of-purchase survey on site at farmers’ markets. Farmers’ market consumers will also be provided with a written fact sheet before completing their interview; a copy of the consumer fact sheet is provided in Appendix G-2.
Procedures to safeguard privacy for the respondents will be described as part of the consent process for all surveys, and the interviewers will be knowledgeable about privacy safeguards and will be prepared to describe them in detail or to answer any related questions raised by respondents. Respondents will additionally be informed prior to survey participation that their responses will be treated in a secure manner, and that all information used in any reports or manuscripts resulting from this evaluation will be reported in the aggregate. Lastly, respondents will be advised of the nature of the activity, the length of time it will require, that participation is purely voluntary, and assured that they will not incur penalties if they wish not to respond to the survey as a whole or to any specific questions.
D. Voluntary Nature of Participation
As part of standard consent procedures for each data collection activity performed as part of this information, respondents will be informed that their participation is voluntary, and that there will be no penalty if they do not participate. Even if respondents agree to participate, they may refuse to respond to any individual question or questions. Their decisions will not affect relationships with CDC, the NYC DOHMH, or the FMFNY.
A-11. Justification for Sensitive Questions
IIF will be collected as part of survey data collection from local community organizations, farmers’ market managers, and farmers’ market vendors, however, there are no personally sensitive questions contained in the farmers’ market vendors focus group interview guide, or in local community organization, farmers’ market manager, and farmers’ market vendor survey instruments. The IIF is necessary for scheduling and tracking purposes.
The farmers’ market consumer and NYC Health Bucks neighborhood resident surveys request some information that might be considered sensitive by a portion of respondents, although the information is not highly sensitive. These potentially sensitive questions are limited to personal demographics, household income ranges, and participation in food and nutrition assistance programs – i.e. Food Stamps or SNAP (Supplemental Nutrition Assistance Program), WIC (Special Supplemental Nutrition Program for Women, Infants, and Children), and similar financial incentive programs (the Senior and WIC Farmers’ Market Nutrition Programs, or FMNP, in particular). The questions employed are either borrowed or adapted from standardized measures and have been used extensively in prior studies with no evidence of harm. As part of standard consent procedures described in greater detail in the previous section, respondents will be explicitly informed that they have the right to refuse to answer any question they may deem sensitive.
The question about annual household income is necessary to ascertain whether or not the program is reaching its intended target population (low-income individuals in three underserved neighborhoods in Brooklyn, the Bronx, and Harlem) and will allow respondents to select the income range that best reflects their current economic situation.
Questions about food and nutrition assistance program participation are critical, as Health Bucks are provided largely as an incentive to users of Food Stamp/SNAP benefits at farmers’ markets, and because food and nutrition assistance program participants comprise an important target population for this study due to high rates of food insecurity and poor nutrition. The survey will allow respondents to select one or more programs of current or past participation, without probing for additional details.
Questions regarding race and ethnicity are necessary to accurately describe NYC Health Bucks participants and non-participants, and both the format and administration of these questions will conform to both HHS and OMB policies regarding the collection of this information. Respondents will first be asked to report whether they are of Hispanic or Latino ethnicity, and then to select the race option(s) that best describe them. Respondents will be able to choose more than one option with respect to race.
A-12. Estimates of Annualized Burden Hours and Costs
A. The proposed data collection does not impose a financial burden on respondents nor will respondents incur any expense other than the time spent completing the surveys and focus groups.
The estimated annual burden for five groups of study respondents (local community organizations, farmers’ market managers, farmers’ market vendors, farmers’ market consumers, and NYC Health Bucks neighborhood residents) as part of seven individual data collection activities (two sets of focus groups and five surveys) is identified in Table A-12.A. The total burden hours are 660 hours for the proposed information collection. Assumptions used to derive these estimates are detailed below.
Surveys
Local community organizations – There were 172 local community organizations that applied to receive Health Bucks during the 2009 farmers’ market season; we assume that approximately 200 will participate in 2010. Surveys (see Appendix C) will be completed online at one time point (end of the 2010 farmers’ market season), and will take approximately ten minutes to complete, for a maximum total burden of approximately thirty-three hours. Data collected will be used to assess organizations’ motivations for participating in the program and any barriers or facilitators encountered. The DOHMH currently requires completion of post-season surveys as a condition of participation in NYC Health Bucks; the proposed data collection will replace the existing DOHMH post-season survey and is thus not expected to place substantial new burden on this group of respondents.
Farmers’ market managers – There are approximately 105 farmers’ markets in the five boroughs in New York City: about 50 markets that accept Health Bucks, and about 55 non-Health Bucks markets. One manager from each of the 50 Health Bucks markets and one manager from each of 40 randomly-selected non-Health Bucks markets will be sampled. (See Section B-1 for a description of the methods to be used in selecting the random sample of non-Health Bucks markets.) Questionnaires (see Appendix D-1) will be administered via mail survey, with in-person follow-up on site at farmers’ markets for managers that do not return the initial mailing. The survey will take approximately eight minutes to complete, and will be administered once to each manager during the 2010 farmers’ market season, for a maximum total burden of approximately twelve hours.
Farmers’ market vendors – A written questionnaire (see Appendix E-1) will be distributed to all vendors selling at the 90 selected New York City farmers’ markets during the 2010 farmers’ market season in order to assess experiences with the Health Bucks program and factors influencing decisions to sell at markets in underserved neighborhoods. Surveys will be administered in person at farmers’ markets to all vendors; vendors who do not wish to immediately fill out the questionnaire (or who feel the survey should be completed by an individual not currently present on site) will be given a business return envelope to allow them to complete and return the survey at their convenience. We assume an average of five vendors from each of the 90 selected markets will complete surveys, for a total of 450 surveys. Each survey will take approximately seven minutes to complete, for an expected total burden of approximately fifty-three hours.
Farmers’ market consumers – In-person interviewers will administer two-page written surveys to farmers’ market consumers in all New York City farmers’ markets (see Appendix G-1). The survey will assess access to fresh fruits and vegetables at farmers’ markets and other sellers, fresh fruit and vegetable purchase and consumption, food insecurity, reasons for shopping at farmers’ markets, and experiences with using Health Bucks and SNAP benefits at farmers’ markets. We assume a total of 2,300 (an average of 40 per Health Bucks market and 20 per non-Health Bucks market) consumers will complete surveys. Each survey will take approximately seven minutes to complete, for an expected total burden of 268 hours.
Residents of NYC Health Bucks neighborhoods – A random-digit-dial (RDD) landline telephone survey (see Appendix I) of neighborhood residents during the farmers’ market season in or adjacent to ZIP codes in which NYC Health Bucks operates will assess access to fresh fruits and vegetables at farmers’ markets and other sellers, fresh fruit and vegetable purchase and consumption, food insecurity, and experiences shopping at farmers’ markets. Surveying respondents in neighborhoods in which the NYC Health Bucks program operates will allow us to characterize general awareness and penetration of the program in these areas. Furthermore, collecting data on characteristics of neighborhood residents as a whole will allow us to make direct comparisons with characteristics of farmers’ market consumers in these neighborhoods surveyed on-site. Approximately 1,000 residents will be surveyed, with equal sample sizes in each of the three New York City neighborhoods in which NYC Health Bucks operates. Each survey will take approximately nine minutes to complete, for an expected total burden of approximately 150 hours.
Focus Groups
Farmers’ market vendors – A total of eight vendors will participate in each of three farmers’ market vendor focus groups during the 2010 farmers’ market season in order to explore decisions to operate in underserved, low-income neighborhoods and experiences with the NYC Health Bucks program, for a total of twenty-four vendor respondents. Vendors will be chosen to represent a mix of market characteristics, including use of wireless EBT technology, participation in NYC Health Bucks, and operation in NYC Health Bucks target neighborhoods. Each focus group will last approximately two hours, for an expected total burden of approximately forty-eight hours (see Appendix F-1).
Farmers’ market consumers – A total of eight consumers will participate in each of six farmers’ market consumer focus groups during the 2010 farmers’ market season to assess motivations for shopping at farmers’ markets and experiences with NYC Health Bucks, for a total of forty-eight consumers. Consumers will be chosen to represent a mix of characteristics including neighborhood of residence, SNAP participation, farmers’ market shopping habits, and participation in NYC Health Bucks. Each focus group will last approximately two hours, for an expected total burden of approximately ninety-six hours (see Appendix H-1).
Table A-12.A. Estimated Annualized Burden Hours |
||||||
Type of Respondents |
Form Name |
No. of Respondents |
No. of Responses per Respondent |
Average Burden (in hours) |
Total Burden (in hours) |
|
Local Community Organizations |
Local Community Organization Survey |
200 |
1 |
10/60 |
33 |
|
Farmers’ Market Managers |
Farmers’ Market Managers Survey |
90 |
1 |
8/60 |
12 |
|
Farmers’ Market Vendors |
Farmers’ Market Vendor Survey |
450 |
1 |
7/60 |
53 |
|
Farmers’ Market Vendor Focus Group |
24 |
1 |
2 |
48 |
||
Farmers' Market Consumers |
Consumer Point-of-Purchase Survey |
2,300 |
1 |
7/60 |
268 |
|
Consumer Focus Group |
48 |
1 |
2 |
96 |
||
NYC Health Bucks Neighborhood Residents |
Neighborhood Resident Survey |
1,000 |
1 |
9/60 |
150 |
|
TOTAL |
660 |
|||||
B. Wage data used to compute the total estimated annual cost of this information collection activity came from the Bureau of Labor Statistics Occupational Employment Statistics, collected in May 2008. To obtain the total estimated annual cost for each data collection component, we multiplied total burden hours for each group of respondents as estimated above by average hourly wage estimates for the closest available occupation code for the New York-White Plains-Wayne, NY-NJ Metropolitan Division. This is the smallest metropolitan statistical area containing the five boroughs of New York City for which wage data were available. In general, since our study is focused on low-income areas in New York City, we expect that actual wages of respondents in our study are lower than assumed in our cost calculations. The total estimated annual cost reported below thus is likely to represent an upper bound cost estimate for this information collection activity.
The average hourly wage for local community organization directors and farmers’ market managers was assumed to be $37.63, based on reported wages for “Social and Community Service Managers” (Occupation Code 11-9151). For farmers’ market vendors, we assumed an average hourly wage of $22.75 based on reported wages for “First-line Supervisors/Managers of Farming, Fishing, and Forestry Workers” (Occupation Code 45-1011); in general, this may represent an overestimate of wages for this group, since many farmers’ market vendors are not the farmers themselves, but rather farm laborers or other vendors contracted by the farmers to sell goods at the markets. Finally, for farmers’ market consumers and residents of NYC Health Bucks neighborhoods, we assumed an average hourly wage of $26.37, based on reported average wages for all occupations in this metropolitan statistical area. Again, this may represent an overestimate of actual wages for this group, since our sampling approach is intended to oversample residents of low-income neighborhoods.
The financial burden on participants will be minimized as consumers and vendors/farmers will be administered the survey while they are already at the market. Farmers’ market managers will be mailed the survey to complete on their own time, and will be provided a pre-stamped envelope in which to return the survey. The focus groups will also aim to minimize the financial burden as they will be held at times during the off-season when vendors/farmers can participate (i.e., not during farmers’ market season), or at times during the day when consumers can participate (i.e., before or after work hours). Based on the estimated annual burden hours above, Table A.12-B below shows estimated annualized cost to participants. The total estimated annual cost to respondents is $17,555.
Table A.12-B. Estimated Annualized Cost to Respondents |
||||||
Type of Respondents |
Form Name |
No. of Respondents |
No. of Responses per Respondent |
Average Burden (in hours) |
Average Hourly Wage |
Total Cost |
Local Community Organizations |
Local Community Organization Survey |
200 |
1 |
10/60 |
$37.63 |
$1,254 |
Farmers’ Market Managers |
Farmers’ Market Managers Survey |
90 |
1 |
8/60 |
$37.63 |
$452 |
Farmers’ Market Vendors |
Farmers’ Market Vendor Survey |
450 |
1 |
7/60 |
$22.75 |
$1,194 |
Farmers’ Market Vendor Focus Group |
24 |
1 |
2 |
$22.75 |
$1,092 |
|
Farmers' Market Consumers |
Consumer Point-of-Purchase Survey |
2,300 |
1 |
7/60 |
$26.37 |
$7,076 |
Consumer Focus Group |
48 |
1 |
2 |
$26.37 |
$2,532 |
|
NYC Health Bucks Neighborhood Residents |
Neighborhood Resident Survey |
1,000 |
1 |
9/60 |
$26.37 |
$3,956 |
TOTAL |
$17,555 |
|||||
A-13. Estimates of Other Total Annual Cost to Respondents or Record Keepers
There are no costs to respondents other than their time, and no record-keeping is required.
A-14. Annualized Cost to the Government
Annualized costs to the government include costs for CDC personnel (Project Officer, Co-Technical Monitors) who are each expected to contribute approximately 15% of their time to managing the project and communicating with the contractor and program coordinator (NYC DOHMH).
The data collection contractor and its subsidiary, Abt SRBI, will receive $589,848 for work relating to designing the evaluation plan, conducting a process evaluation, conducting an outcome evaluation, preparing final reports and a manuscript for publication, and creating an implementation tool kit. The data collection contractor will also oversee a subcontract with a consultant who will lend expertise on secondary data sources and characterizing the food access environment in New York City.
Some of the contractual costs will be incurred outside the requested one-year clearance period, but are related to data collection, management and analysis for this study. For this reason, the annualized cost table presents total costs related to work conducted under the prime contract and the associated subcontract.
The total annualized project costs are estimated at $623,804.
Cost Category |
Total Costs
|
CDC Personnel |
|
15% FTE of GS-13 @93,110 |
$13,967 |
15% FTE of GS-12 @67,422 |
$10,113 |
15% FTE Fellow @ $65,242 |
$9,876 |
Subtotal, CDC |
$33,956 |
Contract to Abt Associates Inc. |
|
Designing evaluation plan Conducting a process evaluation of the Health Bucks Program Conducting an outcome evaluation of the Health Bucks Program Preparing final reports and a manuscript Creating an implementation toolkit |
$404,041 |
Indirect costs |
$128,091 |
Other direct costs |
$10,522 |
Facilities rental and incentives for focus groups |
$9,931 |
Travel costs for study team |
$29,382 |
Subcontract to PhD-level consultant |
$7,881 |
Subtotal, Abt Associates and subcontract |
$589,848 |
|
|
Grand Total |
$623,804 |
A-15. Explanation for Program Changes or Adjustments
This is a new, one-time data collection.
A-16. Plans for Tabulation and Publication and Project Time Schedule
Data collection will take place from July 2010 through December 2010 and must be coordinated with both the 2010 New York City farmers’ market season and OMB’s policy of minimizing Federal-agency sponsored household data collections during the period of the 2010 Census of Population and Housing (March 1 – August 31, 2010). CDC requests OMB approval prior to July 2010 to allow initiation of data collection during the optimal farmers’ market season for study components that are not subject to the policy referenced above. The telephone survey of NYC Health Bucks neighborhood residents will take place after August 2010 in order to comply with the OMB policy.
Data analysis and preliminary reporting will take place from December 2010 through February 2011. A final report will be provided by the prime contractor to the CDC by April 2011. A manuscript will be developed for submission to peer-reviewed journals by August 2011, and a toolkit with practical guidance on how to implement similar programs in other jurisdictions will be completed over the same interval. The table below outlines the expected timeline for data collection and reporting.
Activity |
Time Schedule |
Data Collection
|
July 2010 – December 2010**
**Telephone survey of NYC Health Bucks neighborhood residents will take place after August 2010 in order to comply with OMB policy of minimizing Federal-agency sponsored household data collections in order to encourage resident cooperation with the 2010 Census of Population and Housing. |
Data analysis and preliminary reporting |
December 2010 – February 2011 |
Prepare final report for CDC |
March 2011 – April 2011 |
Develop manuscript for peer-reviewed journal |
April 2011 – August 2011 |
Develop toolkit |
April 2011 – August 2011 |
As detailed above, our proposed mixed-methods evaluation approach involves collection of both qualitative and quantitative data. In the remainder of this section, we describe key analytic approaches to be used in synthesizing and analyzing these data.
Qualitative Data Analysis
For all focus groups, audio transcripts and written notes will be made which will later be analyzed for content. Data will be entered into a qualitative software package such as NVIVO, with specific choice of software guided by needs identified by the CDC and the core evaluation team. Content will first be automatically coded by topic area following the interview or moderator guide. Within each topic, common themes that emerge across multiple focus groups and interviews will be identified. Key words, phrases, ideas and concepts will be noted for frequency of occurrence. Where relevant, the intensity and context of the mentions will also be noted, as well as the level of agreement or disagreement across sources. Salient points from the analysis will be extracted for use in reports along with quotations that are reflective of the discussion.
Quantitative Analysis
We will perform descriptive quantitative analyses on all survey data. We will begin by producing a series of descriptive statistics and cross-tabulations to summarize data from each of the five surveys. Where analytically appropriate, we may conduct additional multivariate regression analyses to identify associations of interest adjusted for the influence of other measurable factors. For example, by combining survey data from the local community organization survey with information from FMFNY administrative data about redemption rates for Health Bucks from each source, we will be able to analyze the independent association of various organizational characteristics (e.g. size, mission, timing of distribution, nutrition/wellness activities) with realized redemption rates. All quantitative analyses of data collected from farmers’ market managers, consumers, and vendors will incorporate appropriately-constructed sampling weights to account for the stratified random sampling strategy employed in non-Health Bucks markets, described in greater detail in Section B-1.
Our quantitative findings will additionally be supplemented with a difference-in-differences analysis of data on perceived access to and consumption of fresh fruits and vegetables from the Community Health Survey (CHS). We will compile CHS data collected both prior to and following the roll-out and implementation of the NYC Health Bucks program, both from ZIP codes in which the program operates, and from New York City ZIP codes outside the three target neighborhoods, supplemented with information about farmers’ market proximity and program implementation from administrative sources. However, because this component of the study does not involve primary data collection, we do not include additional details here.
A-17. Reason(s) Display of OMB Expiration Date is Inappropriate
The OMB expiration date will be displayed.
A-18. Exceptions to Certification for Paperwork Reduction Act Submissions
None.
Anderson, J.V., Bybee, D.I., Brown, R.M., McLean, D.F., Garcia, E.M., Breer, M.L., and Schillo, B.A. (2001). 5 A Day fruit and vegetable intervention improves consumption in a low income population. Journal of the American Dietetic Association, 101(2), 195-202.
Carlton-Tohill, B. (2007). Fruits and vegetables consumption and body weight management. Acta Horticulturae, 744, 39-46.
Centers for Disease Control and Prevention. Obesity: Halting the Epidemic by Making Health Easier (At A Glance 2009). Accessed online at: http://www.cdc.gov/nccdphp/publications/AAG/pdf/obesity_aaa.pdf.
Centers for Disease Control and Prevention. (1999). Framework for program evaluation in public health. MMWR 48 (No. RR-11).
Dauchet, L., Amouyel, P., Hercberg, S., and Dallongeville, J. (2006). Fruit and vegetable consumption and risk of coronary heart disease: A meta-analysis of cohort studies. Journal of Nutrition 136(10), 2588-2593.
Gordon-Larsen, P., Adair, L.S., and Popkin, B.M. (2003). The relationship of ethnicity, socioeconomic factors, and overweight in US adolescents. Obesity Research, 11(1), 121-9.
Guenther, P.M., Dodd, K.W., Reedy, J., and Krebs-Smith, S.M. (2006). Most Americans eat much less than recommended amounts of fruits and vegetables. Journal of the American Dietetic Association, 106(9), 1371-1379.
He, K., Hu, F.B., Colditz, G.A., Manson, J.E., Willett, W.C., and Liu S (2004). Changes in intake of fruits and vegetables in relation to risk of obesity and weight gain among middle-aged women. International Journal of Obesity, 28, 1569-1574.
Herman, D.R., Harrison, G.G., and Jenks, E. (2006). Choices made by low-income women provided with an economic supplement for fresh fruit and vegetable purchase. Journal of the American Dietetic Association, 106(5), 740-744.
Jebb, S.A. (2005). Dietary strategies for the prevention of obesity. Proceedings of the Nutrition Society, 64(2), 217-227.
Krebs-Smith, S.M., Cook, A., Subar, A.F., Cleveland, L., Friday, J., and Kahle, L.L. (1996). Fruit and vegetable intakes of children and adolescents in the United States. Archives of Pediatric and Adolescent Medicine, 150(1), 81–86.
Mirza, N.M., Kadow, K., Palmer, M., et al (2004). Prevalence of overweight among inner city Hispanic-American children and adolescents. Obesity Research, 12, 1298-1310.
Mokdad, A.H., Marks, J.S., Stroup, D.F., et al (2004). Actual causes of death in the United States, 2000. JAMA, 291, 1238–1245.
Ogden, C.L., Carroll, M.D., and Flegal, K.M. (2008). High body mass index for age among US children and adolescents, 2003-2006. Journal of the American Medical Association, 299(20), 2401-2405.
Ogden, C.L., Carroll, M.D., Curtin, L.R., et al (2006). Prevalence of overweight and obesity in the United States, 1999-2004. JAMA, 295, 1549-1555.
Patton, M.Q. (2002). Qualitative Research and Evaluation Methods. Thousand Oaks, CA: Sage Publications.
Riboli, E., and Norat, T. (2003). Epidemiologic evidence of the protective effect of fruit and vegetables on cancer risk. American Journal of Clinical Nutrition, 78(3 Suppl), 559S-569S.
Rolls, B.J., Ello-Martin, J.A., and Carlton-Tohill, B. (2004). What can intervention studies tell us about the relationship between fruit and vegetable consumption and weight management? Nutrition Reviews, 62, 1-17.
Steinmetz, K.A., and Potter, J.D. (1996). Vegetables, fruit, and cancer prevention: a review. Journal of the American Dietetic Association, 96, 1027-1039.
U.S. Department of Health and Human Services. Healthy People 2010. 2nd ed. With Understanding and Improving Health and Objectives for Improving Health. 2 vols. Washington, DC: U.S. Government Printing Office, 2000.
U.S. Government Accountability Office. (2008). Food Stamp Program: Options For Delivering Financial Incentives to Participants for Purchasing Targeted Foods (GAO-08-415). Washington, DC: author.
Wang, Y (2001). Cross-national comparison of childhood obesity: the epidemic and the relationship between obesity and socioeconomic status. International Journal of Epidemiology, 30, 1129-1136.
Wang,Y., and Zhang, Q. (2006). Are American children and adolescents of low socioeconomic status at increased risk of obesity? Changes in the association between overweight and family income between 1971 and 2002. American Journal of Clinical Nutrition, 84(4), 707-716.
| File Type | application/msword |
| File Title | Abt Single-Sided Body Template |
| Author | LintonK |
| Last Modified By | sxw2 |
| File Modified | 2010-03-22 |
| File Created | 2009-12-19 |