ENSP OMB Part A REVISED FINAL OUTCOME fall 2016

REVISION REQUEST FOR A DATA COLLECTION FOR ETH OUTCOME EVALUATION OF THE TITLE III-C NUTRITION SERVICES PROGRAM ORIGINALLY APPROVED UNDER OMB CONTROL NUMBER 0985-0037:

SUPPORTING STATEMENT REQUEST FOR REVISION CLEARANCE: SECTION A

SUBMITTED BY:

Office of Performance and Evaluation

Center for Policy and Evaluation

Administration for Community Living

US Department of Health and Human Services

Mary E Switzer Building

330 C Street, SW, Room 1243

Washington DC 20201

CONTRACT # HPPS23337051T

Part A: Summary

This OMB revision is being submitted to allow for the segment of an information collection approved under OMB number: 0985-0037. The entire data collection, of which the remaining segment is a part, took longer than three years to complete. Specifically, the 12-month client follow up, which we are asking this renewal to cover, is expected to require approximately 6 months past the expiration date of 12/13/2016.

The attached PRA package includes the entire information collection for context, but the sections specifically relevant to this renewal are highlighted in yellow and are summarized below:

This segment of the data collection specifically relates to the third objective of the overall information collection which is to “to assess program effectiveness, as measured by the program’s effects on a variety of important outcomes, including nutrient adequacy, socialization opportunities, health outcomes, and, ultimately, helping elderly people avoid institutionalization (outcome evaluation). “

An announcement was published in the Federal Register on July 19, 2016 (Federal Register https://www.federalregister.gov/documents/2016/07/19/2016-16976/agency-information-collection-activities-submission-for-omb-review-comment-request-oaa-title-iii-c). One comment was received but was not relevant to the data collection and no action was taken.

Under the original information collection there were two items that may have been considered sensitive by the respondent - - social security number and questions about health and income. Those items are not asked about under this final segment of the data collection.

The remaining burden hours are:

Survey	Respondent	Estimated No. Respondent	Responses Annually Per Respondent	Total Annual Responses	Estimated Avg. # of Hours Per Response	Estimated Total Hours	Estimated Hourly Ratea	Total Cost
Home-Delivered Program Participant Survey (1 year follow-up) (CAPI/CATI)	Program participants	600	1	600	0.08	48	$7.25	$348
Home-Delivered Program Non-participant Survey (1 year follow-up) (CAPI/CATI)	Non-participants	600	1	600	0.08	48	$7.25	$348
Congregate Nutrition Program Participant (1 year follow-up) (CAPI/CATI)	Program participants	600	1	600	0.08	48	$7.25	$348
Congregate Nutrition Program Non-participant (1 year follow-up) (CAPI/CATI)	Non-participants	600	1	600	0.08	48	$7.25	$348
TOTALS	-	2400	1	2400	.08	192	$7.25	$1,392

The final segment of the data collection is expected to require 192 hours.

The timeline for the remaining data collection and reporting is shown below:

Approximate Schedule of Tasks and Deliverables

Objectives/Major Tasks	Status
Client and Comparison Group Data Collection	Completed for baseline and 6 month follow up. 12 month follow up is scheduled for fall 2016 and winter 2017
Data Processing, Merging with Medicare and Medicaid Files	Summer 2016 and fall 2017 and
Perform the Title III-C Elderly Nutrition Program Evaluation Data Analysis	Baseline report expected January 2017 Final report expected Spring 2018
Publish reports on ACL website	Approximately 3 months after receipt

PART A

A.1. Circumstances Making the Collection of Information necessary

A.1.1. Authorizing Legislation

The Older Americans Act (OAA) Title III-C Elderly Nutrition Services Program (statutory authority is contained in Title II section 205(a)(2)(A), and Title III sections 311, 331, and 336 of the Older Americans Act (OAA) (42U.S.C. 3032), as amended by the Older Americans Act Amendments of 2006, P.L. 109-365) is part of comprehensive home-and community-based services. Title III, Part C provides grants to States and Territories under approved State Plans for the establishment and operation of nutrition projects for congregate nutrition services (Section 331) and home-delivered nutrition services (Section 332). In addition, Section 311 authorizes the Nutrition Services Incentive Program (NSIP) which provides cash or commodities to States and Tribes for the provision of meals served in compliance with the requirements of the Older Americans Act. Section 339 establishes the requirements for the provision of nutrition services. The legislative purposes of Part C as found in section 330 are “to reduce hunger and food insecurity; to promote socialization of older individuals; and to promote the health and well-being of older individuals by assisting such individuals to gain access to nutrition and other disease prevention and health promotion services to delay the onset of adverse health conditions resulting from poor nutritional health or sedentary behavior. ”

The authorizing legislation for the data collection is found in Title II of the OAA. The requirements stipulated under section 206(a, c) directs ACL to “…measure and evaluate the impact of all programs authorized by this Act, their effectiveness in achieving stated goals in general, and in relation to their cost, their impact on related programs, their effectiveness in targeting for services under this Act unserved older individuals with greatest economic need (including low-income minority individuals and older individuals residing in rural areas) and unserved older individuals with greatest social need (including low-income minority individuals and older individuals residing in rural areas), and their structure and mechanisms for delivery of services, including, where appropriate, comparisons with appropriate control groups composed of persons who have not participated in such programs. Evaluations shall be conducted by persons not immediately involved in the administration of the program or project evaluated.”

A.1.2. Background

1. Ensuring the Nutritional Needs of the Elderly

The Title III-C Elderly Nutrition Services Program (ENSP) represents a key component of America’s strategy for ensuring that the needs of elderly people are adequately met. Every day, millions of Americans, most of them over 65 years of age, receive a nutritious meal at a Senior Center or other congregate meal site. Many others consume a home-delivered meal provided under a different component of the program.

The value of these services to participants goes far beyond the meals themselves. Particularly for participants in congregate meals, Title III-C meals provide an opportunity to socialize and with peers. Further, many other services—from tax preparation to recreational activities to medical screening—are often provided at Senior Centers and other Title III-C sites, and the congregate meals provide a context for helping seniors connect to these services as well. Even for home-delivered meals, which by definition are less focused on social interaction, the daily visit by the meal deliverer, often a volunteer, can represent an elderly recipient’s only human contact of the day.

2. Need to Evaluate the Title III-C Program

An important aspect of the program, critical to understanding how it functions, is the way in which it has developed mechanisms for mobilizing multiple levels of constituencies in the work of serving the elderly. While overall federal coordination is provided by the AoA, the State Units on Aging (SUAs) and the Area Agencies on Aging (AAAs) both support key aspects of program operations. In turn, the direct nutritional services are provided by Local Service Providers (LSPs). Many other governmental and nonprofit groups, as well as some groups organized on a for-profit basis, are also involved in serving the elderly under the program. Often the Title III-C program, with the attraction of its ability to provide inexpensive meals, is a catalyst for this broader involvement.

While the diversity of the organizations involved is a key strength of the Title III-C program, it also creates particular challenges for evaluating the program. Indeed, this diversity makes it particularly complicated (and also particularly important) to examine whether the system operates efficiently overall, and whether it succeeds in delivering services that are of benefit to the elderly, as evidenced by such important outcomes as nutrition, socialization, health, and, ultimately, avoidance of institutionalization. It is also important to examine the targeting of the program, to assess whether its services are reaching the elderly that need them most and to assess whether there may be underserved populations that are not being served by the overall program. So, in addition to the legislative mandate under the OAA, it is important for program integrity and function to evaluation this program.

3. Evaluation Objectives

The overall evaluation of the Title III-C Program has three broad objectives: (1) to provide information to support program planning, including an analysis of program processes (process evaluation), (2) to develop information about program efficiency and cost issues (cost study), and (3) to assess program effectiveness, as measured by the program’s effects on a variety of important outcomes, including nutrient adequacy, socialization opportunities, health outcomes, and, ultimately, helping elderly people avoid institutionalization (outcome evaluation). See Table A.2.1 for more detail about the evaluation questions associated with the Outcome Evaluation. Please note that data collection related to the first and second objectives will be conducted under the first phase of the evaluation and, the relevant data collection tools were be submitted for OMB PRA clearance separately. This request for clearance refers only to evaluation of third objective-the Outcome Evaluation. Limited information about the Process Evaluation and Cost Study are included only to provide context for this information collection and the larger goals of AoA with regards to a comprehensive evaluation of the ENSP.

A.2. Purpose and Use of the Information Collection

The data collected during the evaluation is essential to ACL for meeting the needs of a rigorous evaluation of the impact of the Title III-C Program. There is currently no other national effort that addresses the research objectives of the proposed study. The resulting information will be critical to federal policymakers and will assist all levels of the aging network as ACL attempts to maximize efficiency and service.

Data will be used by ACL staff to improve program operations, provide improved technical assistance and guidance to grantees and service providers, and to support mandated agency reporting to congress and through annual reports. The outcome evaluation data will be analyzed to determine the extent to which ENSP clients as compared with non-clients have:

1. Reduced hunger and food insecurity;

2. Increased socialization of older individuals; and

3. Increased health and well-being of older individuals especially in terms of:

   1. Improved access to nutrition and other disease prevention and health promotion services

   2. Delayed onset of adverse health conditions resulting from poor nutritional health or sedentary behavior.

4. Increase the capacity of older individuals to remain independent and in their communities.

In addition, the data from all three parts of the research will be combined to answer questions about which types of structures and approached are correlated with the most positive client-level outcomes. This information will allow ACL to provide improved guidance to grantees and service providers to help them improve their operations and, ultimately, improve the health and well-being of older Americans.

Table A.2.1. Summary of Evaluation Design: Evaluation Goals, Illustrative Study Questions, Research Strategy, and Data Collection Plan

The Outcome Evaluation: Assess Program Effectiveness
Goals/Study Questions	Research Strategy	Data to be Collected
Do Title III-C services have effects on key outcomes, such as nutrition, socialization, health outcomes, institutionalization?	Compare data on outcome variables between participants and a matched comparison sample of nonparticipants. The main analytic tool will be multivariate methods. Analysis will take account of the fact that some outcomes could be influenced by even brief participation (nutrition and socialization,) while other can only reasonable by expected to occur after substantial lengths of participation (health outcomes and avoiding institutionalization) Thus, length in program is an important variable.	Medicare data will be used to identify a matched comparison group, and to supply some outcome variables A participant survey will supply certain participant characteristics, outcomes, including nutrition data from a 24-hour dietary recall. Similar data will be obtained for comparison group interviews.

A.3. Use of Improved Information Technology and Burden Reduction

The study strives to comply with the E-Government Act of 2002 (Public Law 107-347, 44 U.S.C. Ch 36) by using computer-assisted interviewing. Through the strategic use of technology, we will use multi-mode data collection systems that ensure high quality data collection while minimizing burden on respondents. For example, programmed skip patterns, consistency and data range checks within a computer-assisted instrument reduce data entry error that often necessitate callbacks to respondents to clarify the responses recorded by an interviewer using a paper questionnaire.

Consistent, multi-mode menu surveys. ACL will use the AMPM software, or other equivalent software, to collect data on the nutritional value of Title III-C menus. While the this instrument is used extensively as a computer-assisted personal interview (CAPI) instrument, it can also be used for computer-assisted telephone interview (CATI) data collection. For the menu survey component of the study, we will conduct three brief interviews with the individual who is most knowledgeable about the meals offered to participants during a five-day target week.

CAPI/CATI systems for client outcomes survey For the client outcomes survey, a combination of CATI and CAPI modes will be used to provide the most cost-effective interviewing method. In particular, telephone collection will significantly reduce the costs of the nonparticipant screening effort, yet a personal visit is preferable for the conducting the surveys and dietary intake interviews with program participants and nonparticipants.

• Strategic use of CATI tools. As part of the process of identifying a matched sample of nonparticipants, a CATI screener will be used to screen out Medicare beneficiaries who are currently participating in Title III-C congregate or home-delivered nutrition programs.

• Efficient use of CAPI tools. A CAPI instrument will be used for both the congregate and home-delivered nutrition participants and nonparticipants surveys. The nature of the instrument and survey content is such that a personal collection is more appropriate and likely to result in better data quality than would a telephone interview.

A.4. Efforts to Identify Duplication and Use of Similar Information

The ACL sought to avoid duplication of effort in both design and data by trying to identify existing instruments and data sets relevant to the study. It was concluded that no existing data sources can provide data needed to answer the study’s research questions. But, where possible, data for the process evaluation (such as selected fiscal data) will be pulled from existing data reports submitted by states and confirmed with sites rather than asking them to gather the data anew. ACL will also try to reduce duplication across its evaluation projects by making a concerted effort to ensure that, where possible, questions are shared across projects. This will allow for data aggregation across projects which will permit additional analyses, including comparative analyses, which would otherwise not be possible.

A.5. Impact on Small Businesses or Other Small Entities

Information being requested or required has been held to the minimum required for the intended use. We will request information from LSPs. LSP surveys will be conducted via a web-based survey that will be programmed with customized text and routing to accommodate the 40 percent of AAAs who are also LSPs thereby reducing their burden by eliminating redundant questions. Technical assistance will be provided by telephone.

A.6. Consequences of Collecting the Information Less Frequently.

Each data collection tool will be used to collect data one time per respondent. This is one time total for SUAs, four data collections from AAAs/LSPS, and five collections for each individual. If these data are not collected ACL will not be responsive to the requirement in Title II, Section 206 of the Older American Act of 1965 that all authorized programs should be evaluated. The Act specifically authorizes the ENSP and that the Assistant Secretary of Aging will measure the impact of all programs authorized in the Act (The response A1 above contains text from this section of the Act). Further, if the information is not collected, ACL will lack important information needed for program improvement.

A.7. Special Circumstances Relating to Guidelines of 5 CFR 1320.5

There are no special circumstances. The collection of information is conducted in a manner consistent with guidelines in 5 CFR 1320.5.

A.8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

A.8.a. Federal Register Notice

An announcement was published in the Federal Register on July 19, 2016 (Federal Register https://www.federalregister.gov/documents/2016/07/19/2016-16976/agency-information-collection-activities-submission-for-omb-review-comment-request-oaa-title-iii-c). One comment was received, but was not relevant to the data collection and no action was taken.

A.8.b. Outside Consultations

Planning for data collection in this study has involved extensive consultation between ACL and the Mathematica Policy Research (the research team contracted to conduct the ENSP evaluation). ACL contacted Alana Moshfegh of the Agricultural Research Service to secure rights to use the AMPM software for the menu survey and reviewed the technical rigor of this effort. In addition, this work has been informed by contributions from a Technical Advisory group that includes service providers, nutrition services contractors, Area Agency on Aging staff, State Unit on Aging staff, and evaluation experts.

A.9. Explanations of Any Payment or Gift to Respondents

Even with the least burdensome instruments possible, we believe it is important to identify strategies for maximizing response rates. The role of incentives in increasing survey response rates has been widely documented (Holbrook et al. 2008, Singer et al. 1999, Singer & Ye 2013). Some research has found that the level of incentives matters; in other words, higher incentives engender higher response rates (Rodgers 2011; Datta et al. 2001; Colicchia et al. 2012). There is also evidence that higher incentives help in reducing non-response bias without compromising the quality of responses (Singer & Kulka 2001; Castiglioni & Pforr 2007). The incentive to respondents will be $25 per response in the form of widely accepted gift cards.

A.10. Assurance of Confidentiality

ACL is committed to protecting the security of all study data and, in particular, the confidentiality of Personally Identifiable Information (PII) that institutions and respondents provide. The following data handling and reporting procedures will be employed to maintain the privacy of survey participants and composite electronic files. See also the Privacy Impact Assessment (PIA) that is being submitted as part of this package.

• Confidentiality Agreement. All project staff will be required to sign a confidentiality statement. In this agreement project staff pledge to maintain the confidentiality of all information collected from the respondents and will not disclose it to anyone other than authorized representatives of the evaluation, except where otherwise required by law. Issues of confidentiality are discussed during interviewer training.

• Data on Central Office Computers. Standard backup procedures will be implemented for the central office computer system to protect project data from user error system failure. Backups and inactive files will be maintained on tape or compact disks. The system servers will be maintained inside a secure locked area accessible only to authorized systems personnel. Files will be accessible only by authorized personnel who have been provided project logons and passwords. Access to any of the study files (active, backup, or inactive) on any network multi-user system will be under the central control of the database manager who will ensure that the appropriate network partitions used in the study are appropriately protected (by password access, decryption, or protected or hidden directory partitioning) from access by unauthorized users.

• Data on Laptops and Data Transmission from and to Laptops. Full data security will be employed on data collection laptops in the field, including disk-level encryption and transmission protection.

• Documents Received in Central Office. Once in the central office, documents containing respondent information are kept in locked filing cabinets. At the close of the study, such documents are shredded.

• Personally Identifiable Information. Any respondent-identifying information will be contained only in a master list to be created and protected in secure storage, to which only a limited number of project staff pledged to maintain confidentiality will have access.

The individuals participating in this study will be notified that the information they provide will not be released in a form that identifies them, except as otherwise required by law. No identifying information will be attached to any reports or data supplied to the ACL or any other researchers.

Institutional Review Board

ACL will require the contractor to prepare and submit a request for approval to a recognized Institutional Review Board (IRB) for Research Involving Human Subjects. All study materials and instruments for the program participants and nonparticipants will be submitted to and approved by the IRB.

A.11. Justification for Sensitive Questions

Under the original information collection (OMB Control #0985-0037) there were two items that may be considered sensitive by the respondent¹ - - social security number and questions about health and income. Those items are not asked about under this final segment of the data collection.

A.12. Estimates of Annualized Burden Hours and Costs

Exhibit A.12.1 shows sample sizes and estimates on burden, frequency of response, annual responses per respondent, and annualized cost of respondent burden for each part of the data collection and for total burden.

Exhibit A.12.1. Estimated Respondent Burden

Survey	Respondent	Estimated No. Respondent	Responses Annually Per Respondent	Total Annual Responses	Estimated Avg. # of Hours Per Response	Estimated Total Hours	Estimated Hourly Ratea	Total Cost
Menu Survey
LSP Menu Survey (AMPM) (day 1 menu) (CATI)	LSPs (site managers and nutrition directors)	200	1	200	0.53	106	$35.82	$3,797
LSP Menu Survey (AMPM) (days 2,3 menus) (CATI)	LSPs (site managers and nutrition directors)	200	1	200	0.97	194	$35.82	$6,949
Client Outcomes Survey
Home-Delivered Program Participant Survey (CAPI/CATI)	Program participants	600	1	600	0.78	468	$7.25	$3,393
Home-Delivered Program Participant Survey (6 month follow-up) (CAPI/CATI)	Program participants	600	1	600	0.08	48	$7.25	$348
Home-Delivered Program Participant Survey (1 year follow-up) (CAPI/CATI)	Program participants	600	1	600	0.08	48	$7.25	$348
Home-Delivered Program Non-participant Survey (CAPI/CATI)	Non-participants	600	1	600	0.78	468	$7.25	$3,393
Home-Delivered Program Non-participant Survey (6 month follow-up) (CAPI/CATI)	Non-participants	600	1	600	0.08	48	$7.25	$348
Home-Delivered Program Non-participant Survey (1 year follow-up) (CAPI/CATI)	Non-participants	600	1	600	0.08	48	$7.25	$348
Congregate Nutrition Program Participant (CATI/CAPI)	Program participants	600	1	600	0.78	468	$7.25	$3,393
Congregate Nutrition Program Participant (6 month follow-up) (CAPI/CATI)	Program participants	600	1	600	0.08	48	$7.25	$348
Congregate Nutrition Program Participant (1 year follow-up) (CAPI/CATI)	Program participants	600	1	600	0.08	48	$7.25	$348
Congregate Nutrition Program Non-participant (CATI/CAPI)	Non-participants	600	1	600	0.78	468	$7.25	$3,393
Congregate Nutrition Program Non-participant (6 month follow-up) (CAPI/CATI)	Non-participants	600	1	600	0.08	48	$7.25	$348
Congregate Nutrition Program Non-participant (1 year follow-up) (CAPI/CATI)	Non-participants	600	1	600	0.08	48	$7.25	$348
24-Hour Dietary Recall
Home-Delivered Program Participant Dietary Recall (CAPI/CATI)	Program participants	600	1	600	0.5	300	$7.25	$2,175
Home-Delivered Program Participant Dietary Recall (second recall)(CAPI/CATI)	Program participants	150	1	150	0.5	75	$7.25	$544
Home-Delivered Program Non-participant Dietary Recall (CAPI/CATI)	Non-participants	600	1	600	0.5	300	$7.25	$2,175
Home-Delivered Program Non-participant Dietary Recall (second recall) (CAPI/CATI)	Non-participants	150	1	150	0.5	75	$7.25	$544
Congregate Nutrition Program Participant Dietary Recall (CAPI/CATI)	Program participants	600	1	600	0.5	300	$7.25	$2,175
Congregate Nutrition Program Participant Dietary Recall (second recall)(CAPI/CATI)	Program participants	150	1	150	0.5	75	$7.25	$544
Congregate Nutrition Program Non-participant Dietary Recall (CAPI/CATI)	Non-participants	600	1	600	0.5	300	$7.25	$2,175
Congregate Nutrition Program Non-participant Dietary Recall (second recall) (CAPI/CATI)	Non-participants	150	1	150	0.5	75	$7.25	$544
TOTALS		2,600 [1]		10,600
Total—hours						4,056
Total—minutes						243,360
Average response--hours						.52 [2]
Average response--minutes						31.2 [2]
TOTALS for HIGHLIGHTED ROWS		2400	1	2400	.08	192	$7.25	$1,392

^a Sources: Bureau of Labor Statistics, National Compensation Survey, 2010, May 2011, Bulletin 2753. (http://www.bls.gov/ncs/ncswage2010.htm#Wage_Tables): SUA, AAA, LSP staff: Average hourly wage of state and local government social and community service managers; participants and non-participants: national minimum wage

[1] Because some respondents will be surveyed multiple times over the course of the study this number represents the total number of unique respondents

[2] Average number of hours and minutes are the totals divided by the three-year OMB clearance period.

A.13 Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

There are no capital, start-up, or annualized maintenance costs associated with this data collection for respondents.

A.14 Annualized Cost to the Federal Government

The cost to the Federal government for the all tasks associated with the Title III-C Program evaluation is $1,555,230. This expense includes the costs associated with the contractor conducting the project and the partial salary of the assigned ACL project officer.

A.15 Explanation for Program Changes or Adjustments

The final segment of the data collection is expected to require 192 hours (2400 people interviewed for .08 hours)

A.16 Plans for Tabulation and Publication and Project Time Schedule

The research planned for the study has three broad objectives: (1) to provide information to support program planning, including an analysis of program processes and assessing the nutritional quality of meals offered to program participants; (2) to generate information about program efficiency and costs; and (3) to assess program effectiveness, as measured by the program’s effects on a variety of important elderly participant outcomes, including nutrient adequacy, socialization opportunities, health outcomes, and, ultimately, helping elderly people avoid institutionalization.

For each objective, we will analyze the data collected. We will prepare a final report and conduct a briefing for ACL staff. The report will present findings from both descriptive analyses of agency characteristics, program meals, and costs, as well as descriptive and multivariate analyses of program participant outcomes. This section presents the analysis plans for addressing the study objectives and provides the corresponding project schedule.

Information about program quality and processes will also be obtained from the “customer” side of the program. We will select a random sample of elderly clients who will be surveyed to learn about their key demographic, health, nutrition and lifestyle characteristics, the extent of their use of the program, and their levels of satisfaction with program services. The data obtained in this survey of participants will contribute to the analysis of the program by providing valuable information on program targeting (the program should target elderly people with the greatest economic or social need)² and an assessment of the efficiency of the program from the clients’ point of view.

The study will describe ENSP participant satisfaction with congregate meals and home-delivered meals and related supportive services. We will ascertain congregate nutrition and home-delivered nutrition participants’ overall level of satisfaction with the nutrition program; what they like most and least about the program. We will also assess their perceptions about how the food tastes, smells, and looks, and the extent to which they are satisfied with food variety and meal sizes. Table A.16.3 illustrates how we will present findings on participants’ valuation of meals and supportive services received from the nutrition program. This includes the degree of difficulty in accessing the site and satisfaction and time spent in recreational and social activities (for congregate nutrition participants) and the helpfulness of referrals and other services, as well as the nutrition program.

Table A.16.3. Participants Valuation of Meals and Supportive Services Received From the Nutrition Program

Characteristic	Congregate Nutrition Participants	Home-Delivered Nutrition Participants
Transportation
Ease of getting to the site		NA
Very easy
Somewhat easy
Not too easy
Not easy at all
Recreational and Social Activities
Satisfaction with opportunities to spend time with others		NA
Very satisfied
Somewhat satisfied
Not too satisfied
Not at all satisfied
Time spent participating in other activities or receiving other services at the meal site		NA
A lot of time
Some time
Just a little time
No time
Referrals and Other Services
Received information and/or referral services from nutrition program
Information and/or referral services from the program were:
Very helpful
Somewhat helpful
Not too helpful
Not at all helpful
Helpfulness of Program
Overall helpfulness of the nutrition program
Helped a lot
Helped somewhat
Helped a little
Didn’t help
Made things worse
The nutrition program has helped clients:
Eat healthier foods
Improve health
Follow a special diet
Achieve or maintain a healthy weight
Live independently and stay in own home

Source: Client survey.

1. Nutritional Quality of Program Meals

The Older Americans Act (OAA) of 2006 requires that programs provide meals to participants that (1) provide a minimum of one third of the Dietary Reference Intakes (DRIs) if one meal per day is provided (OAA Sections 339 and 614),³ and (2) comply with the most recent Dietary Guidelines for Americans. The DRIs provide the most up-to-date information on nutrient requirements and include reference values for men and women ages 51 to 70 years and over 70 years. The Dietary Guidelines provide recommendations to help individuals choose foods that comprise a healthy eating pattern—specifically, one that focuses on the consumption of nutrient-dense foods while staying within calories needs. Our assessment of the quality of program meals provided by the ENSP will focus on the extent to which meals offered or delivered to participants meet these Federal nutrition standards for the program. We will also examine the average nutrient and food group content as well as the types and variety of foods included in meals offered or delivered to participants.

These analyses of the nutritional quality of program meals will be based on data from the menu survey collected from LSP staff (described in Section II.A). The menu data will include detailed information on the foods and beverages offered to participants in congregate and home-delivered meals over a three-day period. The data will first be coded using USDA databases to obtain estimates of the nutrient and food group content of the individual foods offered (for example, grams of protein and cups of vegetables). We will then develop estimates of the nutrient and food group content of the average meal offered to participants. All of these analyses of the program meals data will be weighted descriptive tabulations and will include separate estimates for congregate meals, home-delivered meals, and all meals combined.

As mentioned, the first set of analyses will assess how well the meals conform to Federal nutrition standards for the program. We will determine the proportion of LSPs that offer meals that meet one-third of the DRI-based standards for men and women ages 51-70 and over 70 years. This analysis will focus on the nutrients that were identified by an Issue Panel convened by the National Resource Center on Nutrition, Physical Activity, and Aging for use in planning and evaluating meals (Silver et al. 2002). To examine compliance with the Dietary Guidelines, we will use the quantitative recommendations specified for saturated fat, cholesterol, and sodium, as well as one-third of the daily recommended amounts of food groups (for calorie levels appropriate for older adults) specified in the USDA Food Patterns. Table A.16.4 illustrates how the results will be presented for both meal types combined.

We will also tabulate the average nutrient and food group content of the meals based on the portions sizes of the foods and beverages offered. As an example, Table A.16.5 presents the average calorie and nutrient content of program meals. To address the last research question, we plan to assess the types and variety of foods offered by tabulating the frequency of various types of foods offered in program meals—for example, the frequency of fresh fruits and vegetables.

2. Health Status and Medical Care

To examine the health status of participants, tabulations of general measures such as self-reported health status and body mass index, cigarette and alcohol usage, and mobility limitations will be presented (Table A.16.9). Other health characteristics will also be reported, such as the type and frequency of receipt of medical care over the prior year, including the number of emergency visits and hospital stays; time spent in a nursing home, convalescent home, or rehabilitation center; and the types of diagnoses that participants have received from doctors.

Table A.16.4. Proportion of LSPs Offering or Delivering Meals that Meet Standards: All Meals

		Proportion meeting standard on
	Standard/ Recommendation	One Day	Two Days	Three Days	Average
Calories (kcal)	685a
Macronutrients (% of total calories)b
Protein	10-35
Carbohydrate	45-65
Total Fat	20-35
Saturated Fat	< 10c
Vitamins
Vitamin A (ug)	300
Vitamin C (mg)t	30
Vitamin D (ug)	5
Vitamin E (mg)	5
Thiamin (mg)	0.4
Riboflavin (mg)	0.43
Vitamin B6 (mg)	0.57
Folate (ug)	133
Vitamin B12 (ug)	0.79
Minerals
Calcium (mg)	400
Copper (ug)	300
Iron (mg)	2.7
Magnesium (mg)	140
Potassium (mg)	1167
Sodium (mg)	< 500c
Zinc (mg)	3.7
Other Dietary Components
Cholesterol (mg)	< 100c
Dietary Fiber (g/1,000 calories)	14
Number of LSPs

Notes: The standards and recommendations included in the table are based on the Dietary Reference Intakes (DRIs) and Dietary Guidelines for Americans. The standards for all vitamins and minerals except for vitamin D and calcium are Recommended Dietary Allowances (RDAs). The standards for vitamin D and calcium are Adequate Intakes (AIs). The DRIs shown in this table are based on one-third of the highest values for ages 51 and older, males and females. Tables may also be prepared showing percentages meeting DRI standards separately for males and females, or meeting average DRIs for both groups.

^a Based on recommendations for a 75 year old male, height of 5’7’’, and a “low active” physical activity level.

^b The DRIs define Acceptable Macronutrient Distribution Ranges (AMDRs) for intakes of carbohydrate, protein, and fat as a percentage of total calories. The AMDRs reflect the ranges of intake that are associated with reduced risk of chronic disease while providing adequate amounts of essential nutrients.

^c Based on the Dietary Guidelines recommendation.

Table A.16.5. Average Calorie and Nutrient Content of Meals Offered or Delivered to Participants

	Congregate Meals	Home-Delivered Meals	All Meals
Calories (kcal)
Macronutrients (% of total calories)
Protein
Carbohydrate
Total Fat
Saturated Fat
Vitamins
Vitamin A (ug)
Vitamin C (mg)
Vitamin D (ug)
Vitamin E (mg)
Thiamin (mg)
Riboflavin (mg)
Vitamin B6 (mg)
Folate (ug)
Vitamin B12 (ug)
Minerals
Calcium (mg)
Copper (ug)
Iron (mg)
Magnesium (mg)
Potassium (mg)
Sodium (mg)
Zinc (mg)
Other Dietary Components
Cholesterol (mg)
Dietary Fiber (g)
Number of Meals
Number of LSPs

Note: Tables presenting the average food group content of meals offered or delivered to participants will also be prepared.

3. Program Outcomes and Effectiveness

Two objectives of the evaluation of program outcomes and effectiveness are to identify the characteristics of the nutrition program participants as accurately as possible and to estimate the impact of participating in these programs on individuals’ nutrition, food security, socialization activities, and health. Different analytic methods will be used for each objective. Weighted descriptive tabulations of congregate and home-delivered nutrition program participants will be used to describe the demographic, economic, health, social, and nutrition characteristics, as well as the service use, of ENSP participants. These tabulations will allow us to determine whether the congregate and home-delivered nutrition programs are serving clients as intended and the extent to which the programs successfully targets priority subgroups of elderly individuals. In contrast, multivariate analysis will be used to estimate program impacts on a set of outcome measures. This analysis will compare program participants and eligible nonparticipants on selected outcomes, controlling for characteristics that could be related to both program participation and the outcomes studied. Below, both sets of analytic methods are described and example table shells are provided.

4. Participant Characteristics

We will conduct tabular analysis that will describe the characteristics of nutrition program participants and identify key differences and similarities between congregate and home-delivered nutrition participants. This will include demographic and economic characteristics; health status; mobility; eating behavior, diet, and food preparation; food security; program participation experiences; and dietary quality. In the following paragraphs, selected tables serve as examples of the types of descriptive analyses that will be conducted.

5. Demographic and Economic Characteristics

We will present demographic characteristics such as participants’ age, gender, highest grade level of schooling completed, race and ethnicity, marital status, and household size (Table A.16.8). The study will also examine participants’ sources of income, the distribution of income, and the extent to which participants are forced to choose between buying food and other expenses such as medications, utility bills, and housing payments (these characteristics are not shown in the table).

Table A.16.8. Selected Demographic Characteristics of Nutrition Program Participants

Characteristic	Congregate Nutrition Participants	Home-Delivered Nutrition Participants
Age
Less than 60
60 – 74
75 – 84
85 and older
Average Age (years)
Female
Highest grade level completed	V3	V3
5th grade or less
6th – 12th grade (no diploma)
High School Graduate, GED or equivalent
Some College (no degree)
Associate Degree, Occupational or Technical Degree
Bachelors Degree
Masters Degree or higher
Race and ethnicity	V5	V5
White non Hispanic
Black non Hispanic
Asian non Hispanic
American Indian non Hispanic
Other non Hispanic
Hispanic	V4	V4
Marital status	V6	V6
Married or living with partner
Widowed
Divorced
Separated
Never married
Number of people living in household	V8	V8
Live alone
1
2
3 or more

Source: Client Survey.

6. Health Status and Medical Care

Health Status and Medical Care. To examine the health status of participants, tabulations of general measures such as self-reported health status and body mass index, cigarette and alcohol usage, and mobility limitations will be presented (Table A.16.9). Other health characteristics will also be reported, such as the type and frequency of receipt of medical care over the prior year, including the number of emergency visits and hospital stays; time spent in a nursing home, convalescent home, or rehabilitation center; and the types of diagnoses that participants have received from doctors.

Table A.16.9. General Health Status of Nutrition Program Participants

Characteristic	Congregate Nutrition Participants	Home-Delivered Nutrition Participants
General health
Excellent
Very good
Good
Fair
Poor
BMI
Below 18.5 (Underweight)
18.5 – 24.9 (Normal)
25.0 – 29.9 (Overweight)
30.0 and above (Obese)
Unintentional gain or loss of 10 pounds in past 6 months
Participated in physical activity in the past month
Number of days per week
Currently smokes cigarettes
Every day
Some days
Alcohol Consumption
Average number of days per week alcohol is consumed
Average number of alcoholic drinks when consumed
Has a clinic/doctors office for routine care or
Wears dentures
Blood pressure checked in the past 12 months
Fallen more than twice in the past 12 months
Mobility
Able to walk
Uses a cane or walker
Difficulty walking or climbing stairs
Bed bound
Average time bed bound
Chair bound or in a wheelchair
Average time chair bound or in a wheelchair

Source: Client Survey

7. Mobility

To examine participants’ mobility, we will tabulate the percentage of congregate and home-delivered nutrition program participants that are chair bound or in a wheelchair, use a cane or walker, or have serious difficulty walking or climbing stairs. We will also examine the types of difficulties that participants may have doing certain activities such as shopping for personal items, using the telephone, preparing meals, taking medications, or taking a bath or shower.

8. Eating Behaviors, Diet, Food Preparation, and Food Security

We will examine participants’ eating behaviors, diet, and food preparation, as well as their food security. For example, tables will present the percentage of participants that prepare their own meals or help someone else cook, as well as the types of special diets prepared by participants, such as diabetic, low sodium, low sugar, low far, low/high fiber, vegetarian, Lactose-free). We will also estimate the percentage of congregate and home-delivered nutrition participants that are food secure, food insecure with low food security, and food insecure with very low food security (Table A.16.10). These percentages will be based on a six-item, 30-day food security module.

Table A.16.10. Food Security Among Nutrition Program Participants

Food Security	Congregate Nutrition Participants	Home-Delivered Nutrition Participants
Food secure
Food insecure
Food insecure with low food security
Food insecure with very low food security

Source: Client survey.

9. Program Participation Experiences

The study will describe program participation experiences of congregate and home-delivered nutrition program participants. This includes how long current participants have been in the program, how participants found out about the program or were referred to the program, their frequency of site attendance/receipt of home-delivered meals, and their experiences and attitudes about voluntary contributions for meals. An example table may present the distribution of the number of days per week a participant eats at a program site, the distribution of the number of days since the participants’ last visit to the nutrition program, how often they attend the program relative to six months ago, and other participation characteristics.

10. Dietary Quality

The 24-hour dietary recall interviews include detailed descriptions of foods eaten, portions eaten, and the source of the food. A second day of dietary recalls will be collected for a portion of the original sample, to make possible estimation of the distributions of usual intakes of key nutrients. The data on each food will be coded so that information on the nutrients contained and the food group it belongs to can be analyzed.

We will use 24-hour recall data to assess the quality of diets consumed by congregate nutrition and home-delivered nutrition participants in two ways. First, we will analyze usual nutrient intakes relative to DRI standards. This will measure the adequacy of usual intakes of key nutrients and dietary components as well as measures of excessive intakes. Second, we will analyze overall diet quality and food group intakes using the Healthy Eating Index (HEI)-2005 (Guenther et al., 2007). The scores on the HEI-2005 provide a useful summary measure of diet quality relative to DGAs and My Plate recommendations for population groups.

11. Analysis of Usual Nutrient Intakes Relative to DRI Standards

The DRIs are defined on the basis of usual daily intakes, conceptually the long-term average daily intakes of individuals. However, usual intakes can seldom, if ever, be directly observed. Although a single 24-hour recall provides information on an individual’s observed daily (24-hour) intake, it provides a very imprecise estimate of that individual’s usual intake, as well as an inaccurate estimate of the distribution of intake levels across a population group. This is because individuals’ dietary intakes vary from day to day. This source of variation, known as intra-individual variation, is typically even larger than variation from one individual to the next within a population (inter-individual variation). If daily intakes are used to estimate intake distributions, the dispersion of the distribution will be larger than the dispersion of usual intakes, and estimates of the proportion of individuals whose usual intake of a particular nutrient is above or below a specific reference standard will be biased (Beaton et al. 1979).

Thus, to apply the DRIs appropriately, it is necessary to have information about the distribution of usual intakes within population groups. We will use the empirical method recommended by the Institute of Medicine (IOM) for adjusting observed daily nutrient intakes to obtain unbiased estimates of the distribution of usual intakes for a group (Institute of Medicine 2000). The method was first developed by the National Research Council (National Research Council, Subcommittee on Criteria for Dietary Evaluation 1986) and later modified by Nusser et al. (1996). It estimates the intra-individual variation in nutrient intake, based on a subsample of individuals with two days of intake data, and removes this source of variation before estimating the distribution of usual nutrient intakes across a population. The method uses a specialized software package, the Software for Intake Distribution Estimation (SIDE), in conjunction with a single 24-hour recall for all sample members and a second 24-hour recall for a randomly selected subsample, to estimate usual intake distributions.

The nutritional adequacy of diets of individuals and population groups will be assessed by comparing usual daily intakes of energy and nutrients to the DRIs. The DRIs, developed by the IOM, are the most up-to-date scientific standards for determining the proportion of individuals who have inadequate or excessive intakes of specific nutrients or other food components. DRIs have been established both for vitamins and minerals and for energy, fats, carbohydrates, protein, and dietary fiber (IOM 2006); and different values are specified for subgroups based on age, gender, and life stage. Table A.16.11 shows the different types of reference values that are used, depending on the nutrient.⁴ Table A.16.12 compares the usual daily intakes of energy and nutrients to the DRIs for congregate nutrition program and home-delivered nutrition program participants. Nutrition program participants consume substantial proportions of their total daily intake of nutrients from meals from congregate and home-delivered nutrition programs on days when they either attend the congregate meal site or receive home-delivered meals. We will estimate the percentage of total daily dietary intake provided by the program meal for the same set of vitamins, minerals, and the other dietary components and nutrients shown in Table A.16.12 (sample table of these percentages is not shown).

Table A.16.11 Definitions of Dietary Reference Intakes (DRIs)

Estimated Average Requirement (EAR)	The EAR is the level of intake that is estimated to meet the requirements of half of the healthy individuals in a particular life stage and gender group. The EAR is used to assess the prevalence of inadequate intakes using the IOM-recommended “EAR-cutpoint method” (IOM 2006). The EAR cut-point method will be used to analyze all nutrients for which EARs have been established. The EAR cut-point method assumes that nutrient requirements are symmetrically distributed.
Adequate Intake (AI)	An AI was defined when the data available for a particular nutrient were insufficient to estimate requirements and establish an EAR. The AI is the level of intake that is assumed to be adequate, based on observed or experimentally determined estimates of intake. AI cannot be used to determine the proportion of a population with inadequate intakes. Instead, assessment focuses on comparison of mean usual intakes to the AI. Populations with a mean usual intake equivalent to or greater than the population-specific AI can be assumed to have adequate intakes.
Tolerable Upper Intake Level (UL)	The UL is the maximum level of intake that is likely to pose no risks of adverse health effects for all individuals in a population group. As intake increases above the UL, the risk of adverse effects increases. For most nutrients for which ULs have been established, the UL is based on intake from food, water, and dietary supplements (e.g., fluoride, phosphorus, and vitamin C) (IOM, 2006). For some nutrients, the UL applies only to synthetic forms from dietary supplements, fortified foods, or over-the-counter medications (e.g., magnesium, folate, niacin, and vitamin E). The client survey data files will not include nutrients provided by water, dietary supplements, or over-the-counter medications. Thus, our ability to assess usual intakes relative to ULs will be limited. The prevalence of intakes above the UL for nutrients for which a UL is available has been found using nutrient intake data from the NHANES to be very small with the exception of sodium and a handful of results for other nutrients (Cole and Fox 2008). For this reason, we plan to include analyses of intake relative to the UL only for sodium.
Acceptable Macronutrient Distribution Ranges (AMDRs)	The DRIs specify AMDRs for macronutrients (protein, carbohydrate, and total fat) and fatty acids (linoleic acid and alpha-linolenic acid).5 AMDRs define ranges of macronutrient intakes that are associated with reduced risk of chronic disease, while providing recommended intakes of other essential nutrients. AMDRs are expressed as percentages of total energy intake because their requirements are not independent of each other or of the total energy requirement of the individual (IOM 2006). A key feature of AMDRs is that each has lower and upper bounds. Intakes that fall below or exceed these levels of intake may increase risk of chronic disease.

Table A.16.12. Usual Daily Intakes of Congregate Nutrition Program and Home-Delivered Nutrition Program Participants

	Congregate Nutrition Program Participants	Home-Delivered Nutrition Program Participants
Vitamins and Minerals with EARS (Percentage Less Than EAR)
Vitamin A
Vitamin C
Vitamin E
Vitamin B6
Vitamin B12
Folate
Niacin
Riboflavin
Thiamin
Iron
Magnesium
Phosphorus
Zinc
Minerals (Mean as Percentage of AI)
Calcium
Potassium
Sodium
Minerals (Percentage Greater than UL)
Sodium
Other Dietary Components
Fiber (mean as % of AI)
Fiber (mean g/1000 calories)
Cholesterol (% >DGA)
Energy (Mean)
Energy
Macronutrients
Total Fat
% < AMDR
% > AMDR
Saturated Fat
% > than DGA
Carbohydrate
% <EAR
% <AMDR
% >AMDR
Protein
% < EAR
% < AMDR
% > AMDR
Linoleic Acid
% < AMDR
% < AMDR

Source: Client Survey

AMDR=Acceptable Macronutrient Distribution Range, DGA=Dietary Guidelines for Americans recommendation, EAR=Estimated Average Requirement, AI=Adequate Intake, UL=Tolerable Upper Intake Level.

12. Analysis of Healthy Eating Index (HEI)-2005 Scores

The HEI-2005 is an updated version of the index originally developed by USDA’s Center for Nutrition Policy and Promotion (CNPP) in 1995 (Kennedy et al., 1995). The HEI-2005 was developed by a federal interagency workgroup led by CNPP. The index is designed to measure how well individuals’ diets conform to the 2005 Dietary Guidelines, including specific factors that influence diet quality, such as consumption of whole grains, specific types of fat, particular types of nutrient-dense vegetables, and “discretionary calories.” The HEI (in its original and updated form) has been adopted by USDA as the tool used to monitor diet quality of the U.S. population overall as well as progress toward healthier eating habits among food assistance program participants (USDA/FNS, 2000; Basiotis et al., 2002; USDA, 2006). The index includes 12 component scores—nine components assess intake of food groups (total fruit (including juice), whole fruit, total vegetables, dark green and orange vegetables and legumes, total grains, whole grains, milk, meat, and healthy oils), two components assess dietary components that are commonly consumed in excess (saturated fat and sodium), and one component assesses intake of discretionary calories from solid fat, alcohol, and added sugars (Guenther et al., 2007). Scores are assigned for each component based on reference standards that reflect Dietary Guidelines and MyPlate recommendations. Maximum scores for each component range from 5 to 20, with an overall maximum score of 100. HEI-2005 developers recommend that researchers focus on individual component scores rather than a total composite score because the individual scores provide the most useful data on shortcomings in diet quality.

We will estimate HEI-2005 scores for congregate nutrition and home-delivered nutrition program participants following the approach recommended by the interagency group that developed the measure (Table A.16.13). Ideally, the HEI-2005 would be calculated based on the usual dietary intake of each individual. As noted in the preceding discussion of usual nutrient intakes, with only one day of intake data for each sample member, we will not have a reliable estimate of each individual’s usual intake. However, a good estimate of a population’s mean usual intake is the mean of one-day intakes; and the best estimate of the population’s mean HEI scores is based on estimates of total intakes at the population level (Guenther et al., 2007; Freedman et al., 2008). Thus, we will assign HEI-2005 scores at the group rather than individual level, using the single 24-hour recall collected from all congregate nutrition and home-delivered nutrition participants.

Table A.16.13. Mean Healthy Eating Index-2005 Scores for Participants in Congregate Nutrition Programs

	Max. Score	Congregate Nutrition Program Participants’ Mean HEI-2005 Score	Congregate Nutrition Program Participants’ Mean HEI-2005 Score as a Percentage of Maximum Scores	Home-Delivered Nutrition Program Participants’ Mean HEI-2005 Score	Home-Delivered Nutrition Program Participants’ Mean HEI-2005 Score as a Percentage of Maximum Scores
Total Fruit (includes 100% juice)	5.0
Whole Fruit (not Juice)	5.0
Total Vegetables	5.0
Dark Green and Orange Vegetables and Legumesa	5.0
Total Grains	5.0
Whole Grains	5.0
Milkb	10.0
Meat and Beans	10.0
Oilsc	10.0
Saturated Fat	10.0
Sodium	10.0
Calories from SOFAAS	20.0
Total HEI-2005 Score	100.0

Source: Client survey.

SOFAAS = Solid Fats, Alcoholic beverages, and Added Sugars

^aLegumes counted as vegetables only after Meat and Beans standard is met.

^bIncludes all milk products, such as fluid milk, yogurt, cheese, and soy beverages.

^CInlcudes nonhydrogenated vegetable oils and oils in fish, nuts, and seeds.

A.16.2. Program Impacts on Client Outcomes

The analyses described thus far focus on the populations served by congregate nutrition and home-delivered nutrition programs by examining characteristics only of program participants. To assess program outcomes and effectiveness, however, we will use data from nutrition program participants and nonparticipants. The study will compare observed outcomes for a sample of program participants and a set of matched comparison observations, elderly people who are as similar as possible to the participant sample but do not participate in the Title III-C nutrition program. Impacts will be estimated on nutrition, food security, socialization, and health and institutionalization outcomes. Except for the nutrition outcomes (explained below), all analyses will use multivariate regression methods, with control variables reflecting client characteristics. Analyses will be conducted separately for (1) congregate nutrition program participants and nonparticipants and (2) home-delivered nutrition program participants and nonparticipants.

1. Nutrition Outcomes

It is not possible to use multivariate regression to estimate the impact of ENSP participation on the usual intakes of key nutrients because estimates of usual intakes are only available for a group of individuals, and not for each individual in the sample. To estimate differences between participant and nonparticipant groups in proportions with inadequate nutrient intakes, we will rely on the matching process used to identify appropriate nonparticipants for the sample, and estimate the prevalence of inadequate intakes for participants and separately for matched nonparticipants (Mabli et al. 2010). SIDE can also be used to estimate standard errors of these statistics that account for sample clustering. Table A.16.14 is an example of a table examining differences in usual intakes across matched participant-nonparticipant groups. The estimates will be presented by nutrition program type and, if sample sizes permit, by respondents’ age and gender.

Because the HEI-2005 scores will be also assigned at the group rather than individual level, comparisons of HEI-2005 component scores for nutrition program participants and nonparticipants will be made using the matched samples. Like the analysis of the impact on the usual intake of key nutrients, the matching process used to estimate the impact on HEI-2005 scores will include those factors typically included in a regression analysis to account for cross-group differences that are correlated with both the nutrition outcome measure and ENSP participation. Table A.16.15 compares HEI-2005 total scores and component scores across congregate nutrition and home-delivered nutrition participants and nonparticipants.

2. Food Security

The impact of ENSP participation on food security will be estimated within a multivariate regression framework that accounts for compositional differences across participant-comparison groups that might bias the impact estimates. We will use a logistic regression model that relates the probability of an individual’s being food insecure to a variable indicating whether the individual participates in the ENSP program and to a set of individual characteristics. The set of characteristics will include the individual’s gender, race and ethnicity, age, income, region of residence, and indicators for mobility limitations. It will also include variables that describe the composition of the individual’s family, such as whether he or she lives with other family members. Based on descriptive comparisons of participants and nonparticipants, we will assess whether to include variables measuring self-reported health status and other indicators of health status such as whether the individual has hypertension, high blood cholesterol, diabetes, or has had a stroke; whether the person takes vitamin supplements; whether the person has done any exercise, sports, or physical activity in the past 30 days (and how many times per week).

We will present the results of the food security analysis in several ways, using detailed tables and summary tables. First, we will present a table with the regression coefficients and standard errors (Table A.16.16) to help the reader understand what variables are used in the regression and how these results translate into the subsequent set of summary tables. Next, we will present regression-adjusted tables of program impact estimates that resemble the descriptive tables presented earlier (Table A.16.17). This table compares the rates of food insecurity across congregate nutrition program participant groups after accounting or adjusting for compositional differences across groups. A similar table will be produced for home-delivered nutrition program participation. Because we will use logistic regression analysis, the procedure for obtaining the regression-adjusted estimates consists of estimating the regression, using the regression coefficients and variable values for each individual in the sample to obtain a predicted probability of being food insecure, and averaging the predicted probabilities to obtain the adjusted (predicted) rate of food insecurity in the sample. By performing these steps assuming all sample members are participants and then repeating the procedure assuming all sample members are nonparticipants, we obtain two averaged values. The difference between these values is the regression-adjusted estimate of the impact of program participation on food insecurity.

Table A.16.14. Effects of Congregate Nutrition and Home-Delivered Nutrition Programs on Usual Daily Intakes by Participation Status, Age, and Gender

	Congregate Nutrition Participation												Home-Delivered Nutrition Participation
	Older Adult Males (60+)			Older Adult Females (60+)			Older Adults (60+)						Older Adult Males (60+)						Older Adult Females (60+)					Older Adults (60+)
	Part-icipant	Non-part-icipant		Part-icipant	Non-part-icipant		Part-icipant		Non-part-icipant				Part-icipant		Non-part-icipant				Part-icipant		Non-part-icipant			Part-icipant			Non-part-icipant
Vitamins and Minerals with EARS (Percentage Less Than EAR)
Vitamin A
Vitamin C
Vitamin E
Vitamin B6
Vitamin B12
Folate
Niacin
Riboflavin
Thiamin
Iron
Magnesium
Phosphorus
Zinc
Minerals (Mean as Percentage of AI)
Calcium
Potassium
Sodium
Minerals (Percentage Greater than UL)
Sodium
Other Dietary Components
Fiber (mean as % of AI)
Fiber (mean g/1000 calories)
Cholesterol (% >DGA)
Energy (Mean)
Energy
Macronutrients
Total Fat
% < AMDR
% > AMDR
Saturated Fat
% > than DGA
Carbohydrate
% <EAR
% <AMDR
% >AMDR
Protein
% < EAR
% < AMDR
% > AMDR
Linoleic Acid
% < AMDR
% < AMDR
Linolenic Acid
% < AMDR
% > AMDR

Source: Client Survey

Notes: Weighted tabulations based on first and second 24-hour recalls prepared by Mathematica Policy Research. Usual intake distributions were determined for each subgroup using the Software for Intake Distribution Estimation (SIDE).

Matched sample constructed using propensity score matching to adjust for differences in economic and demographic characteristics, including age, gender, race and ethnicity, marital status, household size, and income relative to poverty. Estimates weighted to account for sample design.

AMDR=Acceptable Macronutrient Distribution Range

DGA=Dietary Guidelines for Americans recommendation

EAR=Estimated Average Requirement

AI=Adequate Intake

UL=Tolerable Upper Intake Level

* or ** denotes difference between participant and nonparticipant group is statistically significant at the 0.05 or 0.01 level of significance.

Table A.16.15. Effects of Congregate Nutrition Program and Home-Delivered Nutrition Program Participation on Mean HEI-2005 Scores

		Congregate Nutrition Program			Home-Delivered Nutrition Program
Component	Maximum Possible Score	Participant	Non-participant	Difference	Participant	Non-participant	Difference
Total Fruit
Whole Fruit (not juice)	5
Total Vegetables	5
Dark Green and Orange Vegetables and Legumes	5
Total Grains	5
Whole Grains	5
Milk	10
Meat and Beans	10
Oils	10
Saturated Fat	10
Sodium	10
Calories from Solid Fat, Alcohol, and Added Sugar (SoFAAS)	20
Total Score	100

Source: Client Survey

Notes: Weighted tabulations based on first and second 24-hour recalls prepared by Mathematica Policy Research.

Matched sample constructed using propensity score matching to adjust for differences in economic and demographic characteristics, including age, gender, race and ethnicity, marital status, household size, and income relative to poverty, and other characteristics. Estimates weighted to account for sample design.

Table A.16.16. Regression Coefficients of the Effects of Congregate Nutrition Program and Home-Delivered Nutrition Program Participation and Individual Characteristics on an Individual’s Likelihood of being Food Insecure

	Coefficient	Standard Error
Congregate Nutrition Program Participation
Home-Delivered Nutrition Program Participation
Gender (male is referent group)
Female
Race and Ethnicity (non-Hispanic white is referent group)
Non-Hispanic black
Non-Hispanic other
Hispanic
Age (60 to 65 is referent group)
66-70
71-75
75-80
80 and older
Employed full time
Employed part time
Unemployed
Monthly Income as a Percentage of Poverty (less than 50% is referent group)
50% to 100%
101% to 200%
201% to 300%
Greater than 300%
Household Contains Children (referent group is no children)
Household Contains Other Elderly (referent group is no other elderly)
Household Size (One household member is referent group)
Two
Three
Four or more
Region of Residence (western region is referent group)
Northeast
Mid-Atlantic
Midwest
Southeast
Southwest
Mountain Plains
Mobility
Able to walk
Uses a cane or walker
Difficulty walking or climbing stairs
Bed Bound
Chair bound or in a wheelchair

Source: Client survey.

Table A.16.18. Regression-Adjusted Percentages of Individuals That Are Food Insecure, by Congregate Nutrition Program Participation Status

	Congregate Nutrition Program Participant	Congregate Nutrition Program Nonparticipant	Difference
All Individuals
Gender
Male
Female
Race and Ethnicity
Non-Hispanic white
Non-Hispanic black
Non-Hispanic other
Hispanic
Age
60 to 65
66-70
71-75
75-80
80 and older
Employment Status
Employed full time
Employed part time
Unemployed
Out of the labor force
Monthly Income as a Percentage of Poverty
Less than 50%
50% to 100%
101% to 200%
201% to 300%
Greater than 300%
Household Contains Children
Household Does Not Contain Children
Household Contains Other Elderly
Household Does Not Contain Other Elderly
Household Size
One
Two
Three
Four or more
Region of Residence
Northeast
Mid-Atlantic
Midwest
Southeast
Southwest
Mountain Plains
Western
Mobility
Able to walk
Uses a cane or walker
Difficulty walking or climbing stairs
Bed Bound
Chair bound or in a wheelchair

Source: Client survey

3. Socialization Opportunities

Socialization will be measured using responses to questions about how satisfied individuals are with the opportunities they have had to spend time with other people; and how often they felt that they lack companionship, felt left out, or felt isolated from others. Like the analysis of food security, we will estimate a logistic regression that relates the probability of an individual experiencing social isolation to a variable indicating whether the individual participates in the ENSP program and to a set of individual characteristics. For social isolation questions with response options “hardly ever; some of the time; or often”, we will estimate the probability that an individual experiences these events “often” compared to experiencing them “hardly ever” or “some of the time”. We will examine the sensitivity of the results to the definition of the dependent variable by grouping “often” and “some of the time” in an auxiliary model. The set of explanatory variables, besides congregate or home-delivered nutrition program participation status, likely will be similar to those used in the food security analyses. We will present the results of the regressions by presenting tables of regression-adjusted probabilities of experiencing social isolation, similar in presentation to the regression-adjusted food security tables above. Tables containing the full set of regression coefficients and standard errors will be available in an appendix.

4. Health and Institutionalization Outcomes

To estimate the impact of program participation on participants’ health and institutionalization, we will link the survey data on elderly participants (clients) and non-participants to data from Medicare claims records using client social security numbers. The claims data will be obtained for at least 12 months prior to the interview date and for 6 months after the interview date for both baseline nutrition program participants and nonparticipants. The data will contain information used to construct the health and institutionalization outcome measures, including whether the individuals has been hospitalized, number of hospitalizations, presence and number of 21 chronic conditions the beneficiary was treated for in the prior one to three years, whether had home health care, number of months received home health care, whether had skilled nursing facility (SNF) care, and total Medicare costs.

As described earlier, when evaluating the program impacts on elderly nutrition and food security, we will use dietary data from a 24-hour recall period and the food security information from a 30-day recall period. Both of these recall periods are measured almost concurrently with the respondent’s nutrition program participation status. To evaluate the program impacts on more long-run outcomes such as health and institutionalization, however, we will use client survey data on the history of program participation prior to the baseline interview date and the short-term history of program participation status prior to the follow up interview date. Using nutrition program participation histories at the baseline and follow up interviews will improve our ability to distinguish between (1) whether participating in nutrition programs leads to improved health and (2) whether deteriorating health precedes the start of nutrition program participation. That is, the program participation histories will be helping in estimating the program impact on the longer-term outcomes of health and institutionalization. Variables characterizing these histories will be included in the regression model used to estimate the program impacts. Similar to the food security and socialization tables, impacts on health and institutionalization will be presented using regression-adjusted probabilities of having home health care or being treated for a chronic condition in the prior one to three years and so on.

Compared to the evaluation of program impacts on nutrition, food security, and socialization, the evaluation of impacts on health and institutionalization should be viewed as more exploratory and less definitive given that longer-term outcomes will be evaluated using data from a relatively short period of time. Study findings will likely contain language that emphasizes this is exploratory work, discusses the limitations of the empirical approach, and places the research findings in the larger literature of related research on longer-term health outcomes.

A.16.3. Study Schedule

The schedule shown in Exhibit A.16.19 lists the expected start date for the data collection and reporting. Our data collection plans are designed to provide timely data for the evaluation reports.

Exhibit A16.19: Approximate Schedule of Tasks and Deliverables

Objectives/Major Tasks	Status
Award Date	completed
Work Plan and Project Communications	completed
Refinement of Data Collection Tools	completed
Data Use Request Packets for Access to Medicare and Medicaid Files	completed
Sample Development and Selection	completed
Hire and Train Staff for Process and Cost Studies	completed
Conduct SUA Process Survey	completed
Conduct AAA and LSP Process Surveys	completed
Conduct Cost Study	completed
Perform the Title III-C Elderly Nutrition Process and Cost Study Data Analysis	completed
Prepare and Submit Executive Summary, Methods, and Results and Findings Report	completed
Develop and Draw Client Outcome Sample	completed
Hire and Train CAPI Data Collectors for the Client Outcomes Study	completed
Client and Comparison Group Data Collection	Completed for baseline and 6 month follow up. 12 month follow up is scheduled for fall 2016 and winter 2017
Data Processing, Merging with Medicare and Medicaid Files	Summer 2016 and fall 2017 and
Perform the Title III-C Elderly Nutrition Program Evaluation Data Analysis	Baseline report expected January 2017 Final report expected Spring 2018
Publish reports on ACL website	Approximately 3 months after receipt

A.17. Reason(s) Display of OMB Expiration Date is Inappropriate

ACL does not seek this exemption. All data collection instruments for the ACL Evaluation of the Title III-C Program will display the OMB approval number and expiration date. ACL does not seek this exemption.

A.18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the Certification for Paperwork Reduction Act (5 CFR 1320.9) for this study.

References

Basiotis P., Carlson A., Gerrior S., Juan W. Y., Lino M. The Healthy Eating Index: 1999-2000. Alexandria, VA: U. S. Department of Agriculture, Center for Nutrition Policy and Promotion. Report CNPP-12; 2002.

Beaton, G.H., Milner, J., Corey, P., McGuire, V., Cousins, M., Stewart, E., de Ramos, M., Little, J.A. Sources of variance in 24-hour dietary recall data: Implications For Nutrition Study Design and Interpretation. The American Journal of Clinical Nutrition (1979). Vol 32, Issue 12, Ogs 2546-2559

Freedman L.S., Guenther P.M., Krebs-Smith S., Kott P.S. “A population’s mean Healthy Eating Index-2005 scores are best estimated by the score of the population ratio when one 24-hour recall is available,” J Nutr. 138: 1725-29; 2008.

Guenther, P.M., Reedy, J., Krebs-Smith, Reeve, B.B., & Basiotis, P.P. (2007). Development and Evaluation of the Healthy Eating Index: Technical Report. Center for Nutrition Policy and Promotion, US Department of Agriculture.

Institute of Medicine (2000). Dietary Assessment Intakes: Applications in Dietary Assessment. Washington, DC: National Academies Press.

Kennedy E.T., Ohls J., Carlson S., Fleming, K. “The Healthy Eating Index: Design and applications.” J Am Diet Assoc. 95(10):1103-08; 1995.

National Research Council, Subcommittee on Criteria for Dietary Evaluation. “Nutrient Adequacy: Assessment Using Food Consumption Surveys.” Washington DC: National Academies Press, 1986.

Nusser, S.M., A.L. Carriquiry, K.W. Dodd, and W.A. Fuller. “A Semiparametric Transformation Approach to Estimating Usual Daily Intake Distributions.” Journal of the American Statistical Association, vol. 91, 1996, pp. 1440–1449.

Ponza, M., Ohls, J., Millen, B. E., McCool, A.M., Needels, K.E., Rosenberg, L., Chu, D. Quatromonic, P.A. 1996. Serving Elders At Risk: The Older Americans Act Nutrition Programs: National Evaluation of the Elderly Nutrition Program. Report to the U.S. Department of Health and Human Services, Assistant Secretary for Aging. Washington, DC: Mathematica Policy Research, July.

U. S. Department of Agriculture, Food and Nutrition Service. Food and Nutrition Service (FNS) Strategic Plan 2000-2005. Alexandria, VA; 2000.

U. S. Department of Agriculture. Strategic Plan for FY 2005-2010. Washington, D.C.; 2006.

1 Detailed justification for the need for the sensitive information during earlier phases of the evaluation is available in the original OMB package.

2 The analyses of participant characteristics that form the basis of the targeting analyses are described in Section 3.

3 Meals must provide a minimum of two thirds of the DRIs if two meals per day are provided by the program and 100 percent of the DRIs if three meals per day are provided.

4 The DRIs also provide estimated energy requirements (EERs), which are based on age, gender, median height and weight, and level of physical activity. The actual energy requirement for an individual varies considerably with body size and activity level, which is difficult to measure accurately. Thus, for this study, we will present results for mean usual energy intakes rather than assessing intake relative to EERs.

5 Usual carbohydrate intakes are also assessed relative to EARs, based on total intake (gm/day).

Page 1 of 80