ENSP OMB Part A (formatted) REVISED FINAL PROCESS AND COST ONLY11-18-13....

Part A

A.1. Circumstances Making the Collection of Information necessary

A.1.1. Authorizing Legislation

The Older Americans Act (OAA) Title III-C Elderly Nutrition Services Program (statutory authority is contained in Title II section 205(a)(2)(A), and Title III sections 311, 331, and 336 of the Older Americans Act (OAA) (42U.S.C. 3032), as amended by the Older Americans Act Amendments of 2006, P.L. 109-365) is part of comprehensive home-and community-based services. Title III, Part C provides grants to States and Territories under approved State Plans for the establishment and operation of nutrition projects for congregate nutrition services (Section 331) and home-delivered nutrition services (Section 332). In addition, Section 311 authorizes the Nutrition Services Incentive Program (NSIP) which provides cash or commodities to States and Tribes for the provision of meals served in compliance with the requirements of the Older Americans Act. Section 339 establishes the requirements for the provision of nutrition services. The legislative purposes of Part C as found in section 330 are “to reduce hunger and food insecurity; to promote socialization of older individuals; and to promote the health and well-being of older individuals by assisting such individuals to gain access to nutrition and other disease prevention and health promotion services to delay the onset of adverse health conditions resulting from poor nutritional health or sedentary behavior. ”

The authorizing legislation for the data collection is found in Title II of the OAA. The requirements stipulated under section 206(a, c) directs ACL to “…measure and evaluate the impact of all programs authorized by this Act, their effectiveness in achieving stated goals in general, and in relation to their cost, their impact on related programs, their effectiveness in targeting for services under this Act unserved older individuals with greatest economic need (including low-income minority individuals and older individuals residing in rural areas) and unserved older individuals with greatest social need (including low-income minority individuals and older individuals residing in rural areas), and their structure and mechanisms for delivery of services, including, where appropriate, comparisons with appropriate control groups composed of persons who have not participated in such programs. Evaluations shall be conducted by persons not immediately involved in the administration of the program or project evaluated.”

A.1.2. Background

1. Ensuring the Nutritional Needs of the Elderly

The Title III-C Elderly Nutrition Services Program (ENSP) represents a key component of America’s strategy for ensuring that the needs of elderly people are adequately met. Every day, millions of Americans, most of them over 65 years of age, receive a nutritious meal at a Senior Center or other congregate meal site. Many others consume a home-delivered meal provided under a different component of the program.

The value of these services to participants goes far beyond the meals themselves. Particularly for participants in congregate meals, Title III-C meals provide an opportunity to socialize and with peers. Further, many other services—from tax preparation to recreational activities to medical screening—are often provided at Senior Centers and other Title III-C sites, and the congregate meals provide a context for helping seniors connect to these services as well. Even for home-delivered meals, which by definition are less focused on social interaction, the daily visit by the meal deliverer, often a volunteer, can represent an elderly recipient’s only human contact of the day.

2. Need to Evaluate the Title III-C Program

An important aspect of the program, critical to understanding how it functions, is the way in which it has developed mechanisms for mobilizing multiple levels of constituencies in the work of serving the elderly. While overall federal coordination is provided by the AoA, the State Units on Aging (SUAs) and the Area Agencies on Aging (AAAs) both support key aspects of program operations. In turn, the direct nutritional services are provided by Local Service Providers (LSPs). Many other governmental and nonprofit groups, as well as some groups organized on a for-profit basis, are also involved in serving the elderly under the program. Often the Title III-C program, with the attraction of its ability to provide inexpensive meals, is a catalyst for this broader involvement.

While the diversity of the organizations involved is a key strength of the Title III-C program, it also creates particular challenges for evaluating the program. Indeed, this diversity makes it particularly complicated (and also particularly important) to examine whether the system operates efficiently overall, and whether it succeeds in delivering services that are of benefit to the elderly, as evidenced by such important outcomes as nutrition, socialization, health, and, ultimately, avoidance of institutionalization. It is also important to examine the targeting of the program, to assess whether its services are reaching the elderly that need them most and to assess whether there may be underserved populations that are not being served by the overall program. So, in addition to the legislative mandate under the OAA, it is important for program integrity and function to evaluate this program.

3. Evaluation Objectives

The overall evaluation of the Title III-C Program has three broad objectives: (1) to provide information to support program planning, including an analysis of program processes (process evaluation), (2) to develop information about program efficiency and cost issues (cost study), and (3) to assess program effectiveness, as measured by the program’s effects on a variety of important outcomes, including nutrient adequacy, socialization opportunities, health outcomes, and, ultimately, helping elderly people avoid institutionalization (outcome evaluation). See Table A.2.1 for more detail about the evaluation questions associated with each component. Please note that data collection related to the third objective will be conducted under the second phase of the evaluation and, the relevant data collection tools will be submitted for OMB PRA clearance separately. This request for clearance refers only to evaluation of the first two objectives-the Process Evaluation and the Cost Study. Limited information about a separate Outcome Evaluation is included only to provide context for this information collection and the larger goals of AoA with regards to a comprehensive evaluation of the ENSP.

A.2. Purpose and Use of the Information Collection

The data collected during the evaluation is essential to ACL for meeting the needs of a rigorous evaluation of the impact of the Title III-C Program. There is currently no other national effort that addresses the research objectives of the proposed study. The resulting information will be critical to federal policymakers and will assist all levels of the aging network as ACL attempts to maximize efficiency and service.

Data will be used by ACL staff to improve program operations, provide improved technical assistance and guidance to grantees and service providers, and to support mandated agency reporting to congress and through annual reports. Data from the Process Evaluation and Cost Study will be analyzed to determine the extent to which the ENSP is operating efficiently. With regard to the process evaluation, many levels and types of organizations interact to provide congregate meals, home-delivered meals, and nutrition services (e.g. education and counseling) under the Title III-C program. It will be important to describe these interactions as fully as possible and to examine stakeholder views of whether, and in what ways, this multilevel organization contributes to or interferes with program effectiveness. Other key issues include exploring ways to streamline program operations and ensuring efficient use of technology to guide management decisions within the program.

We will conduct surveys of SUA, AAA, and LSP staff. These surveys will provide data on such factors as the use of technology, program decision processes, characteristics and objectives of the Title III-C organizations, how they are staffed, and the measures used both to coordinate their internal staff and to coordinate with other participant organizations.

The Cost Study will obtain very detailed cost-related data from project sites. This data will include information on the real resources involved in meal production and distribution by category (labor, food, external meal purchases, utilities, and the like) and then obtaining unit price data to value those resources (wages, fringe benefits, food prices, utility prices, and so on). We will then develop cost estimates with this information by summing the products of resource use and prices and dividing by numbers of meals produced and distributed. Sites will be able to use this information to assess their own costs in the context of other similar sites and ACL will use this information to help determine which approaches (and within which circumstances) are most cost effective. This information will be shared with sites to help them manage their programs.

.

Table A.2.1. Summary of Evaluation Design: Evaluation Goals, Illustrative Study Questions, Research Strategy, and Data Collection Plan

The Process Evaluation: Program planning, processes and administrative analysis
Goals/Study Questions	Research Strategy	Data to be Collected
What partnerships are being formed with other organizations? Are there opportunities for more? Is technology being used to ensure strong databases for decision-making? What mechanisms are used to ensure targeting of program services? What is the organizational structure and staffing of agencies and providers? How satisfied are participants with program services?	Descriptive analysis of information from program staff. Tabulations of their use of technology and of decision-making procedures. Tabulations of participant satisfaction and of participant characteristics to examine targeting. Tabulation of process interviews at multiple levels. Cross tabulations to examine how satisfaction and other success measures are correlated with various administrative variables. Another line of analysis will be to undertake menu coding to calculate the nutrient content of the Title III-C meals served	Interviews of SUA, AAA, and LSP staff Participant interviews for satisfaction and other data Analysis of meal content will be based on weekly menu information obtained from sites.
The Cost Study: Examine program improvement opportunities; include cost efficiency issues
Goals/Study Questions	Research Strategy	Data to be Collected
What is the average meal cost? Does it vary by how meals are provided or by meal content? Does it vary by other program characteristics?	Average costs will be estimated using detailed information on “real” resource usage and on resource costs. The determinants of those average costs will then be examined using both tabular and multivariate methods. The analysis will explore such possible determinants as meal preparation, meal service methods, and the content of the meals.	Data on program resource use and costs will be obtained through worksheets and detailed discussions with local food service staff. Meal counts and food service characteristics, together with other operational information will come from interviews with LSP staff.

A.3. Use of Improved Information Technology and Burden Reduction

The questionnaires for the AAA’s and LSP’s will be conducted over the internet as web surveys. We choose this mode to avoid redundancy and reduce burden for the 40% of the AAAs who are also LSPs. Specifically, we will use web technology to administer a single instrument (rather than multiple instruments) that provides customized text and routing depending on the organizations’ characteristics. Question text will be designed for both self- and interviewer- administered collection so that telephone interviewers can also enter respondents’ answers directly in the web survey system.

A.4. Efforts to Identify Duplication and Use of Similar Information

The ACL sought to avoid duplication of effort in both design and data by trying to identify existing instruments and data sets relevant to the study. It was concluded that no existing data sources can provide data needed to answer the study’s research questions. But, where possible, data for the process evaluation (such as selected fiscal data) will be pulled from existing data reports submitted by states and confirmed with sites rather than asking them to gather the data anew. ACL will also try to reduce duplication across its evaluation projects by making a concerted effort to ensure that, where possible, questions are shared across projects. This will allow for data aggregation across projects which will permit additional analyses, including comparative analyses, which would otherwise not be possible.

A.5. Impact on Small Businesses or Other Small Entities

Information being requested or required has been held to the minimum required for the intended use. We will request information from SUAs, AAAs, and LSPs. SUA directors will have the option of completing a self-administered survey or answering the questions over the telephone. AAA and LSP surveys will be conducted via a web-based survey that will be programmed with customized text and routing to accommodate the 40 percent of AAAs who are also LSPs thereby reducing their burden by eliminating redundant questions. Technical assistance will be provided by telephone.

A.6. Consequences of Collecting the Information Less Frequently.

Each data collection tool will be used to collect data one time per respondent. This is one time total for SUAs, four data collections from AAAs/LSPS, and five collections for each individual. If these data are not collected, ACL will not be responsive to the requirement in Title II, Section 206 of the Older American Act of 1965 that all authorized programs should be evaluated. The Act specifically authorizes the ENSP and that the Assistant Secretary of Aging will measure the impact of all programs authorized in the Act (The response A1 above contains text from this section of the Act). Further, if the information is not collected, ACL will lack important information needed for program improvement.

A.7. Special Circumstances Relating to Guidelines of 5 CFR 1320.5

There are no special circumstances. The collection of information is conducted in a manner consistent with guidelines in 5 CFR 1320.5.

A.8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

A.8.a. Federal Register Notice

An announcement was published in the Federal Register on April 5, 2012 (Federal Register Volume 77, Page 20637). One comment was received but was not relevant to the data collection and no action was taken. It should be noted that the Federal Register notice referred to all three components of the proposed evaluation, but this Justification refers only to the first two components of the evaluation (the Process Evaluation and Cost Study).

A.8.b. Outside Consultations

Planning for data collection in this study has involved extensive consultation between ACL and the Mathmatica Policy Research (the research team contracted to conduct the ENSP evaluation). ACL contacted Alana Moshfegh of the Agricultural Research Service to secure rights to use the AMPM software for the menu survey and reviewed the technical rigor of this effort. In addition, this work has been informed by contributions from a Technical Advisory group that includes service providers, nutrition services contractors, Area Agency on Aging staff, State Unit on Aging staff, and evaluation experts.

A.9. Explanations of Any Payment or Gift to Respondents

No incentives will be provided to respondents for the Process Evaluation and Cost Study.

A.10. Assurance of Confidentiality Provided to Respondents

ACL is committed to protecting the security of all study data. While no PII will be collected as part of the Process Evaluation and Cost Study, the following data handling and reporting procedures will be employed to maintain the privacy of composite electronic files.

• Confidentiality Agreement. All project staff will be required to sign a confidentiality statement. In this agreement project staff pledge to maintain the confidentiality of all information collected from the respondents and will not disclose it to anyone other than authorized representatives of the evaluation, except where otherwise required by law. Issues of confidentiality are discussed during interviewer training.

• Data on Central Office Computers. Standard backup procedures will be implemented for the central office computer system to protect project data from user error system failure. Backups and inactive files will be maintained on tape or compact disks. The system servers will be maintained inside a secure locked area accessible only to authorized systems personnel. Files will be accessible only by authorized personnel who have been provided project logons and passwords. Access to any of the study files (active, backup, or inactive) on any network multi-user system will be under the central control of the database manager who will ensure that the appropriate network partitions used in the study are appropriately protected (by password access, decryption, or protected or hidden directory partitioning) from access by unauthorized users.

• Documents Received in Central Office. Once in the central office, documents containing respondent information are kept in locked filing cabinets. At the close of the study, such documents are shredded.

The individuals participating in this study will be notified that the information they provide will not be released in a form that identifies them or their specific organization, except as otherwise required by law. No identifying information will be attached to any reports or data supplied to the ACL or any other researchers.

A.11. Justification for Sensitive Questions

No data of a sensitive nature will be collected through this data collection. All data collected under Phase 1 of the evaluation (Process Evaluation and Cost Study) involves organizations and is focused on the work processes involved in administering the ENSP grants.

A.12. Estimates of Annualized Burden Hours and Costs

Exhibit A.12.1 shows sample sizes and estimates on burden, frequency of response, annual responses per respondent, and annualized cost of respondent burden for each part of the data collection and for total burden.

Exhibit A.12.1. Estimated Respondent Burden

Survey	Respondent	Estimated No. Respondent	Responses Annually Per Respondent	Total Annual Responses	Estimated Avg. # of Hours Per Response	Estimated Total Hours	Estimated Hourly Ratea	Total Cost
Process Survey
SUA (mail and fax-back survey)	SUA directors	56	1	56	1.68	94	$35.82	$3,370
AAA (web and fax-back survey)	AAA directors	300	1	300	1.78	534	$35.82	$19,128
LSP (web and fax-back survey)	LSP directors	200	1	200	2.19	438	$35.82	$15,689
Cost Survey
LSP (paper self-administered questionnaire)	LSP	200	1	200	1	200	$35.82	$7,164
LSP phone follow up for Cost Survey	LSP	200	1	200	1	200	$35.82	$7,164
TOTALS		956		956
Total—hours						1,466
Total—minutes						87,960
Average response--hours						1.5
Average response--minutes						92

^a Sources: Bureau of Labor Statistics, National Compensation Survey, 2010, May 2011, Bulletin 2753. (http://www.bls.gov/ncs/ncswage2010.htm#Wage_Tables): SUA, AAA, LSP staff: Average hourly wage of state and local government social and community service managers; participants and non-participants: national minimum wage

A.13 Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

There are no capital, start-up, or annualized maintenance costs associated with this data collection for respondents.

A.14 Annualized Cost to the Federal Government

The cost to the Federal government for the all tasks associated with Phase 1 of the Title III-C Program evaluation (Process Evaluation and Cost Study) is $1,555,230. This expense includes the costs associated with the contractor conducting the project and the partial salary of the assigned ACL project officer.

A.15 Explanation for Program Changes or Adjustments

This is a new collection of information. The one time, estimated total amount of burden across all respondents for this data collection is 1,466 hours.

A.16 Plans for Tabulation and Publication and Project Time Schedule

The research planned for the study has three broad objectives: (1) to provide information to support program planning, including an analysis of program processes and assessing the nutritional quality of meals offered to program participants; (2) to generate information about program efficiency and costs; and (3) under the Phase 2 Outcome Evaluation to assess program effectiveness, as measured by the program’s effects on a variety of important elderly participant outcomes, including nutrient adequacy, socialization opportunities, health outcomes, and, ultimately, helping elderly people avoid institutionalization.

For each objective, the research team will analyze the data collected. The research team will prepare a final report for each Phase and conduct a briefing for ACL staff. The reports will present findings from both descriptive analyses of agency characteristics, program meals, and costs, as well as descriptive and multivariate analyses of program participant outcomes. This section presents the analysis plans for addressing the study objectives fulfilled by the Process Evaluation and eth Cost Study and provides the corresponding project schedule.

A.16.1. Program Planning, Processes and Administrative Analysis and Assessing Nutritional Quality of Program Meals. (Process Evaluation)

The approach for addressing the first component of the first study objective—program planning, processes and administrative analysis—will draw heavily on information obtained from comprehensive surveys of staff from organizations at all levels of the program hierarchy--SUAs, AAAs, and LSPs. These surveys will elicit information about the use of technology, program decision-making processes, characteristics and objectives of the Title III-C organizations, how they are staffed and staff qualifications, and the approaches used both to coordinate their internal staff and to coordinate with other participant organizations. This descriptive process information will be tabulated for each organizational level. In addition, results will be cross-tabulated with characteristics of the organizations to examine patterns in the data that could explain the tabular findings.

The study will describe the organizational structure of SUAs, AAAs, and LSPs and the populations served by these agencies and programs. For example, Table A.16.1 presents the percentages of SUAs that are independent agencies within the state government, that are part of an umbrella agency, and that are part of a board or commission. Examples of similar tables include describing the types of non-OAA programs the SUAs administer; whether the management structure for AAAs can be characterized as not for profit private agencies, for profit agencies, divisions of city or county government, or other types; and what the urbanicity of AAA’s Planning and Service Areas (PSAs) are and whether it extends across a single county or city or multiple counties or cities. Other tables will describe the types of nutrition and non-nutrition services LSPs provide, the LSP management structure, and the LSP service population and service area, as well as the populations that the agencies and programs serve through.

Table A.16.1. Organizational Structure of SUAs (Process Evaluation)

Characteristic	Percentage
Type of SUA
Independent agency within state government
Part of umbrella agency
Part of a board or commission
Type of umbrella agency among SUAs that are part of umbrella agency
Human service
Health
Medicaid
Welfare
Health and social/human/family services
Governor/Lt. Governor’s office
Community/cultural affairs
None of the above
Non-OAA programs administered by SUA
Type of SUA
Medicaid institutional care
Medicaid waiver(s)
Energy assistance (LIHEAP)
State health insurance counseling and assistance program (SHIP)
Pre-admission screening and resident review screening for mental illness (PASRR)
State funded HCBS
SNAP
CACFP
Emergency food assistance (TEFAP)
Commodity supplemental food program (CSFP)
Senior farmers market (SFMP)
None of the above
Average (Median) Number of AAAs in state
Average percentage of AAAs in state characterized by PSA boundary
Single-county
Multi-county
Single city/Metro area
Multiple city/Metro area
Other
Average (Median) Number of AAAs in state
Average percentage of AAAs in state characterized by PSA boundary
Single-county
Multi-county
Single city/Metro area
Multiple city/Metro area
Other

Source: SUA survey.

The study will also describe nutrition program quality at SUAs and nutrition service operations at AAAs and LSPs. Specifically, we will determine the extent to which the dietary reference intakes (DRIs) and Dietary Guidelines for Americans had been implemented throughout the SUAs, and how frequently SUA policies on this implementation are updated. We will also examine the ways in which AAAs and, to an extent, LSPs, contribute to the quality of nutrition education, nutrition counseling, and the nutrient quality of meals (Table A.16.2).

Table A.16.2. Nutrition Policies and Practices at AAAs and LSPs (Process Evaluation)

Characteristic	Percentage of AAAs	Percentage of LSPs
Contributions to the quality of nutrition education
Require credentialed nutrition professional to conduct education
Conduct a survey of program participants’ need
Use evidence-based education programs
Use cooperative extension materials
Use curricula from a reliable, science-based organization (academia, government, American Heart Association, American Diabetes Association)
None of the above
Contributions to the quality of nutrition counseling
Require credentialed nutrition professional to conduct the counseling
Require use of protocols approved by a respected source such as the American Dietetic Association, Patient Education Association, or Association of Diabetic Educators
Require credentialed non-nutrition professionals (e.g. nurses, diabetes educators, etc.) to conduct nutrition counseling
None of the above
Contributions to the nutrient quality of meals
Conduct computer-assisted menu analysis
Implement meal pattern requirements
Use of dietitian or state credentialed nutrition professional
State Unit on Aging guidance
Older Americans Act guidance
None of the above

Source: AAA survey and LSP survey.

Program Improvement Opportunities, Efficiency and Cost (Cost Study)

To examine issues related to the second study objective—program improvement opportunities, including cost-efficiency issues—the study will use detailed cost-related data collected from project sites. This will include information on the “real” resources involved in meal production and distribution by category (labor, food, equipment, rent, and so on) as well as unit price data to value those resources (wages, fringe benefits, food prices, utility prices, and so on). Next, cost estimates will be developed with this information by summing the products of resource use and prices and by dividing by numbers of meals produced and distributed.

The study will identify the key factors most likely to affect meal costs and program efficiency. Table A.16.6 illustrates an example of how we will present average cost per congregate meal and home-delivered meal overall, and by cost type (labor and nonlabor) and by cost component within each type (labor costs associated with paid versus volunteer labor; and nonlabor costs such as rent, insurance, utilities, equipment, food, and other costs). We will cross-tabulate costs by meal preparation method (on-site, central kitchen, or vendor). Analogous tables (not shown) presenting costs by program size and region of location will also be generated. Program size will be determined by the number of meals served (or delivered) per week and geographic location will be determined using the four Census regions. Although most of the tables will focus on average cost, the distribution of costs will also be examined to identify “outliers” that could signal inefficiency at some sites (or perhaps particularly cost-effective operations at others).

Table A.16.6. Average Cost Per Congregate and Home-Delivered Meal (in dollars)

Cost Component	Congregate Meals	Home-Delivered Meals
Total labor costs
Paid labor
Site
Central kitchen
Central administration
Delivery to site
Delivery to home
Volunteer Labor
Site
Central kitchen
Central administration
Delivery to site
Delivery to homes
Total Non-Labor Costs
Foods/Vendor
Supplies
Rent
Insurance/Utilities
Equipment
Other Costs
Donated food/space
Total Paid Costs
Total Costs (Paid and Non-Paid)
Unweighted Sample Size

Source: Agency Cost Survey.

This component of the study will also use multivariate analysis methods to assess the separate contributions that agency characteristics and services make to unit meal costs controlling jointly for the effects of these factors. Table A.16.7 illustrates how we will present the regression coefficients and probability values for congregate nutrition programs and home- delivered nutrition programs. Explanatory variables will be selected to include in the regressions after performing initial diagnostic work using the data, but the set of variables will likely include program size, region of location, whether the program is located in an urban or rural area, the length of time a program has been in operation, and agency type (not for profit; for profit; governmental; tribal; educational; other).

Table A.16.7. Regression Analysis of the Average Monetary Meal Cost with Program Characteristics

	Congregate Meals		Home-Delivered Meals
Characteristic	Coefficient	P-Value	Coefficient	P-Value
Urban locationa
Large programb
Purchase from vendor
Region
North
Midwest
South
Offer special mealsc
Operating more than 15 years
Constant
R2
Unweighted Sample Size

Source: Agency Cost Survey and LSP Survey.

^aParticipants zip codes were used to define rural location according to the Census definition. As defined by the Census, urban areas comprise (1) urbanized areas (incorporated places and adjacent densely settled territory with a combined minimum population of 50,000), and (2) all other places with 2,500 or more persons. “Rural area” means may area that is not defined as urban

^bA large congregate program serves 1,000 or more meals per week. A large home-delivered nutrition program delivers 1,000 or more meals per week.

^cA project offers special meals to participants if it offers religious, ethic, or therapeutic/modified meals.

The study will also examine sources of funding. In particular, the OAA permits the transfer of funds between the congregate nutrition, home-delivered nutrition, and supportive services programs to allow agencies to direct resources where they are most needed. An examination of the diverse set of funding transfers that support program meals and services will complement the agency cost analysis. We will present the percentage of SUAs and AAAs reporting that they transferred funds between the congregate nutrition, home-delivered nutrition, and supportive services programs. We will also present the average amount transferred among those agencies that transferred funds. Like the program cost tables, the funding transfer tables will be cross-tabulated by meal preparation method, program size, urbanicity, and region of location to identify characteristics associated with these funding transfers.

Although not directly relevant to the data collection for Phase 1, referenced here, Phase 2 of the evaluation will be connected to the Phase 1 information and will focus on information about program quality and processes will also be obtained from the “customer” side of the program. The research team will select a random sample of elderly clients who will be surveyed to learn about their key demographic, health, nutrition and lifestyle characteristics, the extent of their use of the program, and their levels of satisfaction with program services. The data obtained in this survey of participants will contribute to the analysis of the program by providing valuable information on program targeting (the program should target elderly people with the greatest economic or social need)¹ and an assessment of the efficiency of the program from the clients’ point of view. This Phase 2 data will describe ENSP participant satisfaction with congregate meals and home-delivered meals and related supportive services. The research team will ascertain congregate nutrition and home-delivered nutrition participants’ overall level of satisfaction with the nutrition program; what they like most and least about the program. The research team will also assess their perceptions about how the food tastes, smells, and looks, and the extent to which they are satisfied with food variety and meal sizes. Table A.16.3 illustrates how the research team will present findings on participants’ valuation of meals and supportive services received from the nutrition program. This includes the degree of difficulty in accessing the site and satisfaction and time spent in recreational and social activities (for congregate nutrition participants) and the helpfulness of referrals and other services, as well as the nutrition program.

Table A.16.3. Participants Valuation of Meals and Supportive Services Received From the Nutrition Program (Outcome Evaluation)

Characteristic	Congregate Nutrition Participants	Home-Delivered Nutrition Participants
Transportation
Ease of getting to the site		NA
Very easy
Somewhat easy
Not too easy
Not easy at all
Recreational and Social Activities
Satisfaction with opportunities to spend time with others		NA
Very satisfied
Somewhat satisfied
Not too satisfied
Not at all satisfied
Time spent participating in other activities or receiving other services at the meal site		NA
A lot of time
Some time
Just a little time
No time
Referrals and Other Services
Received information and/or referral services from nutrition program
Information and/or referral services from the program were:
Very helpful
Somewhat helpful
Not too helpful
Not at all helpful
Helpfulness of Program
Overall helpfulness of the nutrition program
Helped a lot
Helped somewhat
Helped a little
Didn’t help
Made things worse
The nutrition program has helped clients:
Eat healthier foods
Improve health
Follow a special diet
Achieve or maintain a healthy weight
Live independently and stay in own home

Source: Client survey.

1. Nutritional Quality of Program Meals (Phase 2)

The Older Americans Act (OAA) of 2006 requires that programs provide meals to participants that (1) provide a minimum of one third of the Dietary Reference Intakes (DRIs) if one meal per day is provided (OAA Sections 339 and 614),² and (2) comply with the most recent Dietary Guidelines for Americans. The DRIs provide the most up-to-date information on nutrient requirements and include reference values for men and women ages 51 to 70 years and over 70 years. The Dietary Guidelines provide recommendations to help individuals choose foods that comprise a healthy eating pattern—specifically, one that focuses on the consumption of nutrient-dense foods while staying within calories needs. Our assessment of the quality of program meals provided by the ENSP will focus on the extent to which meals offered or delivered to participants meet these Federal nutrition standards for the program. We will also examine the average nutrient and food group content as well as the types and variety of foods included in meals offered or delivered to participants.

These analyses of the nutritional quality of program meals will be based on data from the menu survey collected from LSP staff (described in Section II.A). The menu data will include detailed information on the foods and beverages offered to participants in congregate and home-delivered meals over a three-day period. The data will first be coded using USDA databases to obtain estimates of the nutrient and food group content of the individual foods offered (for example, grams of protein and cups of vegetables). We will then develop estimates of the nutrient and food group content of the average meal offered to participants. All of these analyses of the program meals data will be weighted descriptive tabulations and will include separate estimates for congregate meals, home-delivered meals, and all meals combined.

As mentioned, the first set of analyses will assess how well the meals conform to Federal nutrition standards for the program. We will determine the proportion of LSPs that offer meals that meet one-third of the DRI-based standards for men and women ages 51-70 and over 70 years. This analysis will focus on the nutrients that were identified by an Issue Panel convened by the National Resource Center on Nutrition, Physical Activity, and Aging for use in planning and evaluating meals (Silver et al. 2002). To examine compliance with the Dietary Guidelines, we will use the quantitative recommendations specified for saturated fat, cholesterol, and sodium, as well as one-third of the daily recommended amounts of food groups (for calorie levels appropriate for older adults) specified in the USDA Food Patterns. Table A.16.4 illustrates how the results will be presented for both meal types combined.

We will also tabulate the average nutrient and food group content of the meals based on the portions sizes of the foods and beverages offered. As an example, Table A.16.5 presents the average calorie and nutrient content of program meals. To address the last research question, we plan to assess the types and variety of foods offered by tabulating the frequency of various types of foods offered in program meals—for example, the frequency of fresh fruits and vegetables.

2. Health Status and Medical Care (Phase 2)

To examine the health status of participants, tabulations of general measures such as self-reported health status and body mass index, cigarette and alcohol usage, and mobility limitations will be presented (Table A.16.9). Other health characteristics will also be reported, such as the type and frequency of receipt of medical care over the prior year, including the number of emergency visits and hospital stays; time spent in a nursing home, convalescent home, or rehabilitation center; and the types of diagnoses that participants have received from doctors.

Table A.16.4. Proportion of LSPs Offering or Delivering Meals that Meet Standards: All Meals

		Proportion meeting standard on
	Standard/ Recommendation	One Day	Two Days	Three Days	Average
Calories (kcal)	685a
Macronutrients (% of total calories)b
Protein	10-35
Carbohydrate	45-65
Total Fat	20-35
Saturated Fat	< 10c
Vitamins
Vitamin A (ug)	300
Vitamin C (mg)t	30
Vitamin D (ug)	5
Vitamin E (mg)	5
Thiamin (mg)	0.4
Riboflavin (mg)	0.43
Vitamin B6 (mg)	0.57
Folate (ug)	133
Vitamin B12 (ug)	0.79
Minerals
Calcium (mg)	400
Copper (ug)	300
Iron (mg)	2.7
Magnesium (mg)	140
Potassium (mg)	1167
Sodium (mg)	< 500c
Zinc (mg)	3.7
Other Dietary Components
Cholesterol (mg)	< 100c
Dietary Fiber (g/1,000 calories)	14
Number of LSPs

Notes: The standards and recommendations included in the table are based on the Dietary Reference Intakes (DRIs) and Dietary Guidelines for Americans. The standards for all vitamins and minerals except for vitamin D and calcium are Recommended Dietary Allowances (RDAs). The standards for vitamin D and calcium are Adequate Intakes (AIs). The DRIs shown in this table are based on one-third of the highest values for ages 51 and older, males and females. Tables may also be prepared showing percentages meeting DRI standards separately for males and females, or meeting average DRIs for both groups.

^a Based on recommendations for a 75 year old male, height of 5’7’’, and a “low active” physical activity level.

^b The DRIs define Acceptable Macronutrient Distribution Ranges (AMDRs) for intakes of carbohydrate, protein, and fat as a percentage of total calories. The AMDRs reflect the ranges of intake that are associated with reduced risk of chronic disease while providing adequate amounts of essential nutrients.

^c Based on the Dietary Guidelines recommendation.

Table A.16.5. Average Calorie and Nutrient Content of Meals Offered or Delivered to Participants

	Congregate Meals	Home-Delivered Meals	All Meals
Calories (kcal)
Macronutrients (% of total calories)
Protein
Carbohydrate
Total Fat
Saturated Fat
Vitamins
Vitamin A (ug)
Vitamin C (mg)
Vitamin D (ug)
Vitamin E (mg)
Thiamin (mg)
Riboflavin (mg)
Vitamin B6 (mg)
Folate (ug)
Vitamin B12 (ug)
Minerals
Calcium (mg)
Copper (ug)
Iron (mg)
Magnesium (mg)
Potassium (mg)
Sodium (mg)
Zinc (mg)
Other Dietary Components
Cholesterol (mg)
Dietary Fiber (g)
Number of Meals
Number of LSPs

Note: Tables presenting the average food group content of meals offered or delivered to participants will also be prepared.

4. Program Outcomes and Effectiveness (Phase 2)

Two objectives of the evaluation of program outcomes and effectiveness are to identify the characteristics of the nutrition program participants as accurately as possible and to estimate the impact of participating in these programs on individuals’ nutrition, food security, socialization activities, and health. Different analytic methods will be used for each objective. Weighted descriptive tabulations of congregate and home-delivered nutrition program participants will be used to describe the demographic, economic, health, social, and nutrition characteristics, as well as the service use, of ENSP participants. These tabulations will allow us to determine whether the congregate and home-delivered nutrition programs are serving clients as intended and the extent to which the programs successfully targets priority subgroups of elderly individuals. In contrast, multivariate analysis will be used to estimate program impacts on a set of outcome measures. This analysis will compare program participants and eligible nonparticipants on selected outcomes, controlling for characteristics that could be related to both program participation and the outcomes studied. Below, both sets of analytic methods are described and example table shells are provided.

5. Participant Characteristics (Phase 2)

We will conduct tabular analysis that will describe the characteristics of nutrition program participants and identify key differences and similarities between congregate and home-delivered nutrition participants. This will include demographic and economic characteristics; health status; mobility; eating behavior, diet, and food preparation; food security; program participation experiences; and dietary quality. In the following paragraphs, selected tables serve as examples of the types of descriptive analyses that will be conducted.

6. Demographic and Economic Characteristics (Phase 2)

We will present demographic characteristics such as participants’ age, gender, highest grade level of schooling completed, race and ethnicity, marital status, and household size (Table A.16.8). The study will also examine participants’ sources of income, the distribution of income, and the extent to which participants are forced to choose between buying food and other expenses such as medications, utility bills, and housing payments (these characteristics are not shown in the table).

Table A.16.8. Selected Demographic Characteristics of Nutrition Program Participants

Characteristic	Congregate Nutrition Participants	Home-Delivered Nutrition Participants
Age
Less than 60
60 – 74
75 – 84
85 and older
Average Age (years)
Female
Highest grade level completed	V3	V3
5th grade or less
6th – 12th grade (no diploma)
High School Graduate, GED or equivalent
Some College (no degree)
Associate Degree, Occupational or Technical Degree
Bachelors Degree
Masters Degree or higher
Race and ethnicity	V5	V5
White non Hispanic
Black non Hispanic
Asian non Hispanic
American Indian non Hispanic
Other non Hispanic
Hispanic	V4	V4
Marital status	V6	V6
Married or living with partner
Widowed
Divorced
Separated
Never married
Number of people living in household	V8	V8
Live alone
1
2
3 or more

Source: Client Survey.

7. Health Status and Medical Care (Phase 2)

Health Status and Medical Care. To examine the health status of participants, tabulations of general measures such as self-reported health status and body mass index, cigarette and alcohol usage, and mobility limitations will be presented (Table A.16.9). Other health characteristics will also be reported, such as the type and frequency of receipt of medical care over the prior year, including the number of emergency visits and hospital stays; time spent in a nursing home, convalescent home, or rehabilitation center; and the types of diagnoses that participants have received from doctors.

Table A.16.9. General Health Status of Nutrition Program Participants

Characteristic	Congregate Nutrition Participants	Home-Delivered Nutrition Participants
General health
Excellent
Very good
Good
Fair
Poor
BMI
Below 18.5 (Underweight)
18.5 – 24.9 (Normal)
25.0 – 29.9 (Overweight)
30.0 and above (Obese)
Unintentional gain or loss of 10 pounds in past 6 months
Participated in physical activity in the past month
Number of days per week
Currently smokes cigarettes
Every day
Some days
Alcohol Consumption
Average number of days per week alcohol is consumed
Average number of alcoholic drinks when consumed
Has a clinic/doctors office for routine care or
Wears dentures
Blood pressure checked in the past 12 months
Fallen more than twice in the past 12 months
Mobility
Able to walk
Uses a cane or walker
Difficulty walking or climbing stairs
Bed bound
Average time bed bound
Chair bound or in a wheelchair
Average time chair bound or in a wheelchair

Source: Client Survey

8. Mobility (Phase 2)

To examine participants’ mobility, we will tabulate the percentage of congregate and home-delivered nutrition program participants that are chair bound or in a wheelchair, use a cane or walker, or have serious difficulty walking or climbing stairs. We will also examine the types of difficulties that participants may have doing certain activities such as shopping for personal items, using the telephone, preparing meals, taking medications, or taking a bath or shower.

9. Eating Behaviors, Diet, Food Preparation, and Food Security (Phase 2)

We will examine participants’ eating behaviors, diet, and food preparation, as well as their food security. For example, tables will present the percentage of participants that prepare their own meals or help someone else cook, as well as the types of special diets prepared by participants, such as diabetic, low sodium, low sugar, low far, low/high fiber, vegetarian, Lactose-free). We will also estimate the percentage of congregate and home-delivered nutrition participants that are food secure, food insecure with low food security, and food insecure with very low food security (Table A.16.10). These percentages will be based on a six-item, 30-day food security module.

Table A.16.10. Food Security Among Nutrition Program Participants

Food Security	Congregate Nutrition Participants	Home-Delivered Nutrition Participants
Food secure
Food insecure
Food insecure with low food security
Food insecure with very low food security

Source: Client survey.

10. Program Participation Experiences (Phase 2)

The study will describe program participation experiences of congregate and home-delivered nutrition program participants. This includes how long current participants have been in the program, how participants found out about the program or were referred to the program, their frequency of site attendance/receipt of home-delivered meals, and their experiences and attitudes about voluntary contributions for meals. An example table may present the distribution of the number of days per week a participant eats at a program site, the distribution of the number of days since the participants’ last visit to the nutrition program, how often they attend the program relative to six months ago, and other participation characteristics.

11. Dietary Quality (Phase 2)

The 24-hour dietary recall interviews include detailed descriptions of foods eaten, portions eaten, and the source of the food. A second day of dietary recalls will be collected for a portion of the original sample, to make possible estimation of the distributions of usual intakes of key nutrients. The data on each food will be coded so that information on the nutrients contained and the food group it belongs to can be analyzed.

We will use 24-hour recall data to assess the quality of diets consumed by congregate nutrition and home-delivered nutrition participants in two ways. First, we will analyze usual nutrient intakes relative to DRI standards. This will measure the adequacy of usual intakes of key nutrients and dietary components as well as measures of excessive intakes. Second, we will analyze overall diet quality and food group intakes using the Healthy Eating Index (HEI)-2005 (Guenther et al., 2007). The scores on the HEI-2005 provide a useful summary measure of diet quality relative to DGAs and My Plate recommendations for population groups.

12. Analysis of Usual Nutrient Intakes Relative to DRI Standards (Phase 2)

The DRIs are defined on the basis of usual daily intakes, conceptually the long-term average daily intakes of individuals. However, usual intakes can seldom, if ever, be directly observed. Although a single 24-hour recall provides information on an individual’s observed daily (24-hour) intake, it provides a very imprecise estimate of that individual’s usual intake, as well as an inaccurate estimate of the distribution of intake levels across a population group. This is because individuals’ dietary intakes vary from day to day. This source of variation, known as intra-individual variation, is typically even larger than variation from one individual to the next within a population (inter-individual variation). If daily intakes are used to estimate intake distributions, the dispersion of the distribution will be larger than the dispersion of usual intakes, and estimates of the proportion of individuals whose usual intake of a particular nutrient is above or below a specific reference standard will be biased (Beaton et al. 1979).

Thus, to apply the DRIs appropriately, it is necessary to have information about the distribution of usual intakes within population groups. We will use the empirical method recommended by the Institute of Medicine (IOM) for adjusting observed daily nutrient intakes to obtain unbiased estimates of the distribution of usual intakes for a group (Institute of Medicine 2000). The method was first developed by the National Research Council (National Research Council, Subcommittee on Criteria for Dietary Evaluation 1986) and later modified by Nusser et al. (1996). It estimates the intra-individual variation in nutrient intake, based on a subsample of individuals with two days of intake data, and removes this source of variation before estimating the distribution of usual nutrient intakes across a population. The method uses a specialized software package, the Software for Intake Distribution Estimation (SIDE), in conjunction with a single 24-hour recall for all sample members and a second 24-hour recall for a randomly selected subsample, to estimate usual intake distributions.

The nutritional adequacy of diets of individuals and population groups will be assessed by comparing usual daily intakes of energy and nutrients to the DRIs. The DRIs, developed by the IOM, are the most up-to-date scientific standards for determining the proportion of individuals who have inadequate or excessive intakes of specific nutrients or other food components. DRIs have been established both for vitamins and minerals and for energy, fats, carbohydrates, protein, and dietary fiber (IOM 2006); and different values are specified for subgroups based on age, gender, and life stage. Table A.16.11 shows the different types of reference values that are used, depending on the nutrient.³ Table A.16.12 compares the usual daily intakes of energy and nutrients to the DRIs for congregate nutrition program and home-delivered nutrition program participants. Nutrition program participants consume substantial proportions of their total daily intake of nutrients from meals from congregate and home-delivered nutrition programs on days when they either attend the congregate meal site or receive home-delivered meals. We will estimate the percentage of total daily dietary intake provided by the program meal for the same set of vitamins, minerals, and the other dietary components and nutrients shown in Table A.16.12 (sample table of these percentages is not shown).

Table A.16.11 Definitions of Dietary Reference Intakes (DRIs)

Estimated Average Requirement (EAR)	The EAR is the level of intake that is estimated to meet the requirements of half of the healthy individuals in a particular life stage and gender group. The EAR is used to assess the prevalence of inadequate intakes using the IOM-recommended “EAR-cutpoint method” (IOM 2006). The EAR cut-point method will be used to analyze all nutrients for which EARs have been established. The EAR cut-point method assumes that nutrient requirements are symmetrically distributed.
Adequate Intake (AI)	An AI was defined when the data available for a particular nutrient were insufficient to estimate requirements and establish an EAR. The AI is the level of intake that is assumed to be adequate, based on observed or experimentally determined estimates of intake. AI cannot be used to determine the proportion of a population with inadequate intakes. Instead, assessment focuses on comparison of mean usual intakes to the AI. Populations with a mean usual intake equivalent to or greater than the population-specific AI can be assumed to have adequate intakes.
Tolerable Upper Intake Level (UL)	The UL is the maximum level of intake that is likely to pose no risks of adverse health effects for all individuals in a population group. As intake increases above the UL, the risk of adverse effects increases. For most nutrients for which ULs have been established, the UL is based on intake from food, water, and dietary supplements (e.g., fluoride, phosphorus, and vitamin C) (IOM, 2006). For some nutrients, the UL applies only to synthetic forms from dietary supplements, fortified foods, or over-the-counter medications (e.g., magnesium, folate, niacin, and vitamin E). The client survey data files will not include nutrients provided by water, dietary supplements, or over-the-counter medications. Thus, our ability to assess usual intakes relative to ULs will be limited. The prevalence of intakes above the UL for nutrients for which a UL is available has been found using nutrient intake data from the NHANES to be very small with the exception of sodium and a handful of results for other nutrients (Cole and Fox 2008). For this reason, we plan to include analyses of intake relative to the UL only for sodium.
Acceptable Macronutrient Distribution Ranges (AMDRs)	The DRIs specify AMDRs for macronutrients (protein, carbohydrate, and total fat) and fatty acids (linoleic acid and alpha-linolenic acid).4 AMDRs define ranges of macronutrient intakes that are associated with reduced risk of chronic disease, while providing recommended intakes of other essential nutrients. AMDRs are expressed as percentages of total energy intake because their requirements are not independent of each other or of the total energy requirement of the individual (IOM 2006). A key feature of AMDRs is that each has lower and upper bounds. Intakes that fall below or exceed these levels of intake may increase risk of chronic disease.

Table A.16.12. Usual Daily Intakes of Congregate Nutrition Program and Home-Delivered Nutrition Program Participants

	Congregate Nutrition Program Participants	Home-Delivered Nutrition Program Participants
Vitamins and Minerals with EARS (Percentage Less Than EAR)
Vitamin A
Vitamin C
Vitamin E
Vitamin B6
Vitamin B12
Folate
Niacin
Riboflavin
Thiamin
Iron
Magnesium
Phosphorus
Zinc
Minerals (Mean as Percentage of AI)
Calcium
Potassium
Sodium
Minerals (Percentage Greater than UL)
Sodium
Other Dietary Components
Fiber (mean as % of AI)
Fiber (mean g/1000 calories)
Cholesterol (% >DGA)
Energy (Mean)
Energy
Macronutrients
Total Fat
% < AMDR
% > AMDR
Saturated Fat
% > than DGA
Carbohydrate
% <EAR
% <AMDR
% >AMDR
Protein
% < EAR
% < AMDR
% > AMDR
Linoleic Acid
% < AMDR
% < AMDR

Source: Client Survey

AMDR=Acceptable Macronutrient Distribution Range, DGA=Dietary Guidelines for Americans recommendation, EAR=Estimated Average Requirement, AI=Adequate Intake, UL=Tolerable Upper Intake Level.

13. Analysis of Healthy Eating Index (HEI)-2005 Scores (Phase 2)

The HEI-2005 is an updated version of the index originally developed by USDA’s Center for Nutrition Policy and Promotion (CNPP) in 1995 (Kennedy et al., 1995). The HEI-2005 was developed by a federal interagency workgroup led by CNPP. The index is designed to measure how well individuals’ diets conform to the 2005 Dietary Guidelines, including specific factors that influence diet quality, such as consumption of whole grains, specific types of fat, particular types of nutrient-dense vegetables, and “discretionary calories.” The HEI (in its original and updated form) has been adopted by USDA as the tool used to monitor diet quality of the U.S. population overall as well as progress toward healthier eating habits among food assistance program participants (USDA/FNS, 2000; Basiotis et al., 2002; USDA, 2006). The index includes 12 component scores—nine components assess intake of food groups (total fruit (including juice), whole fruit, total vegetables, dark green and orange vegetables and legumes, total grains, whole grains, milk, meat, and healthy oils), two components assess dietary components that are commonly consumed in excess (saturated fat and sodium), and one component assesses intake of discretionary calories from solid fat, alcohol, and added sugars (Guenther et al., 2007). Scores are assigned for each component based on reference standards that reflect Dietary Guidelines and MyPlate recommendations. Maximum scores for each component range from 5 to 20, with an overall maximum score of 100. HEI-2005 developers recommend that researchers focus on individual component scores rather than a total composite score because the individual scores provide the most useful data on shortcomings in diet quality.

We will estimate HEI-2005 scores for congregate nutrition and home-delivered nutrition program participants following the approach recommended by the interagency group that developed the measure (Table A.16.13). Ideally, the HEI-2005 would be calculated based on the usual dietary intake of each individual. As noted in the preceding discussion of usual nutrient intakes, with only one day of intake data for each sample member, we will not have a reliable estimate of each individual’s usual intake. However, a good estimate of a population’s mean usual intake is the mean of one-day intakes; and the best estimate of the population’s mean HEI scores is based on estimates of total intakes at the population level (Guenther et al., 2007; Freedman et al., 2008). Thus, we will assign HEI-2005 scores at the group rather than individual level, using the single 24-hour recall collected from all congregate nutrition and home-delivered nutrition participants.

Table A.16.13. Mean Healthy Eating Index-2005 Scores for Participants in Congregate Nutrition Programs

	Max. Score	Congregate Nutrition Program Participants’ Mean HEI-2005 Score	Congregate Nutrition Program Participants’ Mean HEI-2005 Score as a Percentage of Maximum Scores	Home-Delivered Nutrition Program Participants’ Mean HEI-2005 Score	Home-Delivered Nutrition Program Participants’ Mean HEI-2005 Score as a Percentage of Maximum Scores
Total Fruit (includes 100% juice)	5.0
Whole Fruit (not Juice)	5.0
Total Vegetables	5.0
Dark Green and Orange Vegetables and Legumesa	5.0
Total Grains	5.0
Whole Grains	5.0
Milkb	10.0
Meat and Beans	10.0
Oilsc	10.0
Saturated Fat	10.0
Sodium	10.0
Calories from SOFAAS	20.0
Total HEI-2005 Score	100.0

Source: Client survey.

SOFAAS = Solid Fats, Alcoholic beverages, and Added Sugars

^aLegumes counted as vegetables only after Meat and Beans standard is met.

^bIncludes all milk products, such as fluid milk, yogurt, cheese, and soy beverages.

^CInlcudes nonhydrogenated vegetable oils and oils in fish, nuts, and seeds.

A.16.2. Program Impacts on Client Outcomes (Phase 2)

The analyses described thus far focus on the populations served by congregate nutrition and home-delivered nutrition programs by examining characteristics only of program participants. To assess program outcomes and effectiveness, however, we will use data from nutrition program participants and nonparticipants. The study will compare observed outcomes for a sample of program participants and a set of matched comparison observations, elderly people who are as similar as possible to the participant sample but do not participate in the Title III-C nutrition program. Impacts will be estimated on nutrition, food security, socialization, and health and institutionalization outcomes. Except for the nutrition outcomes (explained below), all analyses will use multivariate regression methods, with control variables reflecting client characteristics. Analyses will be conducted separately for (1) congregate nutrition program participants and nonparticipants and (2) home-delivered nutrition program participants and nonparticipants.

1. Nutrition Outcomes

It is not possible to use multivariate regression to estimate the impact of ENSP participation on the usual intakes of key nutrients because estimates of usual intakes are only available for a group of individuals, and not for each individual in the sample. To estimate differences between participant and nonparticipant groups in proportions with inadequate nutrient intakes, we will rely on the matching process used to identify appropriate nonparticipants for the sample, and estimate the prevalence of inadequate intakes for participants and separately for matched nonparticipants (Mabli et al. 2010). SIDE can also be used to estimate standard errors of these statistics that account for sample clustering. Table A.16.14 is an example of a table examining differences in usual intakes across matched participant-nonparticipant groups. The estimates will be presented by nutrition program type and, if sample sizes permit, by respondents’ age and gender.

Because the HEI-2005 scores will be also assigned at the group rather than individual level, comparisons of HEI-2005 component scores for nutrition program participants and nonparticipants will be made using the matched samples. Like the analysis of the impact on the usual intake of key nutrients, the matching process used to estimate the impact on HEI-2005 scores will include those factors typically included in a regression analysis to account for cross-group differences that are correlated with both the nutrition outcome measure and ENSP participation. Table A.16.15 compares HEI-2005 total scores and component scores across congregate nutrition and home-delivered nutrition participants and nonparticipants.

2. Food Security

The impact of ENSP participation on food security will be estimated within a multivariate regression framework that accounts for compositional differences across participant-comparison groups that might bias the impact estimates. We will use a logistic regression model that relates the probability of an individual’s being food insecure to a variable indicating whether the individual participates in the ENSP program and to a set of individual characteristics. The set of characteristics will include the individual’s gender, race and ethnicity, age, income, region of residence, and indicators for mobility limitations. It will also include variables that describe the composition of the individual’s family, such as whether he or she lives with other family members. Based on descriptive comparisons of participants and nonparticipants, we will assess whether to include variables measuring self-reported health status and other indicators of health status such as whether the individual has hypertension, high blood cholesterol, diabetes, or has had a stroke; whether the person takes vitamin supplements; whether the person has done any exercise, sports, or physical activity in the past 30 days (and how many times per week).

We will present the results of the food security analysis in several ways, using detailed tables and summary tables. First, we will present a table with the regression coefficients and standard errors (Table A.16.16) to help the reader understand what variables are used in the regression and how these results translate into the subsequent set of summary tables. Next, we will present regression-adjusted tables of program impact estimates that resemble the descriptive tables presented earlier (Table A.16.17). This table compares the rates of food insecurity across congregate nutrition program participant groups after accounting or adjusting for compositional differences across groups. A similar table will be produced for home-delivered nutrition program participation. Because we will use logistic regression analysis, the procedure for obtaining the regression-adjusted estimates consists of estimating the regression, using the regression coefficients and variable values for each individual in the sample to obtain a predicted probability of being food insecure, and averaging the predicted probabilities to obtain the adjusted (predicted) rate of food insecurity in the sample. By performing these steps assuming all sample members are participants and then repeating the procedure assuming all sample members are nonparticipants, we obtain two averaged values. The difference between these values is the regression-adjusted estimate of the impact of program participation on food insecurity.

Table A.16.14. Effects of Congregate Nutrition and Home-Delivered Nutrition Programs on Usual Daily Intakes by Participation Status, Age, and Gender

	Congregate Nutrition Participation												Home-Delivered Nutrition Participation
	Older Adult Males (60+)			Older Adult Females (60+)			Older Adults (60+)						Older Adult Males (60+)						Older Adult Females (60+)					Older Adults (60+)
	Part-icipant	Non-part-icipant		Part-icipant	Non-part-icipant		Part-icipant		Non-part-icipant				Part-icipant		Non-part-icipant				Part-icipant		Non-part-icipant			Part-icipant			Non-part-icipant
Vitamins and Minerals with EARS (Percentage Less Than EAR)
Vitamin A
Vitamin C
Vitamin E
Vitamin B6
Vitamin B12
Folate
Niacin
Riboflavin
Thiamin
Iron
Magnesium
Phosphorus
Zinc
Minerals (Mean as Percentage of AI)
Calcium
Potassium
Sodium
Minerals (Percentage Greater than UL)
Sodium
Other Dietary Components
Fiber (mean as % of AI)
Fiber (mean g/1000 calories)
Cholesterol (% >DGA)
Energy (Mean)
Energy
Macronutrients
Total Fat
% < AMDR
% > AMDR
Saturated Fat
% > than DGA
Carbohydrate
% <EAR
% <AMDR
% >AMDR
Protein
% < EAR
% < AMDR
% > AMDR
Linoleic Acid
% < AMDR
% < AMDR
Linolenic Acid
% < AMDR
% > AMDR

Source: Client Survey

Notes: Weighted tabulations based on first and second 24-hour recalls prepared by Mathematica Policy Research. Usual intake distributions were determined for each subgroup using the Software for Intake Distribution Estimation (SIDE).

Matched sample constructed using propensity score matching to adjust for differences in economic and demographic characteristics, including age, gender, race and ethnicity, marital status, household size, and income relative to poverty. Estimates weighted to account for sample design.

AMDR=Acceptable Macronutrient Distribution Range

DGA=Dietary Guidelines for Americans recommendation

EAR=Estimated Average Requirement

AI=Adequate Intake

UL=Tolerable Upper Intake Level

* or ** denotes difference between participant and nonparticipant group is statistically significant at the 0.05 or 0.01 level of significance.

Table A.16.15. Effects of Congregate Nutrition Program and Home-Delivered Nutrition Program Participation on Mean HEI-2005 Scores

		Congregate Nutrition Program			Home-Delivered Nutrition Program
Component	Maximum Possible Score	Participant	Non-participant	Difference	Participant	Non-participant	Difference
Total Fruit
Whole Fruit (not juice)	5
Total Vegetables	5
Dark Green and Orange Vegetables and Legumes	5
Total Grains	5
Whole Grains	5
Milk	10
Meat and Beans	10
Oils	10
Saturated Fat	10
Sodium	10
Calories from Solid Fat, Alcohol, and Added Sugar (SoFAAS)	20
Total Score	100

Source: Client Survey

Notes: Weighted tabulations based on first and second 24-hour recalls prepared by Mathematica Policy Research.

Matched sample constructed using propensity score matching to adjust for differences in economic and demographic characteristics, including age, gender, race and ethnicity, marital status, household size, and income relative to poverty, and other characteristics. Estimates weighted to account for sample design.

Table A.16.16. Regression Coefficients of the Effects of Congregate Nutrition Program and Home-Delivered Nutrition Program Participation and Individual Characteristics on an Individual’s Likelihood of being Food Insecure

	Coefficient	Standard Error
Congregate Nutrition Program Participation
Home-Delivered Nutrition Program Participation
Gender (male is referent group)
Female
Race and Ethnicity (non-Hispanic white is referent group)
Non-Hispanic black
Non-Hispanic other
Hispanic
Age (60 to 65 is referent group)
66-70
71-75
75-80
80 and older
Employed full time
Employed part time
Unemployed
Monthly Income as a Percentage of Poverty (less than 50% is referent group)
50% to 100%
101% to 200%
201% to 300%
Greater than 300%
Household Contains Children (referent group is no children)
Household Contains Other Elderly (referent group is no other elderly)
Household Size (One household member is referent group)
Two
Three
Four or more
Region of Residence (western region is referent group)
Northeast
Mid-Atlantic
Midwest
Southeast
Southwest
Mountain Plains
Mobility
Able to walk
Uses a cane or walker
Difficulty walking or climbing stairs
Bed Bound
Chair bound or in a wheelchair

Source: Client survey.

Table A.16.18. Regression-Adjusted Percentages of Individuals That Are Food Insecure, by Congregate Nutrition Program Participation Status

	Congregate Nutrition Program Participant	Congregate Nutrition Program Nonparticipant	Difference
All Individuals
Gender
Male
Female
Race and Ethnicity
Non-Hispanic white
Non-Hispanic black
Non-Hispanic other
Hispanic
Age
60 to 65
66-70
71-75
75-80
80 and older
Employment Status
Employed full time
Employed part time
Unemployed
Out of the labor force
Monthly Income as a Percentage of Poverty
Less than 50%
50% to 100%
101% to 200%
201% to 300%
Greater than 300%
Household Contains Children
Household Does Not Contain Children
Household Contains Other Elderly
Household Does Not Contain Other Elderly
Household Size
One
Two
Three
Four or more
Region of Residence
Northeast
Mid-Atlantic
Midwest
Southeast
Southwest
Mountain Plains
Western
Mobility
Able to walk
Uses a cane or walker
Difficulty walking or climbing stairs
Bed Bound
Chair bound or in a wheelchair

Source: Client survey

3. Socialization Opportunities (Phase 2)

Socialization will be measured using responses to questions about how satisfied individuals are with the opportunities they have had to spend time with other people; and how often they felt that they lack companionship, felt left out, or felt isolated from others. Like the analysis of food security, we will estimate a logistic regression that relates the probability of an individual experiencing social isolation to a variable indicating whether the individual participates in the ENSP program and to a set of individual characteristics. For social isolation questions with response options “hardly ever; some of the time; or often”, we will estimate the probability that an individual experiences these events “often” compared to experiencing them “hardly ever” or “some of the time”. We will examine the sensitivity of the results to the definition of the dependent variable by grouping “often” and “some of the time” in an auxiliary model. The set of explanatory variables, besides congregate or home-delivered nutrition program participation status, likely will be similar to those used in the food security analyses. We will present the results of the regressions by presenting tables of regression-adjusted probabilities of experiencing social isolation, similar in presentation to the regression-adjusted food security tables above. Tables containing the full set of regression coefficients and standard errors will be available in an appendix.

4. Health and Institutionalization Outcomes (Phase 2)

To estimate the impact of program participation on participants’ health and institutionalization, we will link the survey data on elderly participants (clients) and non-participants to data from Medicare claims records using client social security numbers. The claims data will be obtained for at least 12 months prior to the interview date and for 6 months after the interview date for both baseline nutrition program participants and nonparticipants. The data will contain information used to construct the health and institutionalization outcome measures, including whether the individuals has been hospitalized, number of hospitalizations, presence and number of 21 chronic conditions the beneficiary was treated for in the prior one to three years, whether had home health care, number of months received home health care, whether had skilled nursing facility (SNF) care, and total Medicare costs.

As described earlier, when evaluating the program impacts on elderly nutrition and food security, we will use dietary data from a 24-hour recall period and the food security information from a 30-day recall period. Both of these recall periods are measured almost concurrently with the respondent’s nutrition program participation status. To evaluate the program impacts on more long-run outcomes such as health and institutionalization, however, we will use client survey data on the history of program participation prior to the baseline interview date and the short-term history of program participation status prior to the follow up interview date. Using nutrition program participation histories at the baseline and follow up interviews will improve our ability to distinguish between (1) whether participating in nutrition programs leads to improved health and (2) whether deteriorating health precedes the start of nutrition program participation. That is, the program participation histories will be helping in estimating the program impact on the longer-term outcomes of health and institutionalization. Variables characterizing these histories will be included in the regression model used to estimate the program impacts. Similar to the food security and socialization tables, impacts on health and institutionalization will be presented using regression-adjusted probabilities of having home health care or being treated for a chronic condition in the prior one to three years and so on.

Compared to the evaluation of program impacts on nutrition, food security, and socialization, the evaluation of impacts on health and institutionalization should be viewed as more exploratory and less definitive given that longer-term outcomes will be evaluated using data from a relatively short period of time. Study findings will likely contain language that emphasizes this is exploratory work, discusses the limitations of the empirical approach, and places the research findings in the larger literature of related research on longer-term health outcomes.

A.16.3. Study Schedule

The schedule shown in Exhibit A.16.19 lists the expected start date for the data collection and reporting. Our data collection plans are designed to provide timely data for the evaluation reports.

Exhibit A16.19: Approximate Schedule of Tasks and Deliverables

Objectives/Major Tasks	Start Date (Week of)
Award Date	9/10/12
Work Plan and Project Communications	9/10/12
Refinement of Data Collection Tools	9/17/2012
Data Use Request Packets for Access to Medicare and Medicaid Files	9/17/2012
Sample Development and Selection	1/14/2013
Hire and Train Staff for Process and Cost Studies	1/7/2013
Conduct SUA Process Survey	11/22/2013
Conduct AAA and LSP Process Surveys	12/1/2013
Conduct Cost Study	12/29/2013
Perform the Title III-C Elderly Nutrition Process and Cost Study Data Analysis	6/17/2014
Prepare and Submit Executive Summary, Methods, and Results and Findings Report	10/21/2014
The remaining task will be conducted under a separate Phase 2 contract awarded September 2013
Develop and Draw Client Outcome Sample	TBD
Hire and Train CAPI Data Collectors for the Client Outcomes Study	TBD
Client and Comparison Group Data Collection	TBD
Data Processing, Merging with Medicare and Medicaid Files	TBD
Perform the Title III-C Elderly Nutrition Program Evaluation Data Analysis	TBD

A.17. Reason(s) Display of OMB Expiration Date is Inappropriate

ACL does not seek this exemption. All data collection instruments for the ACL Evaluation of the Title III-C Program will display the OMB approval number and expiration date.

A.18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the Certification for Paperwork Reduction Act (5 CFR 1320.9) for this study.

References

Basiotis P., Carlson A., Gerrior S., Juan W. Y., Lino M. The Healthy Eating Index: 1999-2000. Alexandria, VA: U. S. Department of Agriculture, Center for Nutrition Policy and Promotion. Report CNPP-12; 2002.

Beaton, G.H., Milner, J., Corey, P., McGuire, V., Cousins, M., Stewart, E., de Ramos, M., Little, J.A. Sources of variance in 24-hour dietary recall data: Implications For Nutrition Study Design and Interpretation. The American Journal of Clinical Nutrition (1979). Vol 32, Issue 12, Ogs 2546-2559

Freedman L.S., Guenther P.M., Krebs-Smith S., Kott P.S. “A population’s mean Healthy Eating Index-2005 scores are best estimated by the score of the population ratio when one 24-hour recall is available,” J Nutr. 138: 1725-29; 2008.

Guenther, P.M., Reedy, J., Krebs-Smith, Reeve, B.B., & Basiotis, P.P. (2007). Development and Evaluation of the Healthy Eating Index: Technical Report. Center for Nutrition Policy and Promotion, US Department of Agriculture.

Institute of Medicine (2000). Dietary Assessment Intakes: Applications in Dietary Assessment. Washington, DC: National Academies Press.

Kennedy E.T., Ohls J., Carlson S., Fleming, K. “The Healthy Eating Index: Design and applications.” J Am Diet Assoc. 95(10):1103-08; 1995.

National Research Council, Subcommittee on Criteria for Dietary Evaluation. “Nutrient Adequacy: Assessment Using Food Consumption Surveys.” Washington DC: National Academies Press, 1986.

Nusser, S.M., A.L. Carriquiry, K.W. Dodd, and W.A. Fuller. “A Semiparametric Transformation Approach to Estimating Usual Daily Intake Distributions.” Journal of the American Statistical Association, vol. 91, 1996, pp. 1440–1449.

Ponza, M., Ohls, J., Millen, B. E., McCool, A.M., Needels, K.E., Rosenberg, L., Chu, D. Quatromonic, P.A. 1996. Serving Elders At Risk: The Older Americans Act Nutrition Programs: National Evaluation of the Elderly Nutrition Program. Report to the U.S. Department of Health and Human Services, Assistant Secretary for Aging. Washington, DC: Mathematica Policy Research, July.

U. S. Department of Agriculture, Food and Nutrition Service. Food and Nutrition Service (FNS) Strategic Plan 2000-2005. Alexandria, VA; 2000.

U. S. Department of Agriculture. Strategic Plan for FY 2005-2010. Washington, D.C.; 2006.

1 The analyses of participant characteristics that form the basis of the targeting analyses are described in Section 3.

2 Meals must provide a minimum of two thirds of the DRIs if two meals per day are provided by the program and 100 percent of the DRIs if three meals per day are provided.

3 The DRIs also provide estimated energy requirements (EERs), which are based on age, gender, median height and weight, and level of physical activity. The actual energy requirement for an individual varies considerably with body size and activity level, which is difficult to measure accurately. Thus, for this study, we will present results for mean usual energy intakes rather than assessing intake relative to EERs.

4 Usual carbohydrate intakes are also assessed relative to EARs, based on total intake (gm/day).

Page 3 of 31