0023 NSOAAP SS Final w_30-day resp

Application for Data Collection:

Supporting Statement for

15^th National Survey of

Older Americans Act Participants

Updated

April 1, 2021

Table of Contents

Chapter Page

A. Justification 1-1

A.1 Circumstances Making the Collection of Information Necessary 1-1

A.2 Purpose and Use of the Information Collection 1-5

A.3 Use of Improved Information Technology and Burden Reduction 1-5

A.4 Efforts to Identify Duplication and Use of Similar Information 1-7

A.5 Impact on Small Businesses or Other Small Entities 1-8

A.6 Consequences of Collecting the Information Less Frequently 1-8

A.7 Special Circumstances 1-8

A.8 Comments in Response to the Federal Register Notice and

Efforts to Consult Outside the Agency 1-8

A.9 Explanation of Any Payments or Gift to Respondents 1-13

A.10 Assurances of Privacy Provided to Respondents 1-13

A.11 Justification for Sensitive Questions 1-15

A.12 Estimates of Annualized Burden Hours and Costs 1-16

A.13 Estimates of Other Total Annual Cost Burden to

Respondents and Record Keepers 1-17

A.14 Annualized Cost to the Federal Government 1-17

A.15 Explanation for Program Changes or Adjustments 1-17

A.16 Plans for Tabulation and Publication and Project Time

Schedule 1-18

A.17 Reason(s) Display of OMB Expiration Date is Inappropriate 1-19

A.18 Exception to Certification for Paperwork Reducation

Act Submissions 1-19

B. Collection of Information Employing Statistical

Methods 2-1

B.1 Respondent Universe and Sampling Methods 2-1

B.2 Procedures for the Collection of Information 2-5

B.3 Methods to Maximize Response Rates and Deal with

Nonresponse 2-13

B.4 Test of Procedures or Methods to be Undertaken 2-14

B.5 Individuals Consulted on Statistical Aspects and Individuals

Collecting and/or Analyzing Data 2-15

Table of Contents
(continued)

Exhibits Page

A-1 Estimated Hour and Annual Cost Response Burden 1-16

A-2 Total Annualized Cost to the Federal Government 1-17

A-3 Data Collection Timetable 1-18

B-1 Respondent Universe 2-3

Tables Page

A-1 Federal Register Comments and ACL Responses 1-9

B-1 Half-widths of 95 percent confidence intervals by various sample sizes

and estimates of target characteristics 2-8

B-2 Half-widths of 95 percent confidence intervals for the difference

between two estimates by various sample sizes and for various

averages of the two estimates 2-10

Appendices (separate document)

A Pertinent Legislation

B Instructions for Creating Numbered Client Lists for Sampling

C Instructions for AAA Access to the Survey Website & How to Submit Data

D Federal Register Notice Published by ACL/AoA for the Proposed

Information Collection

E Westat Assurance of Confidentiality Agreement

F Notification Letter for Selected Clients

G Notification Letter for State Units on Aging

H Notification Letter for Area Agencies on Aging

I Agency Information Packet

J Survey Instrument/Questionnaire [see separate document]

K COVID Module for 2021 [see separate document]

A. Justification

A.1 Circumstances Making the Collection of Information Necessary

Introduction

This OMB package requests clearance to conduct an annual cross sectional survey of Older Americans Act (OAA) participants. The survey will be the 15th in a series of national surveys of OAA clients. The first 14 surveys provided important cross-sectional data on service recipients (e.g., consumer assessment of services, reported outcomes, physical functioning, quality of life, and demographic information). The survey will continue to provide rich cross-sectional data.

This survey has remained essentially the same since the last OMB action on 8/20/2019 (OMB Control No: 0985-0023). The sampling methodology is been updated to be more precise, modifying the Stage I sampling design to utilize agency client counts rather than budget size. The data collection procedures are identical to the previous survey approved in 2017. The survey questionnaire is unchanged.

The 15th survey originally scheduled to be conducted in 2020 was postponed due to the COVID-19 pandemic and disruptions in service delivery provided by the aging network. As part of a nationwide effort to slow transmission of COVID-19, communities have closed locations such as senior centers, adult day care centers, and congregate meal facilities where many older adults received vital home- and community-based services. In addition, many of the in-home services that help older adults to remain in their homes, such as home-delivered meals, homemaker services, and respite care have been curtailed due to fear of spreading COVID-19.

This, combined with families and friends unable to assist loved ones due to the fear of spreading the virus, have left many older adults isolated and without the programs and services they need to remain living safely in the community. Consequently, ACL is requesting the addition of a one-time module on COVID-19. This module supplants the module on Emergency Preparedness that was included in the 60-day FRN due to the ongoing health crisis older adults are experiencing as a result of COVID-19. The Emergency Preparedness module will be added to the 2022 collection instrument. The purpose of adding questions on COVID-19 to the NSOAAP is to measure the effect the pandemic has had on older adults’ access to and use of Older Americans Act programs and services during the past year.

ACL/AoA’s Strategy of Program Improvement

The Administration for Community Living’s Administration on Aging (ACL/AoA) has an ongoing strategy of program improvement through enhanced program performance measurement, in compliance with requirements of the Office of Management and Budget’s (OMB) program reviews, the GPRA Modernization Act of 2010 (GPRAMA), and the OAA Section 202(f), by proposing to conduct further studies of program outcomes (see Appendix A for the pertinent legislation).

Previously, ACL/AoA conducted 14 cross-sectional surveys. The 14 surveys and their OMB control numbers are listed below:

• Two pilot studies of Older Americans Act Title III Service Recipients in 2003 and 2004 (OMB control numbers 0985-0014 and 0985-0017);

• Third National Survey of OAA Title III Service Recipients conducted in 2005 (OMB control number 0985-0020);

• Fourth National Survey of OAA Title III Service Recipients conducted in 2008 (OMB control number 0985-0023);

• Fifth National Survey of OAA Title III Service Recipients conducted in 2009 (OMB control number 0985-0023).

• Sixth National Survey of OAA Title III Service Recipients conducted in 2011 (OMB control number 0985-0023).

• Seventh National Survey of Older Americans Act Participants conducted in 2012 (OMB control number 0985-0023).

• Eighth National Survey of Older Americans Act Participants conducted in 2013 (OMB control number 0985-0023).

• Ninth National Survey of Older Americans Act Participants conducted in 2014 (OMB control number 0985-0023).

• Tenth National Survey of Older Americans Act Participants conducted in 2015 (OMB control number 0985-0023).

• Eleventh National Survey of Older Americans Act Participants conducted in 2016 (OMB control number 0985-0023).

• Twelfth National Survey of Older Americans Act Participants conducted in 2017 (OMB control number 0985-0023).

• Thirteenth National Survey of Older Americans Act Participants conducted in 2018 (OMB control number 0985-0023).

• Fourteenth National Survey of Older Americans Act Participants conducted in 2019 (OMB control number 0985-0023).

• The Fifteenth National Survey of Older Americans Act Participants was scheduled to be conducted in 2020, but it had to be postponed until 2021 due to the coronavirus pandemic and the resulting disruption in delivery of services to OAA recipients.

The surveys have enabled ACL/AoA to establish baselines and performance targets for annual and long-term outcome measures required by OMB and incorporate new performance information in agency budget justifications and performance plans. Further, the studies demonstrated that services provided under Title III:

• Are effectively targeted to vulnerable populations

• Are provided to individuals who need the services

• Are highly rated by recipients (quality)

• Provide assistance that is instrumental in enabling recipients to maintain their independence.

With this submission, we are requesting OMB approval to conduct a fifteenth national survey, as well as sixteenth and seventeenth surveys if approved for three years.

Performance Measurement Requirements

GPRAMA¹ requires federal agencies to develop annual and long-term performance outcome measures and to report on these measures annually. Section 203(f) of the OAA² requires ACL/AoA to work collaboratively with State agencies and area agencies on aging (AAAs) to develop performance outcome measures.

Since the passage of GPRA in 1993, ACL/AoA has accepted GPRA and GPRAMA as an opportunity to document each year the results that are produced through the programs it administers under the authority of OAA. It is the intent and commitment of ACL/AoA, in concert with State and local program partners, to use the performance measurement tools of GPRAMA to continuously improve OAA programs and services for older adults.

As described on ACL’s website: “The National Surveys of OAA Participants are a collection of annual national surveys of recipients of select Title III services. Their purpose is to obtain performance outcome measurement information, which is then used in AoA’s GPRA plan and PART assessment. The survey instruments focus on consumer assessment of service quality and consumer-reported outcomes. The instruments also measure special needs characteristics such as physical and social functioning of the people who receive services.” ³

OAA, Title III – Home and Community-Based Program

Title III of the OAA establishes a home and community-based care program for older persons and their caregivers, to enable them to live as independently as possible for as long as possible. States and local agencies are given much latitude to design services tailored to the needs of their regions and communities. One challenge for ACL/AoA is to devise a means to improve the performance of the program nationally, while preserving and promoting the diversity of program design. ACL/AoA has chosen to work toward improved program performance throughout the Aging Services Network by working collaboratively with States and AAAs to develop performance outcome measurement tools. The tools identify elements of service quality so that states and AAAs can improve service systems at the local level. These same tools can also be employed by ACL/AoA to measure program performance at the national level.

Performance Outcomes Measures Project (POMP)

For over 10 years, ACL/AoA sponsored the Performance Outcomes Measures Project (POMP) demonstration, in which grants were awarded to states, who then worked collaboratively to develop survey instruments that measured elements of service quality and consumer reported outcomes for various services provided under Title III of the OAA. Surveys were developed for the following topics:

Service Domains:

• Nutrition (including congregate and home-delivered meals)

• Transportation

• Information and Assistance

• Homemaker/Housekeeper

• Personal Care

• Caregiver Support

• Case Management

• Senior Centers

Client Characteristics:

• Physical Functioning

• Demographics

• Emotional Well-Being

• Social Functioning

POMP demonstrated the ability of states and AAAs to apply statistically sound sampling techniques to obtain numeric measures of program performance.

The survey instruments developed under POMP – along with various tools necessary for implementation – can be found at https://acl.gov/programs/pomp. These performance measurement surveys have enabled some local agencies to obtain additional financial support and improve program management. Examples of uses of performance measurement at the state and local level follow:

• The Hawkeye Valley Area Agency on Aging in Waterloo, Iowa compiled information on the level of client support and satisfaction with services and received additional funding from the United Way for exemplary programs.

• The Area Agency on Aging in Cincinnati, Ohio expanded the use of Home Care Client Satisfaction Measure (HCSM) and incorporated it into an ongoing part of its case management process for all clients to improve service quality.

• The Florida Department of Elder Affairs developed a computer simulation model that demonstrated the impact of home care programs on reducing nursing home admissions and showed the savings in Medicaid funds

The recipient and caregiver survey instruments national survey are based on the recipient surveys developed by POMP grantees.

A.2 Purpose and Use of the Information Collection

The results of this information collection will be used to:

• Report on annual performance results as required by OMB.

• Provide national benchmarks for use by states and AAAs.

• Provide secondary data for analysis of various Title III program evaluations.

• Provide performance information for key demographic subgroups, geographical subregions, and different types of AAAs which will enable ACL/AoA to identify variations in performance and examine the need for additional targeted technical assistance.

The data will be used by the Administrator of the Administration for Community Living/Assistant Secretary for Aging in testimony and presentations; it will be incorporated into the agency’s Annual Report; and it will be used by program staff to identify areas that may need attention at the national level. For example, the ACL nutritionist is interested in examining nutritional intake information by key population subgroups to identify potential areas for technical assistance initiatives.

A.3 Use of Improved Information Technology & Burden Reduction

Use of Client Tracking Software to Generate Client Lists for Sampling

The proposed procedures and materials requesting information from the agencies, as well as the telephone surveys of respondents, have been designed in a way that minimizes respondent burden.

To reduce the burden for the Area Agencies on Aging (AAAs), the contractor (Westat) has developed procedures for client sampling that utilize the same client tracking management information systems that are used by States and AAAs to create the required State Program Reports for ACL/AoA. Since the implementation of the fourth national survey in 2008, the contractor (Westat) has worked cooperatively with vendors of commercial off-the-shelf client tracking software programs most commonly used by the State and Area Agencies on Aging to develop step-by-step instructions for the AAAs to use to generate client lists by service to use for a sample frame. It is estimated that over 95% of the AAAs now have this technological capability and are able to follow the instructions to produce their client lists by service. We will provide similar instructions for the 15th survey. Appendix B contains an example of instructions created for agencies which use a commercial client tracking software system known as “PeerPlace.”

In specific states that have their own proprietary client tracking software, the contractor (Westat) has worked directly with an Information Technology Specialist at the State-level to generate electronic client lists for all of the AAAs selected for the national survey. This further reduces the burden for AAAs in states that have their own proprietary software.

Use of Survey Web Site

A National Survey web site application https://aoasurvey.org has been developed to support and assist with data collection. For the 5th-14th surveys, the contractor (Westat) designed and utilized a secure website which the AAAs used to upload their lists of selected clients. That website will be updated and further refined for the 15th national survey.

The web site is divided into two major sections: the public and the restricted-access sections. The public section is accessible to the general public, without restrictions. It includes background information, frequently asked questions, and links to results of previous AoA National Surveys. The purpose of the public section is to provide State and Area Agencies on Aging, professionals in the field of aging, and service recipients and their families with information about the data collection effort and uses of the data.

The restricted-access section of the web site houses an electronic records receipt system. Area Agencies on Aging have the option of submitting private personally identifiable client data to Westat via electronic files using the project web site. The web site was written in Active Server Pages (ASP), HTML, and JavaScript and uses the industry-standard TLS (Transport Layer Security) 1.1/2 encryption for secure data submissions. To further enhance security, it will be upgraded to a newer platform prior to the start of the 15^th national survey. Agencies choosing this option will receive usernames and passwords that enable their staff to sign on to the file upload utility on the web site. This system supports files in a large variety of file formats. Each agency's data file will be processed according to its structure and content.

Westat programming staff will manually map and convert the data items in each agency’s file to create standardized records for further processing. As each file is received, this system will log the source agency, date received, and file type.

Only agencies that have been selected to participate in the survey will have access to this area. Unique user IDs and passwords will be assigned to each AAA at the time they are selected into the sample. The ID and password will be provided with other survey materials to the AAA.

Appendix C contains for instructions for AAA restricted access to the survey website and how to submit data.

Use of Computer Assisted Telephone Interviewing (CATI)

Westat (the contractor) will use computer-assisted telephone interviewing (CATI) technology to conduct the surveys of OAA service recipients and record the responses. Westat’s CATI capability includes customized software systems for scheduling, interviewing, and data handling and utilizes high-speed data networks and centralized voice and data monitoring. A single database is used to monitor and direct the interviewers. The Scheduler, a computerized survey control system, makes interviewer assignments, records the disposition of sample cases, and helps survey managers monitor performance.

Westat will attempt to contact each person in the sample, making multiple calls at different times and days when necessary. To reduce the burden for the respondents, Westat will schedule appointments for calls at times that are convenient for them. Respondents will also have the option of calling a toll-free phone number to reach an interviewer and taking the survey at their convenience. For Spanish-speaking respondents, Westat uses specially trained bilingual interviewers to conduct the interviews in Spanish. If other special arrangements are necessary (e.g., interpreter, proxy needed, mail out requested, interview needed to be conducted over several sessions), the respondent can be further accommodated.

Westat will take the ACL/AoA-approved finalized version of the survey instruments and program them into its CATI system. This involves:

• Inserting specifications into the English version of the questionnaire;

• Preparing the specifications for the CATI programmer;

• Translating the questionnaire from the specifications into Spanish; and

• Programming and testing both versions of the questionnaire into CATI.

Details of how skips will work in the questionnaire are included in the design document, as are the needed question variations. For example, some questions may need to be asked differently, depending on the answers to previous questions. In particular, if a respondent told us they live with others, the next question we would ask would be, “Do you live with your spouse?” However, if the respondent told us they lived alone, the follow-up questions will not be asked, and CATI will automatically skip to the next question.

The use of the CATI system in combination with Westat’s highly structured telephone interviewer training and procedures ensures that interviewers conduct the surveys in a professional, controlled, and consistent manner.

A.4 Efforts to Identify Duplication & Use of Similar Information

Every effort is being made to avoid duplication and minimize respondent burden. Over the last 17 years, Westat conducted the first through 14th National Surveys of Older Americans Act Participants, formerly known as the National Survey of OAA Title III Service Recipients. As a result of the information gathered, modifications have been made to the data collection procedures and to the survey instruments. We believe we have reduced agency and respondent burden to the minimum level possible to achieve the survey's objectives.

In addition, ACL is careful not to duplicate data collection efforts between performance measurement and evaluation efforts. The National Survey data has been used as a secondary data source in a current evaluation effort. When future evaluation efforts require independent data collection, that data collection will be coordinated with the performance measurement surveys.

The NSOAAP is not duplicative of other survey efforts because there is no other representative survey of Older Americans Act participants. The HRS (Health and Retirement Study) collects nationally representative data on older adults every two years. The HRS is not able to identify the experience of OAA service recipients. The NSOAAP is a random sample of Older Americans Act (OAA) service recipients only, and cannot be used to make assertions about the American population of older adults. The purpose of NSOAAP is to obtain performance outcome information that demonstrates the effect of services and illustrates client reported quality of service.  In addition, service recipient demographics, health and wellbeing indicators are collected. However, the two survey efforts are complementary. Utilizing the HRS as a means to compare OAA service recipients to a nationally representative sample of older adults can help ACL better understand its program participants vis-à-vis a nationally representative sample.

A.5 Impact on Small Businesses or Other Small Entities

The data sources affected by the survey covered by this request for review will be agencies of state and local government, public purpose, quasi-governmental agencies, and clients who are private citizens. We have designed the sample to minimize the burden on both the agencies and the client respondents.

A.6 Consequences of Collecting the Information Less Frequently

This proposed fifteenth national survey will be conducted one time. If we are unable to conduct this survey we will be unable to move forward with our program performance measurement strategy as required by the OAA. We will be unable to pursue outcome measurement for the ACL/AoA Strategic Action Plan and our PART assessments will be compromised, especially since this survey format was developed to allow an overall assessment of Title III, while also continuing to allow assessment of individual services offered by Title III. We will be unable to report our FY 2021 consumer assessment performance measures and targets for GPRA and PART reporting purposes. The proposed sample sizes will allow us to analyze results by subgroup (e.g. region, age, race) and, therefore, allow better targeting of services.

A.7 Special Circumstances

The data collection effort will be conducted according to the guidelines specified in 5 CFR § 1320.6. No special circumstances are known that would cause inconsistency with these guidelines.

A.8 Comments in Response to the Federal Register Notice & Efforts to Consult Outside the Agency

Comments in Response to the Federal Register Notice

A 60-day Federal Register Notice published in the Federal Register on November 17, 2020, 85 FR 222 https://www.govinfo.gov/content/pkg/FR-2020-11-17/pdf/2020-25276.pdf (see Appendix D). A 30-day Federal Register Notice published in the Federal Register on April 1, 2021, 86 FR 17153.

ACL received comments from two organizations and from two individuals about the NSOAAP. One organization submitted multiple (10 comments). ACL reviewed all of the comments. The comment from one of the individuals was not relevant. For ease of review, the remaining comments and their responses have been grouped by topic or issue. The ACL responses for each topic/issue are detailed in Table A-1:

Table A-1 60-Day Federal Register Comments and ACL Responses

Topic/Issue	Comment	ACL Response
Food insecurity in older adults	“We support the inclusion of the USDA module to provide national estimates of the rate of food insecurity among OAA program participants. We recommend this module be continuously included in future administration of this survey.”	ACL concurs and plans to maintain the USDA module and associated questions for the foreseeable future.
Risk of malnutrition	“We suggest that ACL consider adding malnutrition screening questions in addition to the USDA module’s food insecurity questions, such as: • Do you ever eat only one meal daily?” • Do limits on chewing, swallowing or physical mobility ever prevent you from eating your home-delivered meals, even though you may be hungry?” • Do limits on chewing, swallowing or physical mobility ever prevent you from getting to your local congregate meal site and eating your meal, even though you may be hungry?”	ACL recognizes the importance of reducing and assessing risk for malnutrition. However, further deliberation is needed to ensure that we select the most appropriate and universally accepted language. ACL will call upon the expertise of a nutrition workgroup to make recommendations to ACL on selecting the best language to use. The workgroup’s recommendation will inform a redesign of future NSOAAP survey collection efforts.
Medically-tailored meals for participants requiring special diets	“We suggest that ACL consider adding a question about the need for therapeutic diets or texture-modified meals to better understand the needs of participants as it relates to medically tailored meals.”	ACL recognizes the importance of capturing data on the capability of OAA nutrition programs to accommodate special diets. ACL will call upon nutrition experts to make recommendations and inform a redesign of future NSOAAP collection efforts.
Ethnic and cultural barriers to communication and nutritional preferences	“We suggest that ACL consider including a question about communication barriers to both the congregate and home-delivered meals modules, such as: • Do you have language or cultural barriers to talking with staff at your congregate meal site/ your home delivery staff? We also suggest that ACL consider adding a question to both the congregate and home-delivered meals modules about meals meeting cultural preferences.	ACL recognizes that ethnic and cultural barriers may affect participants in the nutrition programs. ACL will call upon nutrition experts to make recommendations and inform a redesign of future NSOAAP collection efforts. ACL is considering a special topical module related to equity and underserved populations.
Nutrition counselor: Rephrase question to improve clarity	“SVC1(k) asks whether the respondent has access to a “nutrition counselor” who is providing dietary advice based on the respondent’s condition, medications, and related factors. We question why the survey would not specify “a qualified nutrition professional such as a registered dietitian” (or registered dietitian nutritionist), since these professionals are the most qualified to answer such questions. Moreover, in 28 states, only licensed professionals are legally eligible to provide such advice. The term “nutrition counselor” allows for substantial subjective interpretations, and could theoretically include food service staff or other program participants who may be providing such advice against state law.”	ACL recognizes “nutrition counselor” may offer subjective responses. However, respondents may not know if the qualifications of the person providing dietary advice. ACL will call upon nutrition experts to make recommendations and inform a redesign of future NSOAAP collection efforts. The use of terminology for this item will be tested.
Modify item response in nutrition modules to include positive as well as negative changes.	“We note that the survey asks about changes in meals, but almost all coding options for the interviewer are about reductions or negative changes with few opportunities for interviewers to code any positive changes reported by participants. In addition to coding for both reductions and improvements in quality of the food, we recommend adding the corresponding “positive” option for all other codes. “	ACL concurs with this suggested change. The requested change has been made to the survey instrument.
Consistent language in meals program.	“Ensure that consistent language is used to describe the program, particularly for congregate or senior dining meals. Some questions use the term “meals program” while others use the term “lunch program” or, generically, “this service”, including CNR20-23 where three different terms are used across four consecutive questions. This also applies to SVC1 question in Additional Services module. We recommend using the term “meals program” unless exclusively referring to lunch, as some programs serve breakfast or dinner meals rather than lunch meals.”	ACL recognizes that the language used to describe the meals programs may need to be revised to be more consistent. ACL will call upon nutrition experts to make recommendations and inform a redesign of future NSOAAP collection efforts. The term “meals program” throughout due to the variety of food services during the COVID-19 pandemic.
Living independently	“SVC3 asks about continuing to “live independently” vs. “living at home” (as they do in CS15, CNR23, HNR28, HC9 and TR20). The terminology “living independently” is preferred. Simply continuing to live “at home” does not mean that the person is living independently, is living in their own home (vs that of a relative), or has autonomy over where he or she lives.”	ACL concurs with this suggested change. The requested change has been made to the survey instrument.
Context of “secure”	“SVC3(b) should be more specific with regard to the context of “secure.” We are unsure whether the context is financial, food-based, or related to physical safety.”	The question refers to how participants feel about additional services that they or their care recipient receive. ACL recognizes that the use of the word “secure” may be ambiguous. ACL will call upon experts in the aging network to make recommendations to inform a redesign of future NSOAAP collection efforts.
Multiple meals	“HNR5 should be re-phrased to reflect the fact that many HDM clients receive more than one meal and may consume multiple HDMs in a day.”	ACL recognizes that the question may need to be modified. ACL will call upon nutrition experts to make recommendations and inform a redesign of future NSOAAP collection efforts.
Care recipients under age 60 with dementia	The 2020 reauthorization of OAA allows “individuals living with dementia under the age of 60 to access certain OAA supports like nutritional services and respite care through the National Family Caregiver Support program. We respectfully request that ACL analyze the data collected on these younger individuals and the services they receive so that those services can be tailored and improved.”	While ACL recognizes the value of collecting data on care recipients of the NFCSP who are under 60, that is not the focus of the NSOAAP. The focus of the NSOAAP is the service recipients, that is, the caregivers.
Emergency preparedness	The advocacy organizations “support ACL's request to add an emergency preparedness module to the survey…”.	ACL is not going to add this module for the 2021 administration but still plans to use it in the future.
Question on participant sex (gender identity)	“I encourage the modification of the questions about participant sex to include response options (which are recorded but not verbally offered as options) to include "transgender" and or "other" with a write-in option. The questions this relates to are: DE1 (DEGENDR); CGC (CGPMF); CG83 (CGPMF) - both of the last have the same variable labels and this should be corrected in the survey construction.”	In order to improve demographic data collection related to the LGBT community, ACL is supporting the Measuring Sex, Gender Identity, and Sexual Orientation for the National Institutes of Health an ad hoc panel of the National Academies of Sciences, Engineering, and Medicine which will review current measures and the methodological issues related to measuring sex as a non-binary construct, gender identity, and sexual orientation. The panel will produce a consensus report which is expected in December 2021. ACL will use the report as a foundation for testing new survey questions and administrative data elements. The variable label in the survey instrument for CGPMF was corrected to RGENDER.

Response to 30-day Comments

A 30-day Federal Register Notice published in the Federal Register on April 1, 2021, 86 FR 17153.

There were 11 public comments received during the 30-day public comment period, the comments were on two topics: three on malnutrition and seven on transgender (with one unknown). ACL drafted the following two response summaries (see below).

Malnutrition:  In late 2020/early 2021, a nutrition advisory group was called upon to help ACL construct nutrition questions for the rotating module on COVID-19 that has been added to the 2021 NSOAAP. We will call upon that advisory group again, as well as other experts in the field, to review all the nutrition questions in the NSOAAP beginning in Nov/Dec 2021. The recommendations of the advisory group will feed into the survey redesign that is scheduled to take place for the 2023 NSOAAP.

Transgender: ACL plans on following guidelines from a forthcoming consensus report on Measuring Sex, Gender Identity, and Sexual Orientation for the National Institutes of Health produced by an ad hoc panel of the National Academies of Sciences, Engineering, and Medicine (NASEM). The NASEM ad hoc panel is in the process of reviewing current measures and the methodological issues related to measuring sex as a non-binary construct, gender identity, and sexual orientation. The consensus report is expected in December 2021 allowing us to begin testing the questions in early 2022.

Efforts to Consult Outside the Agency

For updates to the 2019 survey instrument, ACL/AoA called upon the expertise of a work group to review NSOAAP data collection tools and to make recommendations to ACL on selecting the best language to use for revising questions in the survey instruments. The NSOAAP work group was comprised of experts on aging data and survey methodology. The work group’s recommendation will inform a redesign of future NSOAAP survey collection efforts.

For the addition of a special one-time 2021 module on questions related to COVID-19, ACL/AoA convened a new advisory workgroup consisting of members from different State Units on Aging (SUAs), Area Agencies on Aging (AAAs), academia and advocacy organizations on nutrition, aging, and family caregiving. The advisory workgroup members were divided into three subgroups focusing on nutrition, well-being and other services (transportation, case management, and homemaker services), and family caregiving. Workgroup members discussed, evaluated, and ranked proposed COVID-related questions. Through the result of the workgroup members’ voting and prioritization, a final selection of 10-13 questions by service category resulted in the final COVID-19 special module for 2021.

The majority of the remaining questions in the survey instruments for this proposed information collection are based on those developed by ACL/AoA POMP grantees representing State Units on Aging and AAAs. POMP grantees who have worked on the survey instruments include state and local level representatives from Arizona, Florida, Georgia, Massachusetts, New York, North Carolina, and Ohio. The development of the survey instruments has been an iterative process. There were no areas of disagreement during the latest POMP revisions.

The POMP grantees tested the instruments with service recipients at the local AAA-level using several methods:

1. Field-tested the survey instruments with a sample of service recipients and revised the instruments based on their experience.

2. Conducted cognitive testing to ensure that the items on the survey instruments were interpreted as intended.

3. Conducted validity testing on the survey instruments.

Westat (the contractor) has also consulted representatives from different State Units on Aging to develop and test the instructions and procedures for generating client lists used for sampling. The state representatives who have reviewed the instructions and procedures include:

• Robin Tofil, Connecticut Department of Social Services, Aging Services Division

• Jim Burd, Pennsylvania Department of Aging

• Leonard Eshmont, Virginia Department for the Aging

A.9 Explanation of Any Payment or Gift to Respondents

No payments or gifts will be given to respondents.

A.10 Assurance of Privacy Provided to Respondents

After review, ACL has determined that the current Privacy Impact Assessment (PIA) meets the established guidelines. There are no assurances of confidentiality. With that, privacy and anonymity are important parts of the survey design. In response to this concern, ACL will ensure the anonymity of all individuals who provide data. A pledge of privacy and anonymity is a major positive incentive for potential respondents to participate in the survey. Its absence would be a significant deterrent and could create complications in implementing the survey.

Westat in contract with ACL will take the following precautions to ensure the privacy and anonymity of all data collected:

• All Westat project staff, including recruitment specialists, telephone interviewers, research analysts, and systems analysts, will be instructed in the discolousre requirements of the survey and will be required to sign statements affirming their obligation to maintain privacy;

• Only Westat staff who are authorized to work on the National Survey have access to client contact information, completed survey instruments, and data files.

• Data files that are delivered will contain no personal identifiers for program participants; and

• Analysis and publication of survey findings for the participant survey will be in terms of aggregated statistics only.

Appendix E presents the internal corporate “Assurance of Confidentiality Agreement” all Westat project staff must sign. This agreement requires the signer to keep confidential and private an unity and all information about individual respondents to which they may gain access. Any Westat employee who violates this agreement is subject to dismissal and to possible civil and criminal penalties.

Westat, the contractor for administering the survey instrument and collecting the data, has extensive experience in protecting and maintaining the privacy of respondent data collected from surveys. To ensure privacy, Westat has drawn from its experience in designing the data collection procedures incorporated in this program. In addition to the corporate Assurance of Confidentiality Agreement, Westat has implemented several other procedures to protect privacy of survey participants.

1. Data is saved on secure network folders only accessible to authorized users. No data is ever stored on laptop computers. At the end of the survey, all private data is permanently deleted.

2. For the 15th National Survey and subsequent follow-up surveys, AAAs will be instructed to submit private personally identifiable client data to Westat via electronic files using the secure survey web site. Agencies will receive usernames and passwords that enable their staff to sign on to the file upload utility on the web site. The passwords are created by a password generator which creates random passwords that are highly secure due to a combination of lower and upper case letters, numbers and punctuation symbols. The database containing the client survey data is not accessible via the Internet; it resides on a server inside the Westat firewall. Only Westat Data Collection Program staff members have access to the master survey database.

3. For AAAs that may experience problems with the survey website and wish to send client data electronically by email, we instruct the AAAs to password protect the file containing the data. Password protection of client data sent electronically by email is required not only for transmission between the AAA and Westat, but even internally within Westat. Additionally, we provide the AAAs with an email address to a secure dedicated project email box ([email protected]) which cannot be accessed remotely.

4. For the small number of AAAs that are not able to generate client records by service electronically, they can submit client information in a hard copy format (fax, FedEx, U.S. Postal Service). Hard copies of client information are stored in locked filing cabinets within a locked room. At the conclusion of the survey, all hard copies of client data are shredded.

5. A secure fax machine dedicated solely to this survey is used to receive faxes from AAAs that choose to transmit their data by fax. The fax machine is located within a locked project room. AAAs that need to transmit their data by fax are asked to call to Westat staff to alert them to watch for and intercept an incoming fax. If the fax machine is busy, it does not roll over to any other fax machine.

All respondents in this data collection effort are assured of the privacy of their answers. Respondent data are aggregated and estimates are produced and published at the both at the national level and at the geographic regional or demographic sub-group level. No individual-level data are published, nor are they accessible or provided to anyone outside the Westat Data Collection Program staff.

A pre-notification letter mailed to potential respondents contains essential survey information and assurances of privacy that enable the person to make an informed decision regarding his or her voluntary participation in the data collection effort. A sample of the pre-notification informational letter sent to potential survey participants appears in Appendix F.

A.11 Justification for Sensitive Questions

The ACL/AoA National Survey informs respondents that their responses to all questions are voluntary. We assure them that their survey responses will remain private. Respondents can refuse to answer any question, and the interviewer will move on to the next question on the survey instrument. Additionally, respondents are permitted to stop at any point and to continue the interview at a later time.

The physical functioning module contains questions on the ability of respondents to perform certain tasks, such as getting around inside and outside the home, getting in and out of a chair or the bed, getting to and using the toilet, etc., as well as questions asking about their health conditions. These types of questions might be considered to be sensitive; however, we have never had a respondent object to answering these types of questions, especially when they are part of a battery of physical functioning questions. In addition, analysis of Activities of Daily Living (ADL) and Instrumental Activity of Daily Living (IADL) limitations in conjunction with outcomes and the type of services a respondent receives is an important outcome measure. This kind of information, along with responses to questions on health conditions, can tell us about the frailty of the respondents served by the nutrition (home-delivered and congregate meals), home care (homemaker/housekeeping), case management and transportation services, and the people who are able to maintain their independence, rather than enter nursing homes, because of those services. Caregivers will be asked about the health conditions and ADL and IADL limitations of their care recipients, again to allow for analysis of the frailty of care recipients whose caregivers are part of the National Family Caregiver Support Program. Respondents can always refuse to answer any question, and the interviewer will move on to the next question on the survey instrument.

Additionally, a question asking respondents if they identify as lesbian or gay, straight, bisexual, or something else is included in the National Survey Instrument and may be considered sensitive. However, as noted by HHS’s Office of Minority Health and supported by additional research, persons who identify as lesbian, gay, bisexual, or transgender (LGBT) are more likely to suffer disparities in health and access to human service organizations. Therefore, information about the LGBT population is valuable to AoA’s Aging Network, as providers of Older Americans Act services can benefit from a greater understanding of this commonly underserved population; a population that may face unique health challenges and limited access to health care and social services.

OMB (OMB HHS/CDC Control 0920-0222) approved a series of sexual identity questions for use in the National Health Interview Survey fielded by the National Center for Health Statistics. A standard set of questions on sexual identity was created and tested. This set of questions had been proposed for inclusion in the 2014 ACL/AoA National Survey. In 2017, although the follow-up questions were dropped, the original question can be found in the National Survey Instrument’s Demographic module as questions DE1a (DETHINK) on page 98 of Appendix J.

A.12 Estimates of Annualized Burden Hours and Costs

We estimated the respondent burden for the survey instruments based on the 3^rd to 14th National Surveys of OAA Participants. The cost to respondents who participate in the survey will be in terms of their time only. The Service Recipient survey instrument (plus rotating module) takes about 45 minutes (.75 hour). Most of the Service Recipients are retired. The Caregiver survey (plus rotating module) will also take about 45 minutes. Based on the valuation of a participant's time at $25 per hour, the respondent burden for each participant will be $18.75 for the Service Recipient and $18.75 for the Caregiver surveys and $176 for the agency respondent selection process (estimated at 4 hours of agency personnel time). Exhibit A-1 presents the estimated hour and annual cost response burden by respondent.

Exhibit A-1. Estimated Hour and Annual Cost Response Burden

Respondent/Data collection activity	Number of respondents	Responses per respondent	Hours per response	Annual burden hours	Cost per hour	Annual burden (cost)
Area Agency on Aging: Respondent selection process	350	1	4.0	1,400	$44	$61,600
Service recipients (i.e., Congregate and Home-delivered meal nutrition programs; Case Management, Homemaker, and Transportation Services) + Rotating Module	4,400	1	.75	3,300	$25	$82,500
National Family Caregiver Support Program Clients + Rotating Module	2,200	1	.75	1,650	$25	$41,250
Total	6,950	1	.914 (weighted mean)	6,350	Varies	$185,350

* It is important to note that not all of the individual respondents (6,600 for the national survey) will be asked to complete all of the questionnaire modules (see Sampling Plan).

A.13 Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

Total annual cost burden excluding wages of agency time and respondent time is zero (see Exhibit A-1).

A.14 Annualized Cost to the Federal Government

The overall cost of this research to the Federal Government is presented in Exhibit A-2.

Exhibit A-2. Total Annualized Cost to the Federal Government [Based on Year 1]

Category	Costs
Personnel (T&M including staff & indirect costs)	$810,896
Telephone (long-distance telephone survey)	$53,820
Other direct	$27,159
Total direct charges (per task order)	$80,979
Indirect charges	$13,498
Total	$905,373

A.15 Explanation for Program Changes or Adjustments

This is a revision to a currently approved information collection request; there is a program change decrease of -7250 annual burden hours.

A.16 Plans for Tabulation & Publication and Project Time Schedule

In this section, the range of analyses that will be conducted is described using the performance measurement data. The research team suggests several levels of analysis to first describe the characteristics of clients and the range of services provided by State Units on Aging (SUAs) and AAAs, noting similarities and differences in the size and scope of program design and operations. The research team will paint a descriptive profile of OAA clients, paying special attention to risk factors that prior research shows are highly correlated with quality-of-life outcomes for older persons. These risk factors include ADL and IADL limitations, as well as income, educational level, living arrangements, age cohort, gender, race and ethnicity, and area of residence by degree of urbanization. In conjunction with the type and severity of ADL/IADL limitations that clients report, these factors are indicative of risk for loss of independence and other adverse outcomes. For example, it is known that among older adults, those living alone are at higher risk for institutional placement. Educational attainment, in conjunction with age, is a powerful predictor of need for services and outcomes for older persons.

In conjunction with the performance measure on service recipients, the survey will include questions on ADL limitations (e.g., difficulty with personal care activities such as bathing and dressing) and IADL limitations (e.g., difficulty with such home management activities as meal preparation, shopping, and housekeeping), and health conditions. These questions appear in Appendix J in the Physical, Social, and Well-Being Module and are denoted as “PF” for Physical Functioning. In addition, caregivers will be asked about the health conditions and ADL limitations of their care recipients.

The timetable for the baseline data collection and the two follow-up data collections is shown in Exhibit A-3.

Exhibit A-3. Data Collection Timetable

Survey Cycle	Data Collection Activity	End dates
15th National Survey/ Data Collection	Telephone/email contact with agencies to draw sample	1 month after OMB clearance
15th National Survey/ Data Collection	Telephone survey of participants	5 months after OMB clearance
15th National Survey/ Data Collection	Data editing, coding and key entry, data analysis	7 months after OMB clearance
15th National Survey/ Data Collection	Deliver data to ACL	9 months after OMB clearance
15th National Survey/ Baseline Data Collection	Final report on data collection	15 months after OMB clearance
16th National Survey/ Data Collection	Telephone/email contact with agencies to draw sample	13 months after OMB clearance
16th National Survey/ Data Collection	Telephone survey of participants	17 months after OMB clearance
16th National Survey/ Data Collection	Data editing, coding and key entry, data analysis	19 months after OMB clearance
16th National Survey/ Data Collection	Deliver data to ACL	21 months after OMB clearance
16th National Survey/ Data Collection	Final report on baseline data collection	27 months after OMB clearance
17th National Survey/ Data Collection	Telephone/email contact with agencies to draw sample	25 months after OMB clearance
17th National Survey/ Data Collection	Telephone survey of participants	29 months after OMB clearance
17th National Survey/ Data Collection	Data editing, coding and key entry, data analysis	31 months after OMB clearance
17th National Survey/ Data Collection	Deliver data to ACL	33 months after OMB clearance

A.17 Reason(s) Display of OMB Expiration Date is Inappropriate

ACL/AoA is not seeking an exemption from displaying the expiration date of OMB approval.

A.18 Exceptions to Certification for Paperwork Reduction Act Submissions

ACL/AoA is not requesting any exceptions from OMB Form 83-I.

B Collection of Information Employing Statistical Methods

B.1 Respondent Universe and Sampling Methods

Introduction

This Paperwork Reduction Act (PRA) request is to conduct a cross-sectional survey of OAA service recipients.

We will employ a two-stage sample design for the 15th National Survey of Older Americans Act Participants (NSOAAP). The following sections discuss the respondent universe and sampling methods.

Baseline Respondent Universe

For the first stage of the sample design, we will select a probability sample of AAAs proportional to size (PPS) of the total number of clients (actual and predicted) in each stratum, where stratum is defined by Census Division. The Census Divisions are as follows:

Census Division

1 (New England)

2 (Middle Atlantic)

3 (East North Central)

4 (West North Central)

5 (South Atlantic)

6 (East South Central)

7 (West South Central)

8 (Mountain)

9 (Pacific)

As a result of the 14^th NSOAAP, client counts are available for 581 out of the 628 AAAs in the sampling frame, covering approximately 92.5% of the AAAs. We were able to predict the total client counts for those AAAs for which we did not have prior data using regression modeling, where the square root of the AAA budget and Census Division were used as predictors.

When selecting AAAs for the 15th National Survey, Westat will select a sample of AAAs large enough to recruit 350 Area Agencies on Aging. The second stage is the selection of a random sample of service recipients by service within each sampled AAA, including all of the largest ones. In this way all service recipients will have a known probability of selection. A fixed number of service recipients will be selected within each service for a total of 6,600 recipients. It is important to note that clients are sampled independently by service and no client will be asked to participate for more than one service. Further if a client happens to be sampled for more than one service, the client will be assigned to a single service at random.

Exhibit B-1 on the next page presents the respondent universe for each module proposed for the 15th National Survey of Older Americans Act Participants (NSOAAP).

Exhibit B-1. Respondent Universe Service Recipient Survey
PERFORMANCE MEASURES		INDICATOR	TARGET POPULATION OF PARTICIPANTS
Congregate Meals Module		Questions on nutrition intake, nutrition risk, food security and clients’ assessments of the Congregate Meals program.	All service recipients receiving Congregate Meals services
Home-delivered Meals Module		Questions on nutrition intake, nutrition risk, food security and clients’ assessments of the Home-delivered Meals program.	All service recipients receiving Home Delivered Meals
Transportation Module		Questions on client’s experience and assessment of transportation services.	All users of Transportation Services
Case Management Module		Questions on clients’ experiences and assessments of case management services.	All service recipients receiving Case Management services.
Homemaker/Housekeeping Module		Questions on clients’ experiences and assessments of homemaker/housekeeping services.	All service recipients who receive Homemaker/Housekeeping Services
Caregiver Support Module		Questions on clients’ experiences and assessments of caregiver support services.	All service recipients who receive Caregiver Support Services
Additional Services List		Questions asking service recipients if they receive other OAA services.	All service recipients. Caregivers will be asked about services received by their care recipients.
Physical Functioning Module		Revised Katz Activities of Daily Living (ADL) Scale and Quality of life measures from the Behavior Risk Factor Surveillance System (BRFSS) questionnaire.	All service recipients, with the exception of Caregivers; Caregivers will be asked these questions about their care recipients.
Emotional Well-Being Module		Questions on mood and affect from prior surveys of older adults.	All service recipients, except Caregivers.
Social Functioning Module		Degree of satisfaction with social activity and of health effects on social activities.	All service recipients, except Caregivers.
Falls Module		Questions ask about falling and losing one’s balance.	All service recipients and Caregivers.
Social Integration Module		Questions about contact with other people and perception of social isolation.	All service recipients, except Caregivers.
Life Changes Module		What life event prompted client to seek out services.	All service recipients, except Caregivers.
Demographic Information Module		Demographic Information	All service recipients and Caregivers.
Annual Rotating Module
PERFORMANCE MEASURES	INDICATOR		TARGET POPULATION OF PARTICIPANTS
COVID-19 Module (2021)	Questions ask about the effect COVID-19 has had on older adults’ access to and use of Older Americans Act programs and services during the past year.		All service recipients and Caregivers.

Response Rates from other National Surveys of Older Americans Act Participants

This is the 15th time this type of survey will be conducted. This OMB approved survey (0985-0014, 0985-0017, 0985-0020, 0985-0023) was done in 2002, 2003, 2005, 2008, 2009, 2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, and 2019. The research team anticipates an 80 percent response rate for AAAs given the 81% AAA response rate in the 14^th NNOAAP, and an 80 percent cooperation rate for the telephone survey of respondents, based on the success we had with the preceding 14 surveys.

Response rates are calculated by taking the ratio of the number of completed interviews to the number of eligible units (this includes completes and non-respondents). Non-response occurs for a variety of reasons, including, for example, being unwilling to participate, unable to locate respondents, and language issues. In the case of surveys conducted by telephone, such as NSOAAP, potential respondents may have difficulty communicating by phone, another instance of nonresponse. The U.S. Census Bureau defines the following: “Unit nonresponse occurs when respondents are unable or unwilling to participate; interviewers are unable to locate addresses or respondents; or when other barriers exist to completing the interview.”⁴

In NSOAAP, we often refer to completion rates because of our experience in locating and contacting potential respondents. Members of the NSOAAP target population frequently have changes in status, such as relocating to be closer to family members, moving into assisted living or nursing facilities or other such changes. These changes can occur between the time the client lists are created and attempts are made in the field to contact the individuals. Completion rates are calculated by excluding records for whom no contact was ever established, for example, the not-locatable and not resolved cases from the non-respondents.⁵ In a survey like NSOAAP the completion rate may be a more accurate measurement of the degree of success in collecting data which are representative of the population of service clients.

B.2 Procedures for the Collection of Information

B.2.1 Introduction

Several data collection activities will be conducted to support the survey. They are designed to ensure as complete a sample of AAAs (stage one) and service recipients (stage two) as possible. This will provide a representative sample for the analyses and to inform ACL/AoA on results of performance measures for state and community programs on aging under the Older Americans Act.

B.2.2 Data Collection Procedures

B.2.2.1 Telephone Contact with State and Local Agencies on Aging

Information will be collected in a two-step process. The proposed design will employ a probability sample of all AAAs proportional to size (PPS) of the total client counts per AAA. Once an agency is selected, its director will receive an email message from ACL that contains an introductory letter from ACL, an invitation to participate in an orientation webinar, and detailed instructions for the AAA (see Appendices H & I)⁶. Approximately two weeks later, state and AAA staffs will be able to participate in a live webinar to learn about the importance of the survey and how the procedures for selecting clients will work. ACL will also post a recording of the webinar for state and AAA staffs who are unable to participate during the live webinar. For AAAs that do not respond to the initial email from ACL or participate in the webinar, a researcher will reach out to the agencies to address any concerns that they might have and to review instructions for sampling the service recipients. The researcher will explain the numbered participant lists the agency needs to generate from which to select the random sample of service recipients for each of the six services. In addition, we will provide detailed instructions specific to the client tracking software used by the AAA. Previous experience has enabled Westat, the contractor, to streamline the data collection procedures for the AAAs.

B.2.2.2 Telephone Survey of Older Americans Act Participants and Caregivers

Pre-notification Advance Letters

Potential respondents selected for the telephone interview will receive a letter from ACL. The letter contains an introduction to the study, explanation about the nature of participation, and a toll-free phone number to call if they do not wish to participate. Those who opt out of the study are not contacted further. A separate toll-free phone number is included for respondents who wish to reach a telephone interviewer directly, and take the survey at their convenience. A copy of this letter is included in Appendix F].

Telephone Interview

Interviewers participate in intensive training sessions prior to data collection and are monitored during data collection to ensure the protocol is properly followed. The training covers general interviewing techniques, topics specific to administering the 15th National Survey of Older Americans Act Participants, and practice sessions.

The study sample includes older adults who may be living with disabilities. With that in mind, the training designed and conducted for the data collectors/telephone interviewers includes special guidance for interviewing and accommodating respondents who are age 60 or older and who may have disabling conditions and/or communication problems (hearing impairments, speech disorders, cognitive impairments, memory disorders, non-native English speakers.) In certain instances, an interview with an interpreter or a proxy is arranged. Additionally, data collectors are advised to be alert to the respondent’s fatigue and to suggest calling back and completing the interview during another session. For Spanish-speaking respondents, trained bilingual data collectors conduct the interview in Spanish.

Interviewers will conduct a 45-minute telephone survey of a representative sample of Older Americans Act service recipients and caregivers. The interview includes modules for each service (e.g., home delivered meals, congregate meals, case management, caregiver, transportation, and homemaker) as well as modules that are the same for all services on demographics, physical functioning, and quality of life. Additionally, a module on COVID-19 will be asked of all respondents for the 2021 survey. Interviewers administer the appropriate service module (i.e., the module that focuses on the service from which the participant was sampled.)

The service modules include items on the extent to which the respondents use the service, consumer assessment of services, and self-reported outcomes, such as the ability to live independently at home. The demographic module identifies age, living arrangements, race/ethnicity, and income categories. The module on physical functioning identifies the extent to which respondents are able to care for themselves (e.g., bathe dress, eat, etc.) and are able to handle paying bills, going to the doctor, and grocery shopping, for example.

Quality Control Procedures

Westat has quality control procedures in place for every phase of the project. Interviewers participate in rigorous training that includes general interviewer training and project specific training. Trainers observe interviewers conducting practice interviews, and they monitor interviewers during data collection. During data collection, data are checked to ensure that there are no outliers in the dataset. In addition, when questions are raised during an interview, interviewers complete a form documenting an ambiguous or inconsistent response. Researchers review the forms and make any necessary adjustments.

B.2.3 Sampling Plan

B.2.3.1 Sample Design

The sample design for the 15th survey will consist of two stages, with a sample of approximately 350 AAAs in the first stage and a sample of clients, by service type, from each selected AAA, in the second stage. This design is similar to that of the 3^rd-14th surveys. The client sample sizes by service type, as specified by ACL, are as follows:

• Caregiver Services 2,200

• Home Delivered Meals 1,100

• Congregate Meals 1,100

• Case Management Services 550

• Transportation Services 1,100

• Homemaker Services 550

As in the 3rd through 14^th surveys, these sample sizes will permit the production of reliable estimates both at the national level and at the geographic regional or demographic sub-group level.

For a two-stage design, Table B-1 presents the half-widths of the 95 percent confidence intervals (CI) for various sample sizes and for cross-sectional estimates of target characteristics of proportions ranging from 10 percent to 50 percent.⁷ The 50 percent target is a worst-case scenario, where respondents are expected to be fairly evenly split on a particular response item, limiting the reliability of the estimate (e.g., such as trying to predict the outcome of an election where the sample of voters is about evenly divided between two candidates). Also, the precision of any estimate greater than 50 percent is the same as that of its complement, i.e., the precision of a 70 percent estimate is the same as the precision of a 30 percent estimate. The numbers in the tables are half-widths of 95 percent CIs, (i.e., the estimate, the half-width is the CI, where half-width is 1.96 times the standard error (SE) of an estimate). For example, Table B-1 shows that for a sample of size 1,000, for a target characteristic of around 30 percent, the CI would be the estimate 3.24 percent.

The table can be used to assess the adequacy of the sample sizes for both the national, and the regional or sub-group level estimates. For example, if the sample size is 1,000 at the national level then the sixth row in Table B-1 would provide the precision of the estimates at the national level. From the same table, the precision of an estimate at the regional or sub-group level can be obtained by computing the sample size that is expected for a particular region. For instance, if the region covers 25 percent of the target population, then the sample size for that region is expected to be about 250 (out of 1,000) under a proportional allocation, and the precision of the estimates for that region can be checked from the row where the sample size equals 250 in Table B-1. Similarly, if a sub-group covers 10 percent of the target population then the expected sample size for that sub-group is 100 out of 1,000 and the precision of the estimates for that sub-group can be checked from the row with sample size equal 100.

The total size of the target population has a negligible impact on the requirement of the sample size. For example, if a sample size of 250 is required to produce an estimate at the national level, then to estimate the same characteristic for a particular region (with the same level of precision), the required sample size from that region alone would be about 250. If there are four regions, then the required sample size at the national level would be about 1,000 (to guarantee adequate representation in each group). Therefore, to meet the objective of the proposed survey (i.e., to produce estimates at the regional or sub-group level with the same level of precision as the national estimates obtained from previous studies), the required sample size for each target region or sub-group will have to be the same as the total sample size of the previous studies.

For instance, a question was asked in the first national survey about the timeliness of the delivery of meals and an estimated 44 percent of all clients reported that the meals arrived on time, all the time. This estimate was based on a sample of 472 clients and had a CI of 5.2 percent. Table B-1 shows that to achieve a CI of 5.2 percent for an estimate, with a proportion between 40 percent and 50 percent, a sample of size around 480 is required. That means if this estimate is required at the regional level with the same level of confidence as the national, then the sample size in each region will have to be 480 and hence the sample size at the national level will be 480x4=1,920. In that case, the CI for this estimate at the national level would be much more precise than for the region (a little over 2.5 percent). Table B-1 can be used to see the precisions of the estimates that would be achieved at various levels using the expected sample sizes at the respective levels. The table can also be used to check the sample size requirement corresponding to a desired level of precision of an estimate.

Table B-1 Half-widths of 95 percent confidence intervals by various sample sizes and estimates of target characteristics (computed for a two-stage design with a design effect of 1.30)

Sample size	Estimates of Target Characteristics
	10 percent	20 percent	30 percent	40 percent	50 percent
3,500	1.13	1.51	1.73	1.85	1.89
3,000	1.22	1.63	1.87	2.00	2.04
2,500	1.34	1.79	2.05	2.19	2.23
2,000	1.50	2.00	2.29	2.45	2.50
1,500	1.73	2.31	2.64	2.83	2.89
1,000	2.12	2.83	3.24	3.46	3.53
750	2.45	3.26	3.74	4.00	4.08
500	3.00	4.00	4.58	4.90	5.00
400	3.35	4.47	5.12	5.47	5.59
300	3.87	5.16	5.91	6.32	6.45
250	4.24	5.65	6.48	6.92	7.07
200	4.74	6.32	7.24	7.74	7.90
100	6.70	8.94	10.24	10.95	11.17

It is important to note that if the population sizes in the sub-groups or regions vary widely, then the national sample must be allocated appropriately to produce estimates from all individual sub-groups/regions with an equal level of precision. Otherwise, under a proportionate allocation, larger sub-groups will have more than required sample size while the smaller sub-groups will have less than the sample size required. For example, if the estimates are required separately for Whites and African-Americans, then just increasing the national sample would not ensure sufficient sample size for African-Americans, because less than 15 percent of recipients are African-Americans for many services. In this situation, the national sample can be disproportionately allocated by over-sampling smaller sub-groups to ensure that sufficient samples are drawn from all target sub-groups. However, over-sampling an ethnic or demographic group will require that agencies first list all their clients with the characteristic of interest and then select a sample from this list by sub-group (which may exceed the capacity of many AAA information systems).

B.2.3.2 Sample Size for Estimation of Change

If there is interest in comparing estimates from one year with another year (cross-sectionally) , or comparing estimates of one sub-group with another sub-group, the sample size requirements are different from those that show individual point estimates at the same level of precision. The standard error (SE) of the difference between two independent estimates (for example, A and B) can be obtained by

, and the half-width of the 95 percent CI is

Since the variance of the estimate (of a difference between estimates) is the sum of the variances of the relevant individual estimates, the required sample size for estimating a difference or change is higher than for a single point estimate.

Table B-2 presents half-widths of 95 percent CIs under a two-stage design for various sample sizes and various averages of the two estimates to be compared. For example, if the average of the two target characteristics to be compared is around 30 percent (for example, A=25 and B=35) and the sample size in each sub-group is 500, to detect a difference between the two sub-groups with statistical significance, the actual difference between the two sub-group characteristics will have to be at least 6.48 percent. This is much higher than the corresponding half-widths presented in Table B-1 for each of the individual estimates. That means a sample size that is sufficient to produce a reliable point estimate for each sub-group, individually, is not necessarily sufficient to detect the difference between the two sub-groups with the same level of precision.

Therefore, if the survey is designed for use at a region or sub-group level, then the corresponding national estimates can be compared meaningfully from one year to another, or for one service versus another (e.g., the percent of each service’s clients below a certain income level). For example, if the sample size is 1,000 in each year, and if the average response proportion for the two target characteristics is around 30 percent, then a difference of 4.58 percent or more between the years is detectable. The corresponding comparison with a sub-group sample of size 500, would not allow detecting a difference unless it is 6.48 percent or more.

Table B-2 can be used to see the extent of difference that can be detected under a two-stage design, for various sample sizes, and for various characteristics to be compared either at the national or at the sub-group level.

Table B-2 Half-widths of 95 percent confidence intervals for the difference between two estimates by various sample sizes and for various averages of the two estimates (computed for a two-stage design with a design effect of 1.30)

Sample size in each group	Average of the estimates to be compared
	10 percent	20 percent	30 percent	40 percent	50 percent
3,500	1.60	2.14	2.45	2.62	2.67
3,000	1.73	2.31	2.64	2.83	2.89
2,500	1.90	2.53	2.90	3.10	3.16
2,000	2.12	2.83	3.24	3.46	3.53
1,500	2.45	3.26	3.74	4.00	4.08
1,000	3.00	4.00	4.58	4.90	5.00
750	3.46	4.62	5.29	5.65	5.77
500	4.24	5.65	6.48	6.92	7.07
400	4.74	6.32	7.24	7.74	7.90
300	5.47	7.30	8.36	8.94	9.12
250	6.00	8.00	9.16	9.79	9.99
200	6.70	8.94	10.24	10.95	11.17
100	9.48	12.64	14.48	15.48	15.80

Nonresponse adjustment was done as part of the weighting process for the previous surveys and will also be done for the 15th National Survey. The weights of the respondents were inflated to account for the weights of the nonrespondents separately for each service. The adjustment was applied independently within nonresponse adjustment groups defined by census region within service. That means the nonrespondents within a group are represented by the respondents in the same group. The same types of nonresponse adjustment will be done for the 15th survey.

B.2.4 Older Americans Act Participant Survey Instruments

The survey consists of telephone interviews with service recipients and caregivers. The interview is structured and will contain specific questions about the mix of services the person has received and his or her assessment of those services. Whenever appropriate, questions will contain predefined categories. Probes will be used to facilitate obtaining complete responses to all the questions. The interviews of caregivers will not include the questions that ask for physical functioning (except health conditions and ADL and IADL limitations of their care recipients) nor the Emotional Well-being and Social Functioning questionnaires. The interviews will last approximately 45 minutes and cover the topics discussed below. This is the same process followed for each of the previous surveys.

1. Nutrition-Congregate Meals: If a respondent receives Congregate Meals, they will be asked a short questionnaire based on the Congregate Meals survey. This questionnaire asks how long they have been attending the congregate meals program; how often they eat at the site; when the last time was they ate at the site; to rate the program; and how much of their food intake the meal provides on the days they eat at the site.

2. Nutrition-Home-delivered Meals: If a respondent receives Home-delivered Meals, they will be administered a short questionnaire based on the Home-delivered Meals survey. This questionnaire asks how long they have been receiving home-delivered meals; how often they receive home-delivered meals; when the last time was they received a meal; to rate the program; and how much of their food intake the meal provides on the days they receive home-delivered meals.

3. Transportation: All service recipients who use transportation services will be interviewed using this survey module. The module asks how long they have been using the transportation; how often they use it; when the last time was they used it; trip purpose; to rate the transportation service; and about the number of times the respondent uses the service.

4. Homemaker/Housekeeping: Questions on the impact of homecare services will be asked of respondents who receive homemaker or housekeeping services. Again, the set of questions is similar to those asked of the other services: how long respondents have been receiving homemaker services; how often they receive homemaker services; when the last time was they used the services; to rate the program; and if they can depend on their aides to do deliver the allotted services.

5. Case Management: Service recipients who receive case management services will be asked questions about their experiences with the program. They will be asked: how long they have been receiving the services; how they would rate the various aspects of the case management services (e.g. ease of contact with the case managers; if the case managers understand their needs, etc.); to rate the services overall and if they contribute to the decisions about their care.

6. Service List: All service recipients will then be asked about the mix of services they receive and the impact of those services. They will also be asked to rate the services overall.

7. Physical Functioning: This module will be asked of all service recipients (except Caregiver clients). This survey module will include questions on: Activities of Daily Living limitations (e.g., difficulty with personal care activities such as bathing and dressing) and Instrumental Activities of Daily Living limitations (e.g., difficulty with such home management activities as meal preparation, shopping, and housekeeping). Questions about the respondents’ health are also being asked, to help with assessing the frailty of the clients served by OAA services. Caregivers will be asked these questions about their care recipients.

8. Emotional well-being: This module will be asked of all participants in the surveys, except caregivers. The questions ask if the respondent has felt sad or depressed, worried or tense, and if they feel that they did not get enough rest, within the last thirty days. They are also asked to describe their overall emotional well-being by responding to a close-ended question (i.e., “Would you say…Excellent, Very Good, Good, Fair, or Poor?”).

9. Social Functioning: All service recipients except caregivers will be asked four questions from the Social Functioning section of the Short-Form Survey (SF-36). These questions ask if the respondent feels his or her social life is adequate and if health concerns have interfered with the ability to participate in social activities.

10. Falls: All service recipients and caregivers will be asked these five questions. The questions ask about falling and losing one’s balance.

11. Social Integration: All service recipients except caregivers will be asked these four questions about social isolation.

12. Life Changes: All service recipients except caregivers will be asked this one question about why they sought out services.

13. National Family Caregiver Support Program Assessment: Caregivers who receive caregiver support services through the National Family Caregiver Support Program will be surveyed as part of the 15th NSOAAP. This module has questions on services offered to caregivers through the National Family Caregiver Support Program, and the impact of those services. There are also questions about services the care recipient receives and satisfaction with and impact of those services; support the caregiver receives, either as part of a formal support group or from other relatives and friends; and what kinds of other information the caregiver would find valuable. The survey asks about the type of help the caregiver provides for the care recipient, the amount of time they provide care, benefits caregiving provides them (companionship, a sense of accomplishment, etc.), drawbacks of caregiving (financial burdens, lack of private time, etc.), and demographic and health information on the care recipient. Three of the questions for this module were adapted from an AARP survey, Caregiving in the U.S⁸

14. Demographic information of the respondent: Demographic information about the respondent will be collected, including type of area of residence (urban, suburban, or rural), Zip Code, education level, race, gender, living arrangements (living alone, with spouse, or with others), and income level. This module will be administered to all participants. The caregiver survey already includes some demographic questions about the care recipient, but the demographic information on the caregiver will be gathered using this demographic module.

15. COVID-19 Module: For 2021 only, all service recipients and caregivers will be asked questions about the effect COVID-19 has had on their access to and use of Older Americans Act programs and services during the past year.

Many of the national survey questions come from such commonly used vehicles as the Survey of Income and Program Participation (SIPP), (e.g., the ADL and IADL questions), the Behavioral Risk Factor Surveillance System (BRFSS) surveys conducted within each state using HHS/CDC standard questions, and other existing surveys. As mentioned previously, other questions were developed under the POMP contract, as well as with input from an advisory panel of aging experts under a Redesign contract. The survey instrument for the 15^th NSOAAP is virtually the same as the instrument used for the previous 14^th NSOAAP.

B.3 Methods to Maximize Response Rates and Deal with Nonresponse

Procedures for Eliciting Cooperation and Maximizing Response Rates among AAAs

Westat will use a similar procedure to select respondents for the 15th National Survey of OAA Participants as it did in the previous seven surveys, which proved very successful. As part of the recruitment procedures, Westat initially contacts the AAAs by sending via email an introductory letter from ACL and an information package about the survey. A copy of these materials is also sent to each State Unit on Aging that had AAAs sampled for the survey. (See Appendix G for the ACL letter sent to the State Units on Aging; Appendix H for the ACL letter sent to the AAAs; and Appendix I instructions/information package sent to the AAAs.) Following up by email and telephone, the Westat research team works closely with each participating AAA to generate numbered lists of clients (using client ID numbers) by service for the client sample frame. The Westat research team uses the numbered lists of client ID numbers for the random selection of the respondents to be interviewed. To complete the random sampling process, Westat research team members enter the total numbers of participants by service into a computer sampling program. The sampling program randomly selects line numbers from the numbered lists of clients. The number of clients to select per service is already entered into the program. Westat informs the AAAs of the selected line numbers. The AAAs then provide the participant names and telephone numbers associated with those line numbers to Westat.

Westat research assistants serving as recruitment specialists will encourage the participation of all selected agencies by establishing rapport with contacts within each agency, coaching them on how to generate their client lists, and assuring them that the time involved for them to complete the participant selection procedures will be minimal. For agencies that refuse to participate, Westat will send them a refusal conversion letter (already developed for the previous surveys), and call them one more time to try to gain their cooperation. Once an agency refuses a second time, Westat will not try to contact them again. Westat will also work with ACL to communicate with ACL Regional Administrators for help in persuading reluctant AAAs to participate. Additionally, to promote agency participation, we plan to gain the endorsement and support of the National Association for Agencies on Aging (N4A), as well as the Advancing States [formerly known as the National Association of States United for Aging and Disabilities (NASUAD)].

To reduce the burden for the AAAs, Westat works with software vendors of commercial client tracking software programs commonly used by AAAs to develop step-by-step instructions for creating numbered lists of client ID numbers by service. By using agency-assigned client ID numbers to generate numbered lists of clients for the participant sample frame, Westat is able to screen the lists for duplicate client entries. Additionally, the use of agency-assigned client ID numbers helps to decrease the amount of personally identifiable client contact information collected by Westat during the survey.

Numbered lists will be developed for the following services: home delivered meals, congregate meals, transportation, case management, and homemaker services, as well as caregivers who are served by the National Family Caregiver Support Program.

To ensure a high participant response rate, Westat will send participants who are eligible for the telephone survey a letter before they are contacted by an interviewer. The letters will be on each ACL’s letterhead, with toll-free phone numbers provided to answer participants’ questions. Westat will attempt to contact participants at different times of the day and different days of the week to maximize the possibility of contact. Westat is also experienced in refusal conversion procedures, having achieved a refusal conversion rate of 40 percent for the earlier surveys. (See Appendix F)

B.4 Tests of Procedures or Methods to Be Undertaken

As discussed in earlier sections, the individual service modules and the modules on physical functioning, quality of life, and demographics have all been field tested and validated by the POMP participants. For example, for each module the POMP grantees drew samples of service recipients, administered the modules, and analyzed the data. The POMP grantees revised the items on the modules based on the results of the field tests and validity studies. Some new questions added for the 14^th NSOAAP (Life Changes, Social Integration, Falls) were developed by an advisory panel of aging experts and cognitively-tested under a separate NSOAAP Redesign contract.

The majority of the items on the survey instrument for the 15th National Survey are from the previous survey instruments. Over the years, several items have been removed from the survey instrument because of ambiguity of the wording or in cases where the results of the item showed no variation across response options.

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The use of statistical sampling methods is critical to this survey. Under the supervision of ACL, Westat is responsible for selecting the sample, conducting the interviews, data weighting and data analysis. Below are the names and contact information of individuals responsible for the statistical aspects of the study and individual collection and/or analysis of the data.

Administration for Community Living/Administration for Aging Personnel Responsible for Deliverables

Susan Jenkins, Ph.D.

Director, Office of Performance and Evaluation

Administration for Community Living

U.S. Department of Health and Human Services

300 C Street, SW, Rm 1243

Washington, DC 20201

Tel: 202-795-7369 

[email protected]

Westat Staff

Dwight Brock, Ph.D. – Role: Senior statistician overseeing study design, development of the sampling plan, weighting, and data analysis

Senior Statistician

Westat

1600 Research Blvd.

Rockville, Maryland 20850

301-517-4026

[email protected]

Joanne Campione – Role: Project Director

Senior Study Director

Westat

2530 Meridian Pkwy, Suite 200

Durham, NC 27703

919-474-3506

[email protected]

Katie Hubbell– Role: Senior systems analyst with involvement in sampling data weighting, data analysis, and reports.

Senior Systems Analyst

Westat

1600 Research Blvd.

Rockville, Maryland 20850

301-294-2020

[email protected]

Robin Ritter – Role: Recruitment Director with involvement in design, sampling, AAA and respondent recruitment, and data analysis

Senior Study Director

Westat

1600 Research Blvd.

Rockville, Maryland 20850

301-240-314-5804

[email protected]

Jacqueline Severynse – Role: Design of the sampling plan, weighting, and data analysis.

Senior Statistician

Westat

1600 Research Blvd.

Rockville, Maryland 20850

301-517-8058

[email protected]

Lena Watkins – Role: Telephone Operations Manager

Westat

1600 Research Blvd.

Rockville, Maryland 20850

301- 294-3938

[email protected]

1 http://www.gao.gov/key_issues/managing_for_results_in_government/issue_summary

2 https://acl.gov/sites/default/files/about-acl/2020-04/Older%20Americans%20Act%20Of%201965%20as%20amended%20by%20Public%20Law%20116-131%20on%203-25-2020.pdf

3 https://acl.gov/programs/performance-older-americans-act-programs

4 https://www.census.gov/programs-surveys/acs/methodology/sample-size-and-data-quality/response-rates-definitions.html

⁵ https://www.insightsassociation.org/issues-policies/best-practice/casro-definition-response-rates

5

6 State units also receive a letter with a list of AAAs selected in the state.

7⁷ This percent range refers to the client response patterns that may occur; for example, in a yes/no question, it refers to the expected percent of respondents who will answer yes, versus no.

8 National Alliance for Caregiving and AARP (2004, April). Caregiving in the U.S. Appendix C, pp. 16-17 retrieved from AARP Web site: http://assets.aarp.org/rgcenter/il/us_caregiving.pdf

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	7420.01: OMB Package. Section A. Introduction
Author	MARKOVICH_L
File Modified	0000-00-00
File Created	2022-05-04