0023 NSOAAP SS Final

Application for Data Collection:
Supporting Statement for
15th National Survey of
Older Americans Act Participants

Updated
April 1, 2021

Table of Contents
Chapter
A.

Page

JUSTIFICATION ..................................................................................

1-1

A.1
A.2
A.3
A.4
A.5
A.6
A.7
A.8

1-1
1-5
1-5
1-7
1-8
1-8
1-8

A.9
A.10
A.11
A.12
A.13
A.14
A.15
A.16
A.17
A.18

B.

Circumstances Making the Collection of Information Necessary
Purpose and Use of the Information Collection .........................
Use of Improved Information Technology and Burden Reduction
Efforts to Identify Duplication and Use of Similar Information
Impact on Small Businesses or Other Small Entities.................
Consequences of Collecting the Information Less Frequently ..
Special Circumstances ...............................................................
Comments in Response to the Federal Register Notice and
Efforts to Consult Outside the Agency ......................................
Explanation of Any Payments or Gift to Respondents ..............
Assurances of Privacy Provided to Respondents .......................
Justification for Sensitive Questions .........................................
Estimates of Annualized Burden Hours and Costs ...................
Estimates of Other Total Annual Cost Burden to
Respondents and Record Keepers ..............................................
Annualized Cost to the Federal Government .............................
Explanation for Program Changes or Adjustments ...................
Plans for Tabulation and Publication and Project Time
Schedule ....................................................................................
Reason(s) Display of OMB Expiration Date is Inappropriate ...
Exception to Certification for Paperwork Reducation
Act Submissions.........................................................................

COLLECTION OF INFORMATION EMPLOYING STATISTICAL
METHODS ...........................................................................................
B.1
B.2
B.3
B.4
B.5

Respondent Universe and Sampling Methods ...........................
Procedures for the Collection of Information ............................
Methods to Maximize Response Rates and Deal with
Nonresponse ...............................................................................
Test of Procedures or Methods to be Undertaken ......................
Individuals Consulted on Statistical Aspects and Individuals
Collecting and/or Analyzing Data ............................................

1-8
1-13
1-13
1-15
1-16
1-17
1-17
1-17
1-18
1-19
1-19

2-1
2-1
2-5
2-13
2-14
2-15

Table of Contents
(continued)

Exhibits

Page

A-1
A-2
A-3

Estimated Hour and Annual Cost Response Burden .............................
Total Annualized Cost to the Federal Government ..............................
Data Collection Timetable .....................................................................

1-16
1-17
1-18

B-1

Respondent Universe .............................................................................

2-3

Tables

Page

A-1

Federal Register Comments and ACL Responses .................................

B-1

Half-widths of 95 percent confidence intervals by various sample sizes
and estimates of target characteristics...................................................
Half-widths of 95 percent confidence intervals for the difference
between two estimates by various sample sizes and for various
averages of the two estimates ................................................................

B-2

1-9

2-8

2-10

Appendices (separate document)
A
B
C
D
E
F
G
H
I
J
K

Pertinent Legislation
Instructions for Creating Numbered Client Lists for Sampling
Instructions for AAA Access to the Survey Website & How to Submit Data
Federal Register Notice Published by ACL/AoA for the Proposed
Information Collection
Westat Assurance of Confidentiality Agreement
Notification Letter for Selected Clients
Notification Letter for State Units on Aging
Notification Letter for Area Agencies on Aging
Agency Information Packet
Survey Instrument/Questionnaire [see separate document]
COVID Module for 2021 [see separate document]

A.

Justification

A.1

Circumstances Making the Collection of Information
Necessary

Introduction
This OMB package requests clearance to conduct an annual cross sectional survey of Older Americans
Act (OAA) participants. The survey will be the 15th in a series of national surveys of OAA clients. The
first 14 surveys provided important cross-sectional data on service recipients (e.g., consumer assessment
of services, reported outcomes, physical functioning, quality of life, and demographic information). The
survey will continue to provide rich cross-sectional data.
This survey has remained essentially the same since the last OMB action on 8/20/2019 (OMB Control
No: 0985-0023). The sampling methodology is been updated to be more precise, modifying the Stage I
sampling design to utilize agency client counts rather than budget size. The data collection procedures are
identical to the previous survey approved in 2017. The survey questionnaire is unchanged.
The 15th survey originally scheduled to be conducted in 2020 was postponed due to the COVID-19
pandemic and disruptions in service delivery provided by the aging network. As part of a nationwide
effort to slow transmission of COVID-19, communities have closed locations such as senior centers, adult
day care centers, and congregate meal facilities where many older adults received vital home- and
community-based services. In addition, many of the in-home services that help older adults to remain in
their homes, such as home-delivered meals, homemaker services, and respite care have been curtailed due
to fear of spreading COVID-19.
This, combined with families and friends unable to assist loved ones due to the fear of spreading the virus,
have left many older adults isolated and without the programs and services they need to remain living
safely in the community. Consequently, ACL is requesting the addition of a one-time module on COVID19. This module supplants the module on Emergency Preparedness that was included in the 60-day FRN
due to the ongoing health crisis older adults are experiencing as a result of COVID-19. The Emergency
Preparedness module will be added to the 2022 collection instrument. The purpose of adding questions on
COVID-19 to the NSOAAP is to measure the effect the pandemic has had on older adults’ access to and
use of Older Americans Act programs and services during the past year.

1-1

ACL/AoA’s Strategy of Program Improvement
The Administration for Community Living’s Administration on Aging (ACL/AoA) has an ongoing
strategy of program improvement through enhanced program performance measurement, in compliance
with requirements of the Office of Management and Budget’s (OMB) program reviews, the GPRA
Modernization Act of 2010 (GPRAMA), and the OAA Section 202(f), by proposing to conduct
further studies of program outcomes (see Appendix A for the pertinent legislation).
Previously, ACL/AoA conducted 14 cross-sectional surveys. The 14 surveys and their OMB control
numbers are listed below:















Two pilot studies of Older Americans Act Title III Service Recipients in 2003 and 2004 (OMB
control numbers 0985-0014 and 0985-0017);
Third National Survey of OAA Title III Service Recipients conducted in 2005 (OMB control
number 0985-0020);
Fourth National Survey of OAA Title III Service Recipients conducted in 2008 (OMB control
number 0985-0023);
Fifth National Survey of OAA Title III Service Recipients conducted in 2009 (OMB control
number 0985-0023).
Sixth National Survey of OAA Title III Service Recipients conducted in 2011 (OMB control
number 0985-0023).
Seventh National Survey of Older Americans Act Participants conducted in 2012 (OMB control
number 0985-0023).
Eighth National Survey of Older Americans Act Participants conducted in 2013 (OMB control
number 0985-0023).
Ninth National Survey of Older Americans Act Participants conducted in 2014 (OMB control
number 0985-0023).
Tenth National Survey of Older Americans Act Participants conducted in 2015 (OMB control
number 0985-0023).
Eleventh National Survey of Older Americans Act Participants conducted in 2016 (OMB control
number 0985-0023).
Twelfth National Survey of Older Americans Act Participants conducted in 2017 (OMB control
number 0985-0023).
Thirteenth National Survey of Older Americans Act Participants conducted in 2018 (OMB
control number 0985-0023).
Fourteenth National Survey of Older Americans Act Participants conducted in 2019 (OMB
control number 0985-0023).
The Fifteenth National Survey of Older Americans Act Participants was scheduled to be
conducted in 2020, but it had to be postponed until 2021 due to the coronavirus pandemic and the
resulting disruption in delivery of services to OAA recipients.

1-2

The surveys have enabled ACL/AoA to establish baselines and performance targets for annual and longterm outcome measures required by OMB and incorporate new performance information in agency
budget justifications and performance plans. Further, the studies demonstrated that services provided
under Title III:





Are effectively targeted to vulnerable populations
Are provided to individuals who need the services
Are highly rated by recipients (quality)
Provide assistance that is instrumental in enabling recipients to maintain their independence.

With this submission, we are requesting OMB approval to conduct a fifteenth national survey, as
well as sixteenth and seventeenth surveys if approved for three years.

Performance Measurement Requirements
GPRAMA1 requires federal agencies to develop annual and long-term performance outcome measures
and to report on these measures annually. Section 203(f) of the OAA2 requires ACL/AoA to work
collaboratively with State agencies and area agencies on aging (AAAs) to develop performance outcome
measures.
Since the passage of GPRA in 1993, ACL/AoA has accepted GPRA and GPRAMA as an opportunity to
document each year the results that are produced through the programs it administers under the authority
of OAA. It is the intent and commitment of ACL/AoA, in concert with State and local program partners,
to use the performance measurement tools of GPRAMA to continuously improve OAA programs and
services for older adults.
As described on ACL’s website: “The National Surveys of OAA Participants are a collection of annual
national surveys of recipients of select Title III services. Their purpose is to obtain performance outcome
measurement information, which is then used in AoA’s GPRA plan and PART assessment. The survey
instruments focus on consumer assessment of service quality and consumer-reported outcomes. The
instruments also measure special needs characteristics such as physical and social functioning of the
people who receive services.” 3

OAA, Title III – Home and Community-Based Program
Title III of the OAA establishes a home and community-based care program for older persons and their
caregivers, to enable them to live as independently as possible for as long as possible. States and local
agencies are given much latitude to design services tailored to the needs of their regions and communities.
1

http://www.gao.gov/key_issues/managing_for_results_in_government/issue_summary

2

https://acl.gov/sites/default/files/about-acl/202004/Older%20Americans%20Act%20Of%201965%20as%20amended%20by%20Public%20Law%20116-131%20on%203-25-2020.pdf

3

https://acl.gov/programs/performance-older-americans-act-programs

1-3

One challenge for ACL/AoA is to devise a means to improve the performance of the program nationally,
while preserving and promoting the diversity of program design. ACL/AoA has chosen to work toward
improved program performance throughout the Aging Services Network by working collaboratively with
States and AAAs to develop performance outcome measurement tools. The tools identify elements of
service quality so that states and AAAs can improve service systems at the local level. These same tools
can also be employed by ACL/AoA to measure program performance at the national level.

Performance Outcomes Measures Project (POMP)
For over 10 years, ACL/AoA sponsored the Performance Outcomes Measures Project (POMP)
demonstration, in which grants were awarded to states, who then worked collaboratively to develop
survey instruments that measured elements of service quality and consumer reported outcomes for various
services provided under Title III of the OAA. Surveys were developed for the following topics:
Service Domains:
 Nutrition (including congregate and home-delivered meals)
 Transportation
 Information and Assistance
 Homemaker/Housekeeper
 Personal Care
 Caregiver Support
 Case Management
 Senior Centers
Client Characteristics:
 Physical Functioning
 Demographics
 Emotional Well-Being
 Social Functioning
POMP demonstrated the ability of states and AAAs to apply statistically sound sampling techniques to
obtain numeric measures of program performance.
The survey instruments developed under POMP – along with various tools necessary for implementation
– can be found at https://acl.gov/programs/pomp. These performance measurement surveys have enabled
some local agencies to obtain additional financial support and improve program management. Examples
of uses of performance measurement at the state and local level follow:




The Hawkeye Valley Area Agency on Aging in Waterloo, Iowa compiled information on the
level of client support and satisfaction with services and received additional funding from the
United Way for exemplary programs.
The Area Agency on Aging in Cincinnati, Ohio expanded the use of Home Care Client
Satisfaction Measure (HCSM) and incorporated it into an ongoing part of its case
management process for all clients to improve service quality.
The Florida Department of Elder Affairs developed a computer simulation model that
demonstrated the impact of home care programs on reducing nursing home admissions and
showed the savings in Medicaid funds

1-4

The recipient and caregiver survey instruments national survey are based on the recipient surveys developed
by POMP grantees.

A.2

Purpose and Use of the Information Collection

The results of this information collection will be used to:





Report on annual performance results as required by OMB.
Provide national benchmarks for use by states and AAAs.
Provide secondary data for analysis of various Title III program evaluations.
Provide performance information for key demographic subgroups, geographical
subregions, and different types of AAAs which will enable ACL/AoA to identify
variations in performance and examine the need for additional targeted technical
assistance.

The data will be used by the Administrator of the Administration for Community Living/Assistant
Secretary for Aging in testimony and presentations; it will be incorporated into the agency’s Annual
Report; and it will be used by program staff to identify areas that may need attention at the national level.
For example, the ACL nutritionist is interested in examining nutritional intake information by key
population subgroups to identify potential areas for technical assistance initiatives.

A.3

Use of Improved Information Technology & Burden
Reduction

Use of Client Tracking Software to Generate Client Lists for Sampling
The proposed procedures and materials requesting information from the agencies, as well as the telephone
surveys of respondents, have been designed in a way that minimizes respondent burden.
To reduce the burden for the Area Agencies on Aging (AAAs), the contractor (Westat) has developed
procedures for client sampling that utilize the same client tracking management information systems that
are used by States and AAAs to create the required State Program Reports for ACL/AoA. Since the
implementation of the fourth national survey in 2008, the contractor (Westat) has worked cooperatively
with vendors of commercial off-the-shelf client tracking software programs most commonly used by the
State and Area Agencies on Aging to develop step-by-step instructions for the AAAs to use to generate
client lists by service to use for a sample frame. It is estimated that over 95% of the AAAs now have this
technological capability and are able to follow the instructions to produce their client lists by service. We
will provide similar instructions for the 15th survey. Appendix B contains an example of instructions
created for agencies which use a commercial client tracking software system known as “PeerPlace.”
In specific states that have their own proprietary client tracking software, the contractor (Westat) has
worked directly with an Information Technology Specialist at the State-level to generate electronic client

1-5

lists for all of the AAAs selected for the national survey. This further reduces the burden for AAAs in
states that have their own proprietary software.

Use of Survey Web Site
A National Survey web site application https://aoasurvey.org has been developed to support and assist
with data collection. For the 5th-14th surveys, the contractor (Westat) designed and utilized a secure
website which the AAAs used to upload their lists of selected clients. That website will be updated and
further refined for the 15th national survey.
The web site is divided into two major sections: the public and the restricted-access sections. The public
section is accessible to the general public, without restrictions. It includes background information,
frequently asked questions, and links to results of previous AoA National Surveys. The purpose of the
public section is to provide State and Area Agencies on Aging, professionals in the field of aging, and
service recipients and their families with information about the data collection effort and uses of the data.
The restricted-access section of the web site houses an electronic records receipt system. Area Agencies
on Aging have the option of submitting private personally identifiable client data to Westat via electronic
files using the project web site. The web site was written in Active Server Pages (ASP), HTML, and
JavaScript and uses the industry-standard TLS (Transport Layer Security) 1.1/2 encryption for secure data
submissions. To further enhance security, it will be upgraded to a newer platform prior to the start of the
15th national survey. Agencies choosing this option will receive usernames and passwords that enable
their staff to sign on to the file upload utility on the web site. This system supports files in a large variety
of file formats. Each agency's data file will be processed according to its structure and content.
Westat programming staff will manually map and convert the data items in each agency’s file to
create standardized records for further processing. As each file is received, this system will log the source
agency, date received, and file type.
Only agencies that have been selected to participate in the survey will have access to this area. Unique
user IDs and passwords will be assigned to each AAA at the time they are selected into the sample. The
ID and password will be provided with other survey materials to the AAA.
Appendix C contains for instructions for AAA restricted access to the survey website and how to submit
data.

Use of Computer Assisted Telephone Interviewing (CATI)
Westat (the contractor) will use computer-assisted telephone interviewing (CATI) technology to conduct
the surveys of OAA service recipients and record the responses. Westat’s CATI capability includes
customized software systems for scheduling, interviewing, and data handling and utilizes high-speed
data networks and centralized voice and data monitoring. A single database is used to monitor and direct
the interviewers. The Scheduler, a computerized survey control system, makes interviewer assignments,
records the disposition of sample cases, and helps survey managers monitor performance.
Westat will attempt to contact each person in the sample, making multiple calls at different times and
days when necessary. To reduce the burden for the respondents, Westat will schedule appointments for

1-6

calls at times that are convenient for them. Respondents will also have the option of calling a toll-free
phone number to reach an interviewer and taking the survey at their convenience. For Spanish-speaking
respondents, Westat uses specially trained bilingual interviewers to conduct the interviews in Spanish. If
other special arrangements are necessary (e.g., interpreter, proxy needed, mail out requested, interview
needed to be conducted over several sessions), the respondent can be further accommodated.
Westat will take the ACL/AoA-approved finalized version of the survey instruments and program them
into its CATI system. This involves:


Inserting specifications into the English version of the questionnaire;



Preparing the specifications for the CATI programmer;



Translating the questionnaire from the specifications into Spanish; and



Programming and testing both versions of the questionnaire into CATI.

Details of how skips will work in the questionnaire are included in the design document, as are the needed
question variations. For example, some questions may need to be asked differently, depending on the
answers to previous questions. In particular, if a respondent told us they live with others, the next question
we would ask would be, “Do you live with your spouse?” However, if the respondent told us they lived
alone, the follow-up questions will not be asked, and CATI will automatically skip to the next question.
The use of the CATI system in combination with Westat’s highly structured telephone interviewer
training and procedures ensures that interviewers conduct the surveys in a professional, controlled, and
consistent manner.

A.4

Efforts to Identify Duplication & Use of Similar
Information

Every effort is being made to avoid duplication and minimize respondent burden. Over the last 17 years,
Westat conducted the first through 14th National Surveys of Older Americans Act Participants, formerly
known as the National Survey of OAA Title III Service Recipients. As a result of the information
gathered, modifications have been made to the data collection procedures and to the survey instruments.
We believe we have reduced agency and respondent burden to the minimum level possible to achieve the
survey's objectives.
In addition, ACL is careful not to duplicate data collection efforts between performance measurement and
evaluation efforts. The National Survey data has been used as a secondary data source in a current
evaluation effort. When future evaluation efforts require independent data collection, that data collection
will be coordinated with the performance measurement surveys.
The NSOAAP is not duplicative of other survey efforts because there is no other representative survey of
Older Americans Act participants. The HRS (Health and Retirement Study) collects nationally
representative data on older adults every two years. The HRS is not able to identify the experience of

1-7

OAA service recipients. The NSOAAP is a random sample of Older Americans Act (OAA) service
recipients only, and cannot be used to make assertions about the American population of older adults. The
purpose of NSOAAP is to obtain performance outcome information that demonstrates the effect of
services and illustrates client reported quality of service. In addition, service recipient demographics,
health and wellbeing indicators are collected. However, the two survey efforts are complementary.
Utilizing the HRS as a means to compare OAA service recipients to a nationally representative sample of
older adults can help ACL better understand its program participants vis-à-vis a nationally representative
sample.

A.5

Impact on Small Businesses or Other Small Entities

The data sources affected by the survey covered by this request for review will be agencies of state and
local government, public purpose, quasi-governmental agencies, and clients who are private citizens. We
have designed the sample to minimize the burden on both the agencies and the client respondents.

A.6

Consequences of Collecting the Information Less
Frequently

This proposed fifteenth national survey will be conducted one time. If we are unable to conduct this survey
we will be unable to move forward with our program performance measurement strategy as required by the
OAA. We will be unable to pursue outcome measurement for the ACL/AoA Strategic Action Plan and our
PART assessments will be compromised, especially since this survey format was developed to allow an
overall assessment of Title III, while also continuing to allow assessment of individual services offered by
Title III. We will be unable to report our FY 2021 consumer assessment performance measures and targets
for GPRA and PART reporting purposes. The proposed sample sizes will allow us to analyze results by
subgroup (e.g. region, age, race) and, therefore, allow better targeting of services.

A.7

Special Circumstances

The data collection effort will be conducted according to the guidelines specified in 5 CFR § 1320.6. No
special circumstances are known that would cause inconsistency with these guidelines.

A.8

Comments in Response to the Federal Register Notice &
Efforts to Consult Outside the Agency

Comments in Response to the Federal Register Notice

1-8

A 60-day Federal Register Notice published in the Federal Register on November 17, 2020, 85 FR 222
https://www.govinfo.gov/content/pkg/FR-2020-11-17/pdf/2020-25276.pdf (see Appendix D). A 30-day
Federal Register Notice published in the Federal Register on April 1, 2021, 86 FR 17153.
ACL received comments from two organizations and from two individuals about the NSOAAP. One
organization submitted multiple (10 comments). ACL reviewed all of the comments. The comment from
one of the individuals was not relevant. For ease of review, the remaining comments and their responses
have been grouped by topic or issue. The ACL responses for each topic/issue are detailed in Table A-1:

Table A-1

60-Day Federal Register Comments and ACL Responses

Topic/Issue
Food insecurity
in older adults

Comment
“We support the inclusion of the USDA module to
provide national estimates of the rate of food
insecurity among OAA program participants. We
recommend this module be continuously included in
future administration of this survey.”
“We suggest that ACL consider adding malnutrition
screening questions in addition to the USDA
module’s food insecurity questions, such as:
• Do you ever eat only one meal daily?”
• Do limits on chewing, swallowing or physical
mobility ever prevent you from eating your homedelivered meals, even though you may be hungry?”
• Do limits on chewing, swallowing or physical
mobility ever prevent you from getting to your local
congregate meal site and eating your meal, even
though you may be hungry?”

ACL Response
ACL concurs and plans to
maintain the USDA module and
associated questions for the
foreseeable future.

Medicallytailored meals
for participants
requiring
special diets

“We suggest that ACL consider adding a question
about the need for therapeutic diets or texturemodified meals to better understand the needs of
participants as it relates to medically tailored
meals.”

ACL recognizes the importance of
capturing data on the capability
of OAA nutrition programs to
accommodate special diets. ACL
will call upon nutrition experts to
make recommendations and
inform a redesign of future
NSOAAP collection efforts.

Ethnic and
cultural
barriers to
communication

“We suggest that ACL consider including a question
about communication barriers to both the
congregate and home-delivered meals modules,
such as:

ACL recognizes that ethnic and
cultural barriers may affect
participants in the nutrition
programs. ACL will call upon
nutrition experts to make

Risk of
malnutrition

1-9

ACL recognizes the importance of
reducing and assessing risk for
malnutrition. However, further
deliberation is needed to ensure
that we select the most
appropriate and universally
accepted language. ACL will call
upon the expertise of a nutrition
workgroup to make
recommendations to ACL on
selecting the best language to
use. The workgroup’s
recommendation will inform a
redesign of future NSOAAP
survey collection efforts.

and nutritional
preferences

Nutrition
counselor:
Rephrase
question to
improve clarity

Modify item
response in
nutrition
modules to
include
positive as well
as negative
changes.
Consistent
language in
meals
program.

• Do you have language or cultural barriers to
talking with staff at your congregate meal site/ your
home delivery staff?

recommendations and inform a
redesign of future NSOAAP
collection efforts.

We also suggest that ACL consider adding a question
to both the congregate and home-delivered meals
modules about meals meeting cultural preferences.
“SVC1(k) asks whether the respondent has access to
a “nutrition counselor” who is providing dietary
advice based on the respondent’s condition,
medications, and related factors. We question why
the survey would not specify “a qualified nutrition
professional such as a registered dietitian” (or
registered dietitian nutritionist), since these
professionals are the most qualified to answer such
questions. Moreover, in 28 states, only licensed
professionals are legally eligible to provide such
advice. The term “nutrition counselor” allows for
substantial subjective interpretations, and could
theoretically include food service staff or other
program participants who may be providing such
advice against state law.”
“We note that the survey asks about changes in
meals, but almost all coding options for the
interviewer are about reductions or negative
changes with few opportunities for interviewers to
code any positive changes reported by participants.
In addition to coding for both reductions and
improvements in quality of the food, we
recommend adding the corresponding “positive”
option for all other codes. “
“Ensure that consistent language is used to describe
the program, particularly for congregate or senior
dining meals. Some questions use the term “meals
program” while others use the term “lunch
program” or, generically, “this service”, including
CNR20-23 where three different terms are used
across four consecutive questions. This also applies
to SVC1 question in Additional Services module. We
recommend using the term “meals program” unless
exclusively referring to lunch, as some programs
serve breakfast or dinner meals rather than lunch
meals.”

ACL is considering a special
topical module related to equity
and underserved populations.
ACL recognizes “nutrition
counselor” may offer subjective
responses. However,
respondents may not know if the
qualifications of the person
providing dietary advice. ACL will
call upon nutrition experts to
make recommendations and
inform a redesign of future
NSOAAP collection efforts. The
use of terminology for this item
will be tested.

1-10

ACL concurs with this suggested
change. The requested change
has been made to the survey
instrument.

ACL recognizes that the language
used to describe the meals
programs may need to be revised
to be more consistent. ACL will
call upon nutrition experts to
make recommendations and
inform a redesign of future
NSOAAP collection efforts.
The term “meals program”
throughout due to the variety of
food services during the COVID19 pandemic.

Living
independently

“SVC3 asks about continuing to “live independently”
vs. “living at home” (as they do in CS15, CNR23,
HNR28, HC9 and TR20). The terminology “living
independently” is preferred. Simply continuing to
live “at home” does not mean that the person is
living independently, is living in their own home (vs
that of a relative), or has autonomy over where he
or she lives.”
“SVC3(b) should be more specific with regard to the
context of “secure.” We are unsure whether the
context is financial, food-based, or related to
physical safety.”

ACL concurs with this suggested
change. The requested change
has been made to the survey
instrument.

Multiple meals

“HNR5 should be re-phrased to reflect the fact that
many HDM clients receive more than one meal and
may consume multiple HDMs in a day.”

Care recipients
under age 60
with dementia

The 2020 reauthorization of OAA allows “individuals
living with dementia under the age of 60 to access
certain OAA supports like nutritional services and
respite care through the National Family Caregiver
Support program. We respectfully request that ACL
analyze the data collected on these younger
individuals and the services they receive so that
those services can be tailored and improved.”
The advocacy organizations “support ACL's request
to add an emergency preparedness module to the
survey…”.

ACL recognizes that the question
may need to be modified. ACL
will call upon nutrition experts to
make recommendations and
inform a redesign of future
NSOAAP collection efforts.
While ACL recognizes the value of
collecting data on care recipients
of the NFCSP who are under 60,
that is not the focus of the
NSOAAP. The focus of the
NSOAAP is the service recipients,
that is, the caregivers.

Context of
“secure”

Emergency
preparedness

Question on
participant sex
(gender
identity)

“I encourage the modification of the questions
about participant sex to include response options
(which are recorded but not verbally offered as
options) to include "transgender" and or "other"
with a write-in option.
The questions this relates to are: DE1 (DEGENDR);
CGC (CGPMF); CG83 (CGPMF) - both of the last have

1-11

The question refers to how
participants feel about additional
services that they or their care
recipient receive.

ACL recognizes that the use of
the word “secure” may be
ambiguous. ACL will call upon
experts in the aging network to
make recommendations to
inform a redesign of future
NSOAAP collection efforts.

ACL is not going to add this
module for the 2021
administration but still plans to
use it in the future.
In order to improve demographic
data collection related to the
LGBT community, ACL is
supporting the Measuring Sex,
Gender Identity, and Sexual
Orientation for the National
Institutes of Health an ad hoc
panel of the National Academies

the same variable labels and this should be
corrected in the survey construction.”

of Sciences, Engineering, and
Medicine which will review
current measures and the
methodological issues related to
measuring sex as a non-binary
construct, gender identity, and
sexual orientation. The panel will
produce a consensus report
which is expected in December
2021. ACL will use the report as a
foundation for testing new
survey questions and
administrative data elements.
The variable label in the survey
instrument for CGPMF was
corrected to RGENDER.

Efforts to Consult Outside the Agency
For updates to the 2019 survey instrument, ACL/AoA called upon the expertise of a work group to review
NSOAAP data collection tools and to make recommendations to ACL on selecting the best language to
use for revising questions in the survey instruments. The NSOAAP work group was comprised of experts
on aging data and survey methodology. The work group’s recommendation will inform a redesign of
future NSOAAP survey collection efforts.
For the addition of a special one-time 2021 module on questions related to COVID-19, ACL/AoA
convened a new advisory workgroup consisting of members from different State Units on Aging (SUAs),
Area Agencies on Aging (AAAs), academia and advocacy organizations on nutrition, aging, and family
caregiving. The advisory workgroup members were divided into three subgroups focusing on nutrition,
well-being and other services (transportation, case management, and homemaker services), and family
caregiving. Workgroup members discussed, evaluated, and ranked proposed COVID-related questions.
Through the result of the workgroup members’ voting and prioritization, a final selection of 10-13
questions by service category resulted in the final COVID-19 special module for 2021.
The majority of the remaining questions in the survey instruments for this proposed information
collection are based on those developed by ACL/AoA POMP grantees representing State Units on Aging
and AAAs. POMP grantees who have worked on the survey instruments include state and local level
representatives from Arizona, Florida, Georgia, Massachusetts, New York, North Carolina, and Ohio. The
development of the survey instruments has been an iterative process. There were no areas of disagreement
during the latest POMP revisions.
The POMP grantees tested the instruments with service recipients at the local AAA-level using several
methods:
1. Field-tested the survey instruments with a sample of service recipients and revised the
instruments based on their experience.

1-12

2. Conducted cognitive testing to ensure that the items on the survey instruments were interpreted as
intended.
3. Conducted validity testing on the survey instruments.
Westat (the contractor) has also consulted representatives from different State Units on Aging to develop
and test the instructions and procedures for generating client lists used for sampling. The state
representatives who have reviewed the instructions and procedures include:




Robin Tofil, Connecticut Department of Social Services, Aging Services Division
Jim Burd, Pennsylvania Department of Aging
Leonard Eshmont, Virginia Department for the Aging

A.9

Explanation of Any Payment or Gift to Respondents

No payments or gifts will be given to respondents.

A.10

Assurance of Privacy Provided to Respondents

After review, ACL has determined that the current Privacy Impact Assessment (PIA) meets the
established guidelines. There are no assurances of confidentiality. With that, privacy and anonymity are
important parts of the survey design. In response to this concern, ACL will ensure the anonymity of all
individuals who provide data. A pledge of privacy and anonymity is a major positive incentive for
potential respondents to participate in the survey. Its absence would be a significant deterrent and could
create complications in implementing the survey.
Westat in contract with ACL will take the following precautions to ensure the privacy and anonymity of
all data collected:


All Westat project staff, including recruitment specialists, telephone interviewers,
research analysts, and systems analysts, will be instructed in the discolousre
requirements of the survey and will be required to sign statements affirming their
obligation to maintain privacy;



Only Westat staff who are authorized to work on the National Survey have access to
client contact information, completed survey instruments, and data files.



Data files that are delivered will contain no personal identifiers for program
participants; and



Analysis and publication of survey findings for the participant survey will be in terms
of aggregated statistics only.

1-13

Appendix E presents the internal corporate “Assurance of Confidentiality Agreement” all Westat project
staff must sign. This agreement requires the signer to keep confidential and private an unity and all
information about individual respondents to which they may gain access. Any Westat employee who
violates this agreement is subject to dismissal and to possible civil and criminal penalties.
Westat, the contractor for administering the survey instrument and collecting the data, has extensive
experience in protecting and maintaining the privacy of respondent data collected from surveys. To
ensure privacy, Westat has drawn from its experience in designing the data collection procedures
incorporated in this program. In addition to the corporate Assurance of Confidentiality Agreement,
Westat has implemented several other procedures to protect privacy of survey participants.
1. Data is saved on secure network folders only accessible to authorized users. No data is ever
stored on laptop computers. At the end of the survey, all private data is permanently deleted.
2. For the 15th National Survey and subsequent follow-up surveys, AAAs will be instructed to
submit private personally identifiable client data to Westat via electronic files using the secure
survey web site. Agencies will receive usernames and passwords that enable their staff to sign on
to the file upload utility on the web site. The passwords are created by a password generator
which creates random passwords that are highly secure due to a combination of lower and upper
case letters, numbers and punctuation symbols. The database containing the client survey data is
not accessible via the Internet; it resides on a server inside the Westat firewall. Only Westat Data
Collection Program staff members have access to the master survey database.
3. For AAAs that may experience problems with the survey website and wish to send client data
electronically by email, we instruct the AAAs to password protect the file containing the data.
Password protection of client data sent electronically by email is required not only for
transmission between the AAA and Westat, but even internally within Westat. Additionally, we
provide the AAAs with an email address to a secure dedicated project email box
([email protected]) which cannot be accessed remotely.
4. For the small number of AAAs that are not able to generate client records by service
electronically, they can submit client information in a hard copy format (fax, FedEx, U.S. Postal
Service). Hard copies of client information are stored in locked filing cabinets within a locked
room. At the conclusion of the survey, all hard copies of client data are shredded.
5. A secure fax machine dedicated solely to this survey is used to receive faxes from AAAs that
choose to transmit their data by fax. The fax machine is located within a locked project room.
AAAs that need to transmit their data by fax are asked to call to Westat staff to alert them to
watch for and intercept an incoming fax. If the fax machine is busy, it does not roll over to any
other fax machine.

All respondents in this data collection effort are assured of the privacy of their answers. Respondent data
are aggregated and estimates are produced and published at the both at the national level and at the
geographic regional or demographic sub-group level. No individual-level data are published, nor are they
accessible or provided to anyone outside the Westat Data Collection Program staff.

1-14

A pre-notification letter mailed to potential respondents contains essential survey information and
assurances of privacy that enable the person to make an informed decision regarding his or her voluntary
participation in the data collection effort. A sample of the pre-notification informational letter sent to
potential survey participants appears in Appendix F.

A.11

Justification for Sensitive Questions

The ACL/AoA National Survey informs respondents that their responses to all questions are voluntary.
We assure them that their survey responses will remain private. Respondents can refuse to answer any
question, and the interviewer will move on to the next question on the survey instrument. Additionally,
respondents are permitted to stop at any point and to continue the interview at a later time.
The physical functioning module contains questions on the ability of respondents to perform certain tasks,
such as getting around inside and outside the home, getting in and out of a chair or the bed, getting to and
using the toilet, etc., as well as questions asking about their health conditions. These types of questions
might be considered to be sensitive; however, we have never had a respondent object to answering these
types of questions, especially when they are part of a battery of physical functioning questions. In
addition, analysis of Activities of Daily Living (ADL) and Instrumental Activity of Daily Living (IADL)
limitations in conjunction with outcomes and the type of services a respondent receives is an important
outcome measure. This kind of information, along with responses to questions on health conditions, can
tell us about the frailty of the respondents served by the nutrition (home-delivered and congregate meals),
home care (homemaker/housekeeping), case management and transportation services, and the people who
are able to maintain their independence, rather than enter nursing homes, because of those services.
Caregivers will be asked about the health conditions and ADL and IADL limitations of their care
recipients, again to allow for analysis of the frailty of care recipients whose caregivers are part of the
National Family Caregiver Support Program. Respondents can always refuse to answer any question, and
the interviewer will move on to the next question on the survey instrument.
Additionally, a question asking respondents if they identify as lesbian or gay, straight, bisexual, or
something else is included in the National Survey Instrument and may be considered sensitive. However,
as noted by HHS’s Office of Minority Health and supported by additional research, persons who identify
as lesbian, gay, bisexual, or transgender (LGBT) are more likely to suffer disparities in health and access
to human service organizations. Therefore, information about the LGBT population is valuable to AoA’s
Aging Network, as providers of Older Americans Act services can benefit from a greater understanding of
this commonly underserved population; a population that may face unique health challenges and limited
access to health care and social services.
OMB (OMB HHS/CDC Control 0920-0222) approved a series of sexual identity questions for use in the
National Health Interview Survey fielded by the National Center for Health Statistics. A standard set of
questions on sexual identity was created and tested. This set of questions had been proposed for inclusion
in the 2014 ACL/AoA National Survey. In 2017, although the follow-up questions were dropped, the
original question can be found in the National Survey Instrument’s Demographic module as questions
DE1a (DETHINK) on page 98 of Appendix J.

1-15

A.12

Estimates of Annualized Burden Hours and Costs

We estimated the respondent burden for the survey instruments based on the 3rd to 14th National Surveys
of OAA Participants. The cost to respondents who participate in the survey will be in terms of their time
only. The Service Recipient survey instrument (plus rotating module) takes about 45 minutes (.75 hour).
Most of the Service Recipients are retired. The Caregiver survey (plus rotating module) will also take
about 45 minutes. Based on the valuation of a participant's time at $25 per hour, the respondent burden
for each participant will be $18.75 for the Service Recipient and $18.75 for the Caregiver surveys and
$176 for the agency respondent selection process (estimated at 4 hours of agency personnel time). Exhibit
A-1 presents the estimated hour and annual cost response burden by respondent.

Exhibit A-1. Estimated Hour and Annual Cost Response Burden
Respondent/Data
collection
activity
Area Agency on
Aging:
Respondent
selection process
Service
recipients (i.e.,
Congregate and
Home-delivered
meal nutrition
programs; Case
Management,
Homemaker, and
Transportation
Services) +
Rotating Module
National Family
Caregiver
Support Program
Clients +
Rotating Module
Total

Number of
respondents
350

Responses
per
respondent
1

Hours per
response
4.0

Annual
burden
hours
1,400

4,400

1

.75

2,200

1

6,950

1

Cost per
hour
$44

Annual
burden
(cost)
$61,600

3,300

$25

$82,500

.75

1,650

$25

$41,250

.914
(weighted
mean)

6,350

Varies

$185,350

*It is important to note that not all of the individual respondents (6,600 for the national survey) will be
asked to complete all of the questionnaire modules (see Sampling Plan).

1-16

A.13

Estimates of Other Total Annual Cost Burden to
Respondents and Record Keepers

Total annual cost burden excluding wages of agency time and respondent time is zero (see Exhibit A-1).

A.14

Annualized Cost to the Federal Government

The overall cost of this research to the Federal Government is presented in Exhibit A-2.

Exhibit A-2. Total Annualized Cost to the Federal Government [Based on Year 1]
Category
Personnel (T&M including staff & indirect costs)

A.15

Costs
$810,896

Telephone (long-distance telephone survey)

$53,820

Other direct

$27,159

Total direct charges (per task order)

$80,979

Indirect charges

$13,498

Total

$905,373

Explanation for Program Changes or Adjustments

This is a revision to a currently approved information collection request; there is a program change
decrease of -7250 annual burden hours.

1-17

A.16

Plans for Tabulation & Publication and Project Time
Schedule

In this section, the range of analyses that will be conducted is described using the performance
measurement data. The research team suggests several levels of analysis to first describe the
characteristics of clients and the range of services provided by State Units on Aging (SUAs) and AAAs,
noting similarities and differences in the size and scope of program design and operations. The research
team will paint a descriptive profile of OAA clients, paying special attention to risk factors that prior
research shows are highly correlated with quality-of-life outcomes for older persons. These risk factors
include ADL and IADL limitations, as well as income, educational level, living arrangements, age cohort,
gender, race and ethnicity, and area of residence by degree of urbanization. In conjunction with the type
and severity of ADL/IADL limitations that clients report, these factors are indicative of risk for loss of
independence and other adverse outcomes. For example, it is known that among older adults, those living
alone are at higher risk for institutional placement. Educational attainment, in conjunction with age, is a
powerful predictor of need for services and outcomes for older persons.
In conjunction with the performance measure on service recipients, the survey will include questions on
ADL limitations (e.g., difficulty with personal care activities such as bathing and dressing) and IADL
limitations (e.g., difficulty with such home management activities as meal preparation, shopping, and
housekeeping), and health conditions. These questions appear in Appendix J in the Physical, Social, and
Well-Being Module and are denoted as “PF” for Physical Functioning. In addition, caregivers will be
asked about the health conditions and ADL limitations of their care recipients.
The timetable for the baseline data collection and the two follow-up data collections is shown in Exhibit
A-3.

Exhibit A-3. Data Collection Timetable
Survey Cycle
15th National Survey/
Data Collection
15th National Survey/
Data Collection
15th National Survey/
Data Collection
15th National Survey/
Data Collection
15th National Survey/
Baseline Data Collection
16th National Survey/
Data Collection
16th National Survey/
Data Collection

Data Collection Activity
Telephone/email contact with agencies
to draw sample
Telephone survey of participants

End dates
1 month after OMB clearance

Data editing, coding and key entry,
data analysis
Deliver data to ACL

7 months after OMB clearance

Final report on data collection

15 months after OMB clearance

Telephone/email contact with agencies
to draw sample
Telephone survey of participants

13 months after OMB clearance

1-18

5 months after OMB clearance

9 months after OMB clearance

17 months after OMB clearance

16th National Survey/
Data Collection
16th National Survey/
Data Collection
16th National Survey/
Data Collection
17th National Survey/
Data Collection
17th National Survey/
Data Collection
17th National Survey/
Data Collection
17th National Survey/
Data Collection

A.17

Data editing, coding and key entry,
data analysis
Deliver data to ACL

19 months after OMB clearance

Final report on baseline data collection

27 months after OMB clearance

Telephone/email contact with agencies
to draw sample
Telephone survey of participants

25 months after OMB clearance

Data editing, coding and key entry,
data analysis
Deliver data to ACL

31 months after OMB clearance

21 months after OMB clearance

29 months after OMB clearance

33 months after OMB clearance

Reason(s) Display of OMB Expiration Date is Inappropriate

ACL/AoA is not seeking an exemption from displaying the expiration date of OMB approval.

A.18

Exceptions to Certification for Paperwork Reduction Act
Submissions

ACL/AoA is not requesting any exceptions from OMB Form 83-I.

1-19

B

Collection of Information Employing Statistical
Methods

B.1

Respondent Universe and Sampling Methods

Introduction
This Paperwork Reduction Act (PRA) request is to conduct a cross-sectional survey of OAA service
recipients.
We will employ a two-stage sample design for the 15th National Survey of Older Americans Act
Participants (NSOAAP). The following sections discuss the respondent universe and sampling methods.

Baseline Respondent Universe
For the first stage of the sample design, we will select a probability sample of AAAs proportional to size
(PPS) of the total number of clients (actual and predicted) in each stratum, where stratum is defined by
Census Division. The Census Divisions are as follows:

Census Division
1 (New England)
2 (Middle Atlantic)
3 (East North Central)
4 (West North Central)
5 (South Atlantic)
6 (East South Central)
7 (West South Central)
8 (Mountain)
9 (Pacific)

As a result of the 14th NSOAAP, client counts are available for 581 out of the 628 AAAs in the
sampling frame, covering approximately 92.5% of the AAAs. We were able to predict the total
client counts for those AAAs for which we did not have prior data using regression modeling,
where the square root of the AAA budget and Census Division were used as predictors.

2-1

When selecting AAAs for the 15th National Survey, Westat will select a sample of AAAs large enough to
recruit 350 Area Agencies on Aging. The second stage is the selection of a random sample of service
recipients by service within each sampled AAA, including all of the largest ones. In this way all service
recipients will have a known probability of selection. A fixed number of service recipients will be
selected within each service for a total of 6,600 recipients. It is important to note that clients are sampled
independently by service and no client will be asked to participate for more than one service. Further if a
client happens to be sampled for more than one service, the client will be assigned to a single service at
random.
Exhibit B-1 on the next page presents the respondent universe for each module proposed for the 15th
National Survey of Older Americans Act Participants (NSOAAP).

2-2

Exhibit B-1. Respondent Universe

Service Recipient Survey
PERFORMANCE MEASURES

INDICATOR

TARGET POPULATION OF PARTICIPANTS

Congregate Meals Module

Questions on nutrition intake, nutrition risk, food
security and clients’ assessments of the Congregate
Meals program.

All service recipients receiving Congregate
Meals services

Home-delivered Meals
Module

Questions on nutrition intake, nutrition risk, food
security and clients’ assessments of the Homedelivered Meals program.
Questions on client’s experience and assessment of
transportation services.
Questions on clients’ experiences and assessments of
case management services.
Questions on clients’ experiences and assessments of
homemaker/housekeeping services.

All service recipients receiving Home Delivered
Meals

Questions on clients’ experiences and assessments of
caregiver support services.
Questions asking service recipients if they receive
other OAA services.

All service recipients who receive Caregiver
Support Services
All service recipients. Caregivers will be asked
about services received by their care
recipients.
All service recipients, with the exception of
Caregivers; Caregivers will be asked these
questions about their care recipients.
All service recipients, except Caregivers.

Transportation Module
Case Management Module
Homemaker/Housekeeping
Module
Caregiver Support Module
Additional Services List

Physical Functioning
Module
Emotional Well-Being
Module

Revised Katz Activities of Daily Living (ADL) Scale and
Quality of life measures from the Behavior Risk Factor
Surveillance System (BRFSS) questionnaire.
Questions on mood and affect from prior surveys of
older adults.

All users of Transportation Services
All service recipients receiving Case
Management services.
All service recipients who receive
Homemaker/Housekeeping Services

Degree of satisfaction with social activity and of
health effects on social activities.
Questions ask about falling and losing one’s balance.

All service recipients, except Caregivers.

Social Integration Module

Questions about contact with other people and
perception of social isolation.

All service recipients, except Caregivers.

Life Changes Module

What life event prompted client to seek out services.

All service recipients, except Caregivers.

Demographic Information
Module

Demographic Information

All service recipients and Caregivers.

Social Functioning Module
Falls Module

All service recipients and Caregivers.

Annual Rotating Module
PERFORMANCE MEASURES

COVID-19 Module (2021)

INDICATOR

TARGET POPULATION OF PARTICIPANTS

Questions ask about the effect COVID-19 has had on
older adults’ access to and use of Older Americans Act
programs and services during the past year.

2-3

All service recipients and Caregivers.

Response Rates from other National Surveys of Older Americans Act Participants
This is the 15th time this type of survey will be conducted. This OMB approved survey (0985-0014,
0985-0017, 0985-0020, 0985-0023) was done in 2002, 2003, 2005, 2008, 2009, 2011, 2012, 2013, 2014,
2015, 2016, 2017, 2018, and 2019. The research team anticipates an 80 percent response rate for AAAs
given the 81% AAA response rate in the 14th NNOAAP, and an 80 percent cooperation rate for the
telephone survey of respondents, based on the success we had with the preceding 14 surveys.
Response rates are calculated by taking the ratio of the number of completed interviews to the number of
eligible units (this includes completes and non-respondents). Non-response occurs for a variety of reasons,
including, for example, being unwilling to participate, unable to locate respondents, and language issues.
In the case of surveys conducted by telephone, such as NSOAAP, potential respondents may have difficulty
communicating by phone, another instance of nonresponse. The U.S. Census Bureau defines the following:
“Unit nonresponse occurs when respondents are unable or unwilling to participate; interviewers are unable
to locate addresses or respondents; or when other barriers exist to completing the interview.”4
In NSOAAP, we often refer to completion rates because of our experience in locating and contacting
potential respondents. Members of the NSOAAP target population frequently have changes in status, such
as relocating to be closer to family members, moving into assisted living or nursing facilities or other such
changes. These changes can occur between the time the client lists are created and attempts are made in the
field to contact the individuals. Completion rates are calculated by excluding records for whom no contact
was ever established, for example, the not-locatable and not resolved cases from the non-respondents.5 In
a survey like NSOAAP the completion rate may be a more accurate measurement of the degree of success
in collecting data which are representative of the population of service clients.

4

https://www.census.gov/programs-surveys/acs/methodology/sample-size-and-data-quality/response-rates-definitions.html

5

https://www.insightsassociation.org/issues-policies/best-practice/casro-definition-response-rates

2-4

B.2

Procedures for the Collection of Information

B.2.1

Introduction

Several data collection activities will be conducted to support the survey. They are designed to ensure as
complete a sample of AAAs (stage one) and service recipients (stage two) as possible. This will provide a
representative sample for the analyses and to inform ACL/AoA on results of performance measures for
state and community programs on aging under the Older Americans Act.

B.2.2

Data Collection Procedures

B.2.2.1

Telephone Contact with State and Local Agencies on Aging

Information will be collected in a two-step process. The proposed design will employ a probability
sample of all AAAs proportional to size (PPS) of the total client counts per AAA. Once an agency is
selected, its director will receive an email message from ACL that contains an introductory letter from
ACL, an invitation to participate in an orientation webinar, and detailed instructions for the AAA (see
Appendices H & I)6. Approximately two weeks later, state and AAA staffs will be able to participate in a
live webinar to learn about the importance of the survey and how the procedures for selecting clients will
work. ACL will also post a recording of the webinar for state and AAA staffs who are unable to
participate during the live webinar. For AAAs that do not respond to the initial email from ACL or
participate in the webinar, a researcher will reach out to the agencies to address any concerns that they
might have and to review instructions for sampling the service recipients. The researcher will explain the
numbered participant lists the agency needs to generate from which to select the random sample of
service recipients for each of the six services. In addition, we will provide detailed instructions specific to
the client tracking software used by the AAA. Previous experience has enabled Westat, the contractor, to
streamline the data collection procedures for the AAAs.

B.2.2.2

Telephone Survey of Older Americans Act Participants and Caregivers

Pre-notification Advance Letters
Potential respondents selected for the telephone interview will receive a letter from ACL. The letter
contains an introduction to the study, explanation about the nature of participation, and a toll-free phone
number to call if they do not wish to participate. Those who opt out of the study are not contacted further.
A separate toll-free phone number is included for respondents who wish to reach a telephone interviewer
directly, and take the survey at their convenience. A copy of this letter is included in Appendix F].

6

State units also receive a letter with a list of AAAs selected in the state.

2-5

Telephone Interview
Interviewers participate in intensive training sessions prior to data collection and are monitored during
data collection to ensure the protocol is properly followed. The training covers general interviewing
techniques, topics specific to administering the 15th National Survey of Older Americans Act
Participants, and practice sessions.
The study sample includes older adults who may be living with disabilities. With that in mind, the
training designed and conducted for the data collectors/telephone interviewers includes special guidance
for interviewing and accommodating respondents who are age 60 or older and who may have disabling
conditions and/or communication problems (hearing impairments, speech disorders, cognitive
impairments, memory disorders, non-native English speakers.) In certain instances, an interview with an
interpreter or a proxy is arranged. Additionally, data collectors are advised to be alert to the respondent’s
fatigue and to suggest calling back and completing the interview during another session. For Spanishspeaking respondents, trained bilingual data collectors conduct the interview in Spanish.
Interviewers will conduct a 45-minute telephone survey of a representative sample of Older Americans
Act service recipients and caregivers. The interview includes modules for each service (e.g., home
delivered meals, congregate meals, case management, caregiver, transportation, and homemaker) as well
as modules that are the same for all services on demographics, physical functioning, and quality of life.
Additionally, a module on COVID-19 will be asked of all respondents for the 2021 survey. Interviewers
administer the appropriate service module (i.e., the module that focuses on the service from which the
participant was sampled.)
The service modules include items on the extent to which the respondents use the service, consumer
assessment of services, and self-reported outcomes, such as the ability to live independently at home. The
demographic module identifies age, living arrangements, race/ethnicity, and income categories. The
module on physical functioning identifies the extent to which respondents are able to care for themselves
(e.g., bathe dress, eat, etc.) and are able to handle paying bills, going to the doctor, and grocery shopping,
for example.
Quality Control Procedures
Westat has quality control procedures in place for every phase of the project. Interviewers participate in
rigorous training that includes general interviewer training and project specific training. Trainers observe
interviewers conducting practice interviews, and they monitor interviewers during data collection. During
data collection, data are checked to ensure that there are no outliers in the dataset. In addition, when
questions are raised during an interview, interviewers complete a form documenting an ambiguous or
inconsistent response. Researchers review the forms and make any necessary adjustments.

2-6

B.2.3

Sampling Plan

B.2.3.1

Sample Design

The sample design for the 15th survey will consist of two stages, with a sample of approximately 350
AAAs in the first stage and a sample of clients, by service type, from each selected AAA, in the second
stage. This design is similar to that of the 3rd-14th surveys. The client sample sizes by service type, as
specified by ACL, are as follows:







Caregiver Services
Home Delivered Meals
Congregate Meals
Case Management Services
Transportation Services
Homemaker Services

2,200
1,100
1,100
550
1,100
550

As in the 3rd through 14th surveys, these sample sizes will permit the production of reliable estimates both
at the national level and at the geographic regional or demographic sub-group level.
For a two-stage design, Table B-1 presents the half-widths of the 95 percent confidence intervals (CI) for
various sample sizes and for cross-sectional estimates of target characteristics of proportions ranging from
10 percent to 50 percent.7 The 50 percent target is a worst-case scenario, where respondents are expected
to be fairly evenly split on a particular response item, limiting the reliability of the estimate (e.g., such as
trying to predict the outcome of an election where the sample of voters is about evenly divided between
two candidates). Also, the precision of any estimate greater than 50 percent is the same as that of its
complement, i.e., the precision of a 70 percent estimate is the same as the precision of a 30 percent
estimate. The numbers in the tables are half-widths of 95 percent CIs, (i.e., the estimate,  the half-width
is the CI, where half-width is 1.96 times the standard error (SE) of an estimate). For example, Table B-1
shows that for a sample of size 1,000, for a target characteristic of around 30 percent, the CI would be the
estimate  3.24 percent.
The table can be used to assess the adequacy of the sample sizes for both the national, and the regional or
sub-group level estimates. For example, if the sample size is 1,000 at the national level then the sixth row
in Table B-1 would provide the precision of the estimates at the national level. From the same table, the
precision of an estimate at the regional or sub-group level can be obtained by computing the sample size
that is expected for a particular region. For instance, if the region covers 25 percent of the target
population, then the sample size for that region is expected to be about 250 (out of 1,000) under a
proportional allocation, and the precision of the estimates for that region can be checked from the row
where the sample size equals 250 in Table B-1. Similarly, if a sub-group covers 10 percent of the target
population then the expected sample size for that sub-group is 100 out of 1,000 and the precision of the
estimates for that sub-group can be checked from the row with sample size equal 100.

7

This percent range refers to the client response patterns that may occur; for example, in a yes/no question, it refers to the expected percent of
respondents who will answer yes, versus no.

2-7

The total size of the target population has a negligible impact on the requirement of the sample size. For
example, if a sample size of 250 is required to produce an estimate at the national level, then to estimate
the same characteristic for a particular region (with the same level of precision), the required sample size
from that region alone would be about 250. If there are four regions, then the required sample size at the
national level would be about 1,000 (to guarantee adequate representation in each group). Therefore, to
meet the objective of the proposed survey (i.e., to produce estimates at the regional or sub-group level
with the same level of precision as the national estimates obtained from previous studies), the required
sample size for each target region or sub-group will have to be the same as the total sample size of the
previous studies.
For instance, a question was asked in the first national survey about the timeliness of the delivery of
meals and an estimated 44 percent of all clients reported that the meals arrived on time, all the time. This
estimate was based on a sample of 472 clients and had a CI of  5.2 percent. Table B-1 shows that to
achieve a CI of  5.2 percent for an estimate, with a proportion between 40 percent and 50 percent, a
sample of size around 480 is required. That means if this estimate is required at the regional level with the
same level of confidence as the national, then the sample size in each region will have to be 480 and
hence the sample size at the national level will be 480x4=1,920. In that case, the CI for this estimate at the
national level would be much more precise than for the region (a little over  2.5 percent). Table B-1 can
be used to see the precisions of the estimates that would be achieved at various levels using the expected
sample sizes at the respective levels. The table can also be used to check the sample size requirement
corresponding to a desired level of precision of an estimate.

Table B-1 Half-widths of 95 percent confidence intervals by various sample sizes and
estimates of target characteristics (computed for a two-stage design with a
design effect of 1.30)

Estimates of Target Characteristics
Sample size
3,500
3,000
2,500
2,000
1,500
1,000
750
500
400
300
250
200
100

10 percent

20 percent

30 percent

40 percent

50 percent

1.13
1.22
1.34
1.50
1.73
2.12
2.45
3.00
3.35
3.87
4.24
4.74
6.70

1.51
1.63
1.79
2.00
2.31
2.83
3.26
4.00
4.47
5.16
5.65
6.32
8.94

1.73
1.87
2.05
2.29
2.64
3.24
3.74
4.58
5.12
5.91
6.48
7.24
10.24

1.85
2.00
2.19
2.45
2.83
3.46
4.00
4.90
5.47
6.32
6.92
7.74
10.95

1.89
2.04
2.23
2.50
2.89
3.53
4.08
5.00
5.59
6.45
7.07
7.90
11.17

2-8

It is important to note that if the population sizes in the sub-groups or regions vary widely, then the
national sample must be allocated appropriately to produce estimates from all individual subgroups/regions with an equal level of precision. Otherwise, under a proportionate allocation, larger subgroups will have more than required sample size while the smaller sub-groups will have less than the
sample size required. For example, if the estimates are required separately for Whites and AfricanAmericans, then just increasing the national sample would not ensure sufficient sample size for AfricanAmericans, because less than 15 percent of recipients are African-Americans for many services. In this
situation, the national sample can be disproportionately allocated by over-sampling smaller sub-groups to
ensure that sufficient samples are drawn from all target sub-groups. However, over-sampling an ethnic or
demographic group will require that agencies first list all their clients with the characteristic of interest
and then select a sample from this list by sub-group (which may exceed the capacity of many AAA
information systems).

B.2.3.2

Sample Size for Estimation of Change

If there is interest in comparing estimates from one year with another year (cross-sectionally) , or
comparing estimates of one sub-group with another sub-group, the sample size requirements are different
from those that show individual point estimates at the same level of precision. The standard error (SE) of
the difference between two independent estimates (for example, A and B) can be obtained by

SE ( A  B )  SE 2 ( A)  SE 2 ( B ) , and the half-width of the 95 percent CI is 1.96  SE( A  B)
Since the variance of the estimate (of a difference between estimates) is the sum of the variances of the
relevant individual estimates, the required sample size for estimating a difference or change is higher than
for a single point estimate.
Table B-2 presents half-widths of 95 percent CIs under a two-stage design for various sample sizes and
various averages of the two estimates to be compared. For example, if the average of the two target
characteristics to be compared is around 30 percent (for example, A=25 and B=35) and the sample size in
each sub-group is 500, to detect a difference between the two sub-groups with statistical significance, the
actual difference between the two sub-group characteristics will have to be at least 6.48 percent. This is
much higher than the corresponding half-widths presented in Table B-1 for each of the individual
estimates. That means a sample size that is sufficient to produce a reliable point estimate for each subgroup, individually, is not necessarily sufficient to detect the difference between the two sub-groups with
the same level of precision.
Therefore, if the survey is designed for use at a region or sub-group level, then the corresponding national
estimates can be compared meaningfully from one year to another, or for one service versus another (e.g.,
the percent of each service’s clients below a certain income level). For example, if the sample size is
1,000 in each year, and if the average response proportion for the two target characteristics is around 30
percent, then a difference of 4.58 percent or more between the years is detectable. The corresponding
comparison with a sub-group sample of size 500, would not allow detecting a difference unless it is 6.48
percent or more.
Table B-2 can be used to see the extent of difference that can be detected under a two-stage design, for
various sample sizes, and for various characteristics to be compared either at the national or at the subgroup level.

2-9

Table B-2
Half-widths of 95 percent confidence intervals for the difference between two
estimates by various sample sizes and for various averages of the two estimates
(computed for a two-stage design with a design effect of 1.30)

Sample size in
each group
3,500
3,000
2,500
2,000
1,500
1,000
750
500
400
300
250
200
100

Average of the estimates to be compared
10 percent

20 percent

30 percent

40 percent

50 percent

1.60
1.73
1.90
2.12
2.45
3.00
3.46
4.24
4.74
5.47
6.00
6.70
9.48

2.14
2.31
2.53
2.83
3.26
4.00
4.62
5.65
6.32
7.30
8.00
8.94
12.64

2.45
2.64
2.90
3.24
3.74
4.58
5.29
6.48
7.24
8.36
9.16
10.24
14.48

2.62
2.83
3.10
3.46
4.00
4.90
5.65
6.92
7.74
8.94
9.79
10.95
15.48

2.67
2.89
3.16
3.53
4.08
5.00
5.77
7.07
7.90
9.12
9.99
11.17
15.80

Nonresponse adjustment was done as part of the weighting process for the previous surveys and will also
be done for the 15th National Survey. The weights of the respondents were inflated to account for the
weights of the nonrespondents separately for each service. The adjustment was applied independently
within nonresponse adjustment groups defined by census region within service. That means the
nonrespondents within a group are represented by the respondents in the same group. The same types of
nonresponse adjustment will be done for the 15th survey.

B.2.4

Older Americans Act Participant Survey Instruments

The survey consists of telephone interviews with service recipients and caregivers. The interview is
structured and will contain specific questions about the mix of services the person has received and his or
her assessment of those services. Whenever appropriate, questions will contain predefined categories.
Probes will be used to facilitate obtaining complete responses to all the questions. The interviews of
caregivers will not include the questions that ask for physical functioning (except health conditions and
ADL and IADL limitations of their care recipients) nor the Emotional Well-being and Social Functioning
questionnaires. The interviews will last approximately 45 minutes and cover the topics discussed below.
This is the same process followed for each of the previous surveys.

2-10

1.

Nutrition-Congregate Meals: If a respondent receives Congregate Meals, they will
be asked a short questionnaire based on the Congregate Meals survey. This
questionnaire asks how long they have been attending the congregate meals program;
how often they eat at the site; when the last time was they ate at the site; to rate the
program; and how much of their food intake the meal provides on the days they eat at
the site.

2.

Nutrition-Home-delivered Meals: If a respondent receives Home-delivered Meals,
they will be administered a short questionnaire based on the Home-delivered Meals
survey. This questionnaire asks how long they have been receiving home-delivered
meals; how often they receive home-delivered meals; when the last time was they
received a meal; to rate the program; and how much of their food intake the meal
provides on the days they receive home-delivered meals.

3.

Transportation: All service recipients who use transportation services will be
interviewed using this survey module. The module asks how long they have been
using the transportation; how often they use it; when the last time was they used it; trip
purpose; to rate the transportation service; and about the number of times the
respondent uses the service.

4.

Homemaker/Housekeeping: Questions on the impact of homecare services will be
asked of respondents who receive homemaker or housekeeping services. Again, the set
of questions is similar to those asked of the other services: how long respondents have
been receiving homemaker services; how often they receive homemaker services;
when the last time was they used the services; to rate the program; and if they can
depend on their aides to do deliver the allotted services.

5.

Case Management: Service recipients who receive case management services will be
asked questions about their experiences with the program. They will be asked: how
long they have been receiving the services; how they would rate the various aspects of
the case management services (e.g. ease of contact with the case managers; if the case
managers understand their needs, etc.); to rate the services overall and if they
contribute to the decisions about their care.

6.

Service List: All service recipients will then be asked about the mix of services they
receive and the impact of those services. They will also be asked to rate the services
overall.

7.

Physical Functioning: This module will be asked of all service recipients (except
Caregiver clients). This survey module will include questions on: Activities of Daily
Living limitations (e.g., difficulty with personal care activities such as bathing and
dressing) and Instrumental Activities of Daily Living limitations (e.g., difficulty with
such home management activities as meal preparation, shopping, and housekeeping).
Questions about the respondents’ health are also being asked, to help with assessing
the frailty of the clients served by OAA services. Caregivers will be asked these
questions about their care recipients.

8.

Emotional well-being: This module will be asked of all participants in the surveys,
except caregivers. The questions ask if the respondent has felt sad or depressed, worried
or tense, and if they feel that they did not get enough rest, within the last thirty days.
They are also asked to describe their overall emotional well-being by responding to a
close-ended question (i.e., “Would you say…Excellent, Very Good, Good, Fair, or

2-11

Poor?”).
9.

Social Functioning: All service recipients except caregivers will be asked four
questions from the Social Functioning section of the Short-Form Survey (SF-36).
These questions ask if the respondent feels his or her social life is adequate and if
health concerns have interfered with the ability to participate in social activities.

10.

Falls: All service recipients and caregivers will be asked these five questions. The
questions ask about falling and losing one’s balance.

11.

Social Integration: All service recipients except caregivers will be asked these four
questions about social isolation.

12.

Life Changes: All service recipients except caregivers will be asked this one question
about why they sought out services.

13.

National Family Caregiver Support Program Assessment: Caregivers who receive
caregiver support services through the National Family Caregiver Support Program
will be surveyed as part of the 15th NSOAAP. This module has questions on services
offered to caregivers through the National Family Caregiver Support Program, and the
impact of those services. There are also questions about services the care recipient
receives and satisfaction with and impact of those services; support the caregiver
receives, either as part of a formal support group or from other relatives and friends;
and what kinds of other information the caregiver would find valuable. The survey
asks about the type of help the caregiver provides for the care recipient, the amount of
time they provide care, benefits caregiving provides them (companionship, a sense of
accomplishment, etc.), drawbacks of caregiving (financial burdens, lack of private
time, etc.), and demographic and health information on the care recipient. Three of the
questions for this module were adapted from an AARP survey, Caregiving in the U.S8

14.

Demographic information of the respondent: Demographic information about the
respondent will be collected, including type of area of residence (urban, suburban, or
rural), Zip Code, education level, race, gender, living arrangements (living alone, with
spouse, or with others), and income level. This module will be administered to all
participants. The caregiver survey already includes some demographic questions about
the care recipient, but the demographic information on the caregiver will be gathered
using this demographic module.

15.

COVID-19 Module: For 2021 only, all service recipients and caregivers will be asked
questions about the effect COVID-19 has had on their access to and use of Older
Americans Act programs and services during the past year.

Many of the national survey questions come from such commonly used vehicles as the Survey of Income
and Program Participation (SIPP), (e.g., the ADL and IADL questions), the Behavioral Risk Factor
Surveillance System (BRFSS) surveys conducted within each state using HHS/CDC standard questions,
and other existing surveys. As mentioned previously, other questions were developed under the POMP
contract, as well as with input from an advisory panel of aging experts under a Redesign contract. The
survey instrument for the 15th NSOAAP is virtually the same as the instrument used for the previous 14th
NSOAAP.
8

National Alliance for Caregiving and AARP (2004, April). Caregiving in the U.S. Appendix C, pp. 16-17 retrieved from AARP Web site:
http://assets.aarp.org/rgcenter/il/us_caregiving.pdf

2-12

B.3

Methods to Maximize Response Rates and Deal with
Nonresponse

Procedures for Eliciting Cooperation and Maximizing Response Rates among AAAs
Westat will use a similar procedure to select respondents for the 15th National Survey of OAA
Participants as it did in the previous seven surveys, which proved very successful. As part of the
recruitment procedures, Westat initially contacts the AAAs by sending via email an introductory letter
from ACL and an information package about the survey. A copy of these materials is also sent to each
State Unit on Aging that had AAAs sampled for the survey. (See Appendix G for the ACL letter sent to
the State Units on Aging; Appendix H for the ACL letter sent to the AAAs; and Appendix I
instructions/information package sent to the AAAs.) Following up by email and telephone, the Westat
research team works closely with each participating AAA to generate numbered lists of clients (using
client ID numbers) by service for the client sample frame. The Westat research team uses the numbered
lists of client ID numbers for the random selection of the respondents to be interviewed. To complete the
random sampling process, Westat research team members enter the total numbers of participants by
service into a computer sampling program. The sampling program randomly selects line numbers from
the numbered lists of clients. The number of clients to select per service is already entered into the
program. Westat informs the AAAs of the selected line numbers. The AAAs then provide the participant
names and telephone numbers associated with those line numbers to Westat.
Westat research assistants serving as recruitment specialists will encourage the participation of all
selected agencies by establishing rapport with contacts within each agency, coaching them on how to
generate their client lists, and assuring them that the time involved for them to complete the participant
selection procedures will be minimal. For agencies that refuse to participate, Westat will send them a
refusal conversion letter (already developed for the previous surveys), and call them one more time to try
to gain their cooperation. Once an agency refuses a second time, Westat will not try to contact them
again. Westat will also work with ACL to communicate with ACL Regional Administrators for help in
persuading reluctant AAAs to participate. Additionally, to promote agency participation, we plan to gain
the endorsement and support of the National Association for Agencies on Aging (N4A), as well as the
Advancing States [formerly known as the National Association of States United for Aging and
Disabilities (NASUAD)].
To reduce the burden for the AAAs, Westat works with software vendors of commercial client tracking
software programs commonly used by AAAs to develop step-by-step instructions for creating numbered
lists of client ID numbers by service. By using agency-assigned client ID numbers to generate numbered
lists of clients for the participant sample frame, Westat is able to screen the lists for duplicate client
entries. Additionally, the use of agency-assigned client ID numbers helps to decrease the amount of
personally identifiable client contact information collected by Westat during the survey.
Numbered lists will be developed for the following services: home delivered meals, congregate meals,
transportation, case management, and homemaker services, as well as caregivers who are served by the
National Family Caregiver Support Program.

2-13

To ensure a high participant response rate, Westat will send participants who are eligible for the telephone
survey a letter before they are contacted by an interviewer. The letters will be on each ACL’s letterhead,
with toll-free phone numbers provided to answer participants’ questions. Westat will attempt to contact
participants at different times of the day and different days of the week to maximize the possibility of
contact. Westat is also experienced in refusal conversion procedures, having achieved a refusal
conversion rate of 40 percent for the earlier surveys. (See Appendix F)

B.4

Tests of Procedures or Methods to Be Undertaken

As discussed in earlier sections, the individual service modules and the modules on physical functioning,
quality of life, and demographics have all been field tested and validated by the POMP participants. For
example, for each module the POMP grantees drew samples of service recipients, administered the
modules, and analyzed the data. The POMP grantees revised the items on the modules based on the
results of the field tests and validity studies. Some new questions added for the 14th NSOAAP (Life
Changes, Social Integration, Falls) were developed by an advisory panel of aging experts and cognitivelytested under a separate NSOAAP Redesign contract.
The majority of the items on the survey instrument for the 15th National Survey are from the previous
survey instruments. Over the years, several items have been removed from the survey instrument because
of ambiguity of the wording or in cases where the results of the item showed no variation across response
options.

2-14

B.5

Individuals Consulted on Statistical Aspects and
Individuals Collecting and/or Analyzing Data

The use of statistical sampling methods is critical to this survey. Under the supervision of ACL,
Westat is responsible for selecting the sample, conducting the interviews, data weighting and data
analysis. Below are the names and contact information of individuals responsible for the
statistical aspects of the study and individual collection and/or analysis of the data.

Administration for Community Living/Administration for Aging Personnel Responsible for
Deliverables
Susan Jenkins, Ph.D.
Director, Office of Performance and Evaluation
Administration for Community Living
U.S. Department of Health and Human Services
300 C Street, SW, Rm 1243
Washington, DC 20201
Tel: 202-795-7369
[email protected]

Westat Staff
Dwight Brock, Ph.D. – Role: Senior statistician overseeing study design, development of the
sampling plan, weighting, and data analysis
Senior Statistician
Westat
1600 Research Blvd.
Rockville, Maryland 20850
301-517-4026
[email protected]
Joanne Campione – Role: Project Director
Senior Study Director
Westat
2530 Meridian Pkwy, Suite 200
Durham, NC 27703
919-474-3506
[email protected]
Katie Hubbell– Role: Senior systems analyst with involvement in sampling data weighting, data
analysis, and reports.
Senior Systems Analyst
Westat
1600 Research Blvd.
Rockville, Maryland 20850
301-294-2020
[email protected]
Robin Ritter – Role: Recruitment Director with involvement in design, sampling, AAA and
respondent recruitment, and data analysis

Senior Study Director
Westat
1600 Research Blvd.
Rockville, Maryland 20850
301-240-314-5804
[email protected]
Jacqueline Severynse – Role: Design of the sampling plan, weighting, and data analysis.
Senior Statistician
Westat
1600 Research Blvd.
Rockville, Maryland 20850
301-517-8058
[email protected]
Lena Watkins – Role: Telephone Operations Manager
Westat
1600 Research Blvd.
Rockville, Maryland 20850
301- 294-3938
[email protected]