0023 Part B Collection of Information Employing Statistical Methods

B

Collection of Information Employing Statistical
Methods

B.1

Respondent Universe and Sampling Methods

Introduction
This Paperwork Reduction Act (PRA) request is to conduct a cross-sectional survey of OAA service
recipients.
We will employ a two-stage sample design for the 15th National Survey of Older Americans Act
Participants (NSOAAP). The following sections discuss the respondent universe and sampling methods.

Baseline Respondent Universe
For the first stage of the sample design, we will select a probability sample of AAAs proportional to size
(PPS) of the total number of clients (actual and predicted) in each stratum, where stratum is defined by
Census Division. The Census Divisions are as follows:

Census Division
1 (New England)
2 (Middle Atlantic)
3 (East North Central)
4 (West North Central)
5 (South Atlantic)
6 (East South Central)
7 (West South Central)
8 (Mountain)
9 (Pacific)

As a result of the 14th NSOAAP, client counts are available for 581 out of the 628 AAAs in the
sampling frame, covering approximately 92.5% of the AAAs. We were able to predict the total
client counts for those AAAs for which we did not have prior data using regression modeling,
where the square root of the AAA budget and Census Division were used as predictors.

2-1

When selecting AAAs for the 15th National Survey, Westat will select a sample of AAAs large enough to
recruit 350 Area Agencies on Aging. The second stage is the selection of a random sample of service
recipients by service within each sampled AAA, including all of the largest ones. In this way all service
recipients will have a known probability of selection. A fixed number of service recipients will be
selected within each service for a total of 6,600 recipients. It is important to note that clients are sampled
independently by service and no client will be asked to participate for more than one service. Further if a
client happens to be sampled for more than one service, the client will be assigned to a single service at
random.
Exhibit B-1 on the next page presents the respondent universe for each module proposed for the 15th
National Survey of Older Americans Act Participants (NSOAAP).

2-2

Exhibit B-1. Respondent Universe

Service Recipient Survey
PERFORMANCE MEASURES

INDICATOR

TARGET POPULATION OF PARTICIPANTS

Congregate Meals Module

Questions on nutrition intake, nutrition risk, food
security and clients’ assessments of the Congregate
Meals program.

All service recipients receiving Congregate
Meals services

Home-delivered Meals
Module

Questions on nutrition intake, nutrition risk, food
security and clients’ assessments of the Homedelivered Meals program.
Questions on client’s experience and assessment of
transportation services.
Questions on clients’ experiences and assessments of
case management services.
Questions on clients’ experiences and assessments of
homemaker/housekeeping services.

All service recipients receiving Home Delivered
Meals

Questions on clients’ experiences and assessments of
caregiver support services.
Questions asking service recipients if they receive
other OAA services.

All service recipients who receive Caregiver
Support Services
All service recipients. Caregivers will be asked
about services received by their care
recipients.
All service recipients, with the exception of
Caregivers; Caregivers will be asked these
questions about their care recipients.
All service recipients, except Caregivers.

Transportation Module
Case Management Module
Homemaker/Housekeeping
Module
Caregiver Support Module
Additional Services List

Physical Functioning
Module
Emotional Well-Being
Module

Revised Katz Activities of Daily Living (ADL) Scale and
Quality of life measures from the Behavior Risk Factor
Surveillance System (BRFSS) questionnaire.
Questions on mood and affect from prior surveys of
older adults.

All users of Transportation Services
All service recipients receiving Case
Management services.
All service recipients who receive
Homemaker/Housekeeping Services

Degree of satisfaction with social activity and of
health effects on social activities.
Questions ask about falling and losing one’s balance.

All service recipients, except Caregivers.

Social Integration Module

Questions about contact with other people and
perception of social isolation.

All service recipients, except Caregivers.

Life Changes Module

What life event prompted client to seek out services.

All service recipients, except Caregivers.

Demographic Information
Module

Demographic Information

All service recipients and Caregivers.

Social Functioning Module
Falls Module

All service recipients and Caregivers.

Annual Rotating Module
PERFORMANCE MEASURES

COVID-19 Module (2021)

INDICATOR

TARGET POPULATION OF PARTICIPANTS

Questions ask about the effect COVID-19 has had on
older adults’ access to and use of Older Americans Act
programs and services during the past year.

2-3

All service recipients and Caregivers.

Response Rates from other National Surveys of Older Americans Act Participants
This is the 15th time this type of survey will be conducted. This OMB approved survey (0985-0014,
0985-0017, 0985-0020, 0985-0023) was done in 2002, 2003, 2005, 2008, 2009, 2011, 2012, 2013, 2014,
2015, 2016, 2017, 2018, and 2019. The research team anticipates an 80 percent response rate for AAAs
given the 81% AAA response rate in the 14th NNOAAP, and an 80 percent cooperation rate for the
telephone survey of respondents, based on the success we had with the preceding 14 surveys.
Response rates are calculated by taking the ratio of the number of completed interviews to the number of
eligible units (this includes completes and non-respondents). Non-response occurs for a variety of reasons,
including, for example, being unwilling to participate, unable to locate respondents, and language issues.
In the case of surveys conducted by telephone, such as NSOAAP, potential respondents may have difficulty
communicating by phone, another instance of nonresponse. The U.S. Census Bureau defines the following:
“Unit nonresponse occurs when respondents are unable or unwilling to participate; interviewers are unable
to locate addresses or respondents; or when other barriers exist to completing the interview.”1
In NSOAAP, we often refer to completion rates because of our experience in locating and contacting
potential respondents. Members of the NSOAAP target population frequently have changes in status, such
as relocating to be closer to family members, moving into assisted living or nursing facilities or other such
changes. These changes can occur between the time the client lists are created and attempts are made in the
field to contact the individuals. Completion rates are calculated by excluding records for whom no contact
was ever established, for example, the not-locatable and not resolved cases from the non-respondents.2 In
a survey like NSOAAP the completion rate may be a more accurate measurement of the degree of success
in collecting data which are representative of the population of service clients.

1

https://www.census.gov/programs-surveys/acs/methodology/sample-size-and-data-quality/response-rates-definitions.html

5

https://www.insightsassociation.org/issues-policies/best-practice/casro-definition-response-rates

2-4

B.2

Procedures for the Collection of Information

B.2.1

Introduction

Several data collection activities will be conducted to support the survey. They are designed to ensure as
complete a sample of AAAs (stage one) and service recipients (stage two) as possible. This will provide a
representative sample for the analyses and to inform ACL/AoA on results of performance measures for
state and community programs on aging under the Older Americans Act.

B.2.2

Data Collection Procedures

B.2.2.1

Telephone Contact with State and Local Agencies on Aging

Information will be collected in a two-step process. The proposed design will employ a probability
sample of all AAAs proportional to size (PPS) of the total client counts per AAA. Once an agency is
selected, its director will receive an email message from ACL that contains an introductory letter from
ACL, an invitation to participate in an orientation webinar, and detailed instructions for the AAA (see
Appendices H & I)6. Approximately two weeks later, state and AAA staffs will be able to participate in a
live webinar to learn about the importance of the survey and how the procedures for selecting clients will
work. ACL will also post a recording of the webinar for state and AAA staffs who are unable to
participate during the live webinar. For AAAs that do not respond to the initial email from ACL or
participate in the webinar, a researcher will reach out to the agencies to address any concerns that they
might have and to review instructions for sampling the service recipients. The researcher will explain the
numbered participant lists the agency needs to generate from which to select the random sample of
service recipients for each of the six services. In addition, we will provide detailed instructions specific to
the client tracking software used by the AAA. Previous experience has enabled Westat, the contractor, to
streamline the data collection procedures for the AAAs.

B.2.2.2

Telephone Survey of Older Americans Act Participants and Caregivers

Pre-notification Advance Letters
Potential respondents selected for the telephone interview will receive a letter from ACL. The letter
contains an introduction to the study, explanation about the nature of participation, and a toll-free phone
number to call if they do not wish to participate. Those who opt out of the study are not contacted further.
A separate toll-free phone number is included for respondents who wish to reach a telephone interviewer
directly, and take the survey at their convenience. A copy of this letter is included in Appendix F].

6

State units also receive a letter with a list of AAAs selected in the state.

2-5

Telephone Interview
Interviewers participate in intensive training sessions prior to data collection and are monitored during
data collection to ensure the protocol is properly followed. The training covers general interviewing
techniques, topics specific to administering the 15th National Survey of Older Americans Act
Participants, and practice sessions.
The study sample includes older adults who may be living with disabilities. With that in mind, the
training designed and conducted for the data collectors/telephone interviewers includes special guidance
for interviewing and accommodating respondents who are age 60 or older and who may have disabling
conditions and/or communication problems (hearing impairments, speech disorders, cognitive
impairments, memory disorders, non-native English speakers.) In certain instances, an interview with an
interpreter or a proxy is arranged. Additionally, data collectors are advised to be alert to the respondent’s
fatigue and to suggest calling back and completing the interview during another session. For Spanishspeaking respondents, trained bilingual data collectors conduct the interview in Spanish.
Interviewers will conduct a 45-minute telephone survey of a representative sample of Older Americans
Act service recipients and caregivers. The interview includes modules for each service (e.g., home
delivered meals, congregate meals, case management, caregiver, transportation, and homemaker) as well
as modules that are the same for all services on demographics, physical functioning, and quality of life.
Additionally, a module on COVID-19 will be asked of all respondents for the 2021 survey. Interviewers
administer the appropriate service module (i.e., the module that focuses on the service from which the
participant was sampled.)
The service modules include items on the extent to which the respondents use the service, consumer
assessment of services, and self-reported outcomes, such as the ability to live independently at home. The
demographic module identifies age, living arrangements, race/ethnicity, and income categories. The
module on physical functioning identifies the extent to which respondents are able to care for themselves
(e.g., bathe dress, eat, etc.) and are able to handle paying bills, going to the doctor, and grocery shopping,
for example.
Quality Control Procedures
Westat has quality control procedures in place for every phase of the project. Interviewers participate in
rigorous training that includes general interviewer training and project specific training. Trainers observe
interviewers conducting practice interviews, and they monitor interviewers during data collection. During
data collection, data are checked to ensure that there are no outliers in the dataset. In addition, when
questions are raised during an interview, interviewers complete a form documenting an ambiguous or
inconsistent response. Researchers review the forms and make any necessary adjustments.

2-6

B.2.3

Sampling Plan

B.2.3.1

Sample Design

The sample design for the 15th survey will consist of two stages, with a sample of approximately 350
AAAs in the first stage and a sample of clients, by service type, from each selected AAA, in the second
stage. This design is similar to that of the 3rd-14th surveys. The client sample sizes by service type, as
specified by ACL, are as follows:







Caregiver Services
Home Delivered Meals
Congregate Meals
Case Management Services
Transportation Services
Homemaker Services

2,200
1,100
1,100
550
1,100
550

As in the 3rd through 14th surveys, these sample sizes will permit the production of reliable estimates both
at the national level and at the geographic regional or demographic sub-group level.
For a two-stage design, Table B-1 presents the half-widths of the 95 percent confidence intervals (CI) for
various sample sizes and for cross-sectional estimates of target characteristics of proportions ranging from
10 percent to 50 percent.7 The 50 percent target is a worst-case scenario, where respondents are expected
to be fairly evenly split on a particular response item, limiting the reliability of the estimate (e.g., such as
trying to predict the outcome of an election where the sample of voters is about evenly divided between
two candidates). Also, the precision of any estimate greater than 50 percent is the same as that of its
complement, i.e., the precision of a 70 percent estimate is the same as the precision of a 30 percent
estimate. The numbers in the tables are half-widths of 95 percent CIs, (i.e., the estimate,  the half-width
is the CI, where half-width is 1.96 times the standard error (SE) of an estimate). For example, Table B-1
shows that for a sample of size 1,000, for a target characteristic of around 30 percent, the CI would be the
estimate  3.24 percent.
The table can be used to assess the adequacy of the sample sizes for both the national, and the regional or
sub-group level estimates. For example, if the sample size is 1,000 at the national level then the sixth row
in Table B-1 would provide the precision of the estimates at the national level. From the same table, the
precision of an estimate at the regional or sub-group level can be obtained by computing the sample size
that is expected for a particular region. For instance, if the region covers 25 percent of the target
population, then the sample size for that region is expected to be about 250 (out of 1,000) under a
proportional allocation, and the precision of the estimates for that region can be checked from the row
where the sample size equals 250 in Table B-1. Similarly, if a sub-group covers 10 percent of the target
population then the expected sample size for that sub-group is 100 out of 1,000 and the precision of the
estimates for that sub-group can be checked from the row with sample size equal 100.

7

This percent range refers to the client response patterns that may occur; for example, in a yes/no question, it refers to the expected percent of
respondents who will answer yes, versus no.

2-7

The total size of the target population has a negligible impact on the requirement of the sample size. For
example, if a sample size of 250 is required to produce an estimate at the national level, then to estimate
the same characteristic for a particular region (with the same level of precision), the required sample size
from that region alone would be about 250. If there are four regions, then the required sample size at the
national level would be about 1,000 (to guarantee adequate representation in each group). Therefore, to
meet the objective of the proposed survey (i.e., to produce estimates at the regional or sub-group level
with the same level of precision as the national estimates obtained from previous studies), the required
sample size for each target region or sub-group will have to be the same as the total sample size of the
previous studies.
For instance, a question was asked in the first national survey about the timeliness of the delivery of
meals and an estimated 44 percent of all clients reported that the meals arrived on time, all the time. This
estimate was based on a sample of 472 clients and had a CI of  5.2 percent. Table B-1 shows that to
achieve a CI of  5.2 percent for an estimate, with a proportion between 40 percent and 50 percent, a
sample of size around 480 is required. That means if this estimate is required at the regional level with the
same level of confidence as the national, then the sample size in each region will have to be 480 and
hence the sample size at the national level will be 480x4=1,920. In that case, the CI for this estimate at the
national level would be much more precise than for the region (a little over  2.5 percent). Table B-1 can
be used to see the precisions of the estimates that would be achieved at various levels using the expected
sample sizes at the respective levels. The table can also be used to check the sample size requirement
corresponding to a desired level of precision of an estimate.

Table B-1 Half-widths of 95 percent confidence intervals by various sample sizes and
estimates of target characteristics (computed for a two-stage design with a
design effect of 1.30)

Estimates of Target Characteristics
Sample size
3,500
3,000
2,500
2,000
1,500
1,000
750
500
400
300
250
200
100

10 percent

20 percent

30 percent

40 percent

50 percent

1.13
1.22
1.34
1.50
1.73
2.12
2.45
3.00
3.35
3.87
4.24
4.74
6.70

1.51
1.63
1.79
2.00
2.31
2.83
3.26
4.00
4.47
5.16
5.65
6.32
8.94

1.73
1.87
2.05
2.29
2.64
3.24
3.74
4.58
5.12
5.91
6.48
7.24
10.24

1.85
2.00
2.19
2.45
2.83
3.46
4.00
4.90
5.47
6.32
6.92
7.74
10.95

1.89
2.04
2.23
2.50
2.89
3.53
4.08
5.00
5.59
6.45
7.07
7.90
11.17

2-8

It is important to note that if the population sizes in the sub-groups or regions vary widely, then the
national sample must be allocated appropriately to produce estimates from all individual subgroups/regions with an equal level of precision. Otherwise, under a proportionate allocation, larger subgroups will have more than required sample size while the smaller sub-groups will have less than the
sample size required. For example, if the estimates are required separately for Whites and AfricanAmericans, then just increasing the national sample would not ensure sufficient sample size for AfricanAmericans, because less than 15 percent of recipients are African-Americans for many services. In this
situation, the national sample can be disproportionately allocated by over-sampling smaller sub-groups to
ensure that sufficient samples are drawn from all target sub-groups. However, over-sampling an ethnic or
demographic group will require that agencies first list all their clients with the characteristic of interest
and then select a sample from this list by sub-group (which may exceed the capacity of many AAA
information systems).

B.2.3.2

Sample Size for Estimation of Change

If there is interest in comparing estimates from one year with another year (cross-sectionally) , or
comparing estimates of one sub-group with another sub-group, the sample size requirements are different
from those that show individual point estimates at the same level of precision. The standard error (SE) of
the difference between two independent estimates (for example, A and B) can be obtained by

SE ( A  B )  SE 2 ( A)  SE 2 ( B ) , and the half-width of the 95 percent CI is 1.96  SE( A  B)
Since the variance of the estimate (of a difference between estimates) is the sum of the variances of the
relevant individual estimates, the required sample size for estimating a difference or change is higher than
for a single point estimate.
Table B-2 presents half-widths of 95 percent CIs under a two-stage design for various sample sizes and
various averages of the two estimates to be compared. For example, if the average of the two target
characteristics to be compared is around 30 percent (for example, A=25 and B=35) and the sample size in
each sub-group is 500, to detect a difference between the two sub-groups with statistical significance, the
actual difference between the two sub-group characteristics will have to be at least 6.48 percent. This is
much higher than the corresponding half-widths presented in Table B-1 for each of the individual
estimates. That means a sample size that is sufficient to produce a reliable point estimate for each subgroup, individually, is not necessarily sufficient to detect the difference between the two sub-groups with
the same level of precision.
Therefore, if the survey is designed for use at a region or sub-group level, then the corresponding national
estimates can be compared meaningfully from one year to another, or for one service versus another (e.g.,
the percent of each service’s clients below a certain income level). For example, if the sample size is
1,000 in each year, and if the average response proportion for the two target characteristics is around 30
percent, then a difference of 4.58 percent or more between the years is detectable. The corresponding
comparison with a sub-group sample of size 500, would not allow detecting a difference unless it is 6.48
percent or more.
Table B-2 can be used to see the extent of difference that can be detected under a two-stage design, for
various sample sizes, and for various characteristics to be compared either at the national or at the subgroup level.

2-9

Table B-2
Half-widths of 95 percent confidence intervals for the difference between two
estimates by various sample sizes and for various averages of the two estimates
(computed for a two-stage design with a design effect of 1.30)

Sample size in
each group
3,500
3,000
2,500
2,000
1,500
1,000
750
500
400
300
250
200
100

Average of the estimates to be compared
10 percent

20 percent

30 percent

40 percent

50 percent

1.60
1.73
1.90
2.12
2.45
3.00
3.46
4.24
4.74
5.47
6.00
6.70
9.48

2.14
2.31
2.53
2.83
3.26
4.00
4.62
5.65
6.32
7.30
8.00
8.94
12.64

2.45
2.64
2.90
3.24
3.74
4.58
5.29
6.48
7.24
8.36
9.16
10.24
14.48

2.62
2.83
3.10
3.46
4.00
4.90
5.65
6.92
7.74
8.94
9.79
10.95
15.48

2.67
2.89
3.16
3.53
4.08
5.00
5.77
7.07
7.90
9.12
9.99
11.17
15.80

Nonresponse adjustment was done as part of the weighting process for the previous surveys and will also
be done for the 15th National Survey. The weights of the respondents were inflated to account for the
weights of the nonrespondents separately for each service. The adjustment was applied independently
within nonresponse adjustment groups defined by census region within service. That means the
nonrespondents within a group are represented by the respondents in the same group. The same types of
nonresponse adjustment will be done for the 15th survey.

B.2.4

Older Americans Act Participant Survey Instruments

The survey consists of telephone interviews with service recipients and caregivers. The interview is
structured and will contain specific questions about the mix of services the person has received and his or
her assessment of those services. Whenever appropriate, questions will contain predefined categories.
Probes will be used to facilitate obtaining complete responses to all the questions. The interviews of
caregivers will not include the questions that ask for physical functioning (except health conditions and
ADL and IADL limitations of their care recipients) nor the Emotional Well-being and Social Functioning
questionnaires. The interviews will last approximately 45 minutes and cover the topics discussed below.
This is the same process followed for each of the previous surveys.

2-10

1.

Nutrition-Congregate Meals: If a respondent receives Congregate Meals, they will
be asked a short questionnaire based on the Congregate Meals survey. This
questionnaire asks how long they have been attending the congregate meals program;
how often they eat at the site; when the last time was they ate at the site; to rate the
program; and how much of their food intake the meal provides on the days they eat at
the site.

2.

Nutrition-Home-delivered Meals: If a respondent receives Home-delivered Meals,
they will be administered a short questionnaire based on the Home-delivered Meals
survey. This questionnaire asks how long they have been receiving home-delivered
meals; how often they receive home-delivered meals; when the last time was they
received a meal; to rate the program; and how much of their food intake the meal
provides on the days they receive home-delivered meals.

3.

Transportation: All service recipients who use transportation services will be
interviewed using this survey module. The module asks how long they have been
using the transportation; how often they use it; when the last time was they used it; trip
purpose; to rate the transportation service; and about the number of times the
respondent uses the service.

4.

Homemaker/Housekeeping: Questions on the impact of homecare services will be
asked of respondents who receive homemaker or housekeeping services. Again, the set
of questions is similar to those asked of the other services: how long respondents have
been receiving homemaker services; how often they receive homemaker services;
when the last time was they used the services; to rate the program; and if they can
depend on their aides to do deliver the allotted services.

5.

Case Management: Service recipients who receive case management services will be
asked questions about their experiences with the program. They will be asked: how
long they have been receiving the services; how they would rate the various aspects of
the case management services (e.g. ease of contact with the case managers; if the case
managers understand their needs, etc.); to rate the services overall and if they
contribute to the decisions about their care.

6.

Service List: All service recipients will then be asked about the mix of services they
receive and the impact of those services. They will also be asked to rate the services
overall.

7.

Physical Functioning: This module will be asked of all service recipients (except
Caregiver clients). This survey module will include questions on: Activities of Daily
Living limitations (e.g., difficulty with personal care activities such as bathing and
dressing) and Instrumental Activities of Daily Living limitations (e.g., difficulty with
such home management activities as meal preparation, shopping, and housekeeping).
Questions about the respondents’ health are also being asked, to help with assessing
the frailty of the clients served by OAA services. Caregivers will be asked these
questions about their care recipients.

8.

Emotional well-being: This module will be asked of all participants in the surveys,
except caregivers. The questions ask if the respondent has felt sad or depressed, worried
or tense, and if they feel that they did not get enough rest, within the last thirty days.
They are also asked to describe their overall emotional well-being by responding to a
close-ended question (i.e., “Would you say…Excellent, Very Good, Good, Fair, or

2-11

Poor?”).
9.

Social Functioning: All service recipients except caregivers will be asked four
questions from the Social Functioning section of the Short-Form Survey (SF-36).
These questions ask if the respondent feels his or her social life is adequate and if
health concerns have interfered with the ability to participate in social activities.

10.

Falls: All service recipients and caregivers will be asked these five questions. The
questions ask about falling and losing one’s balance.

11.

Social Integration: All service recipients except caregivers will be asked these four
questions about social isolation.

12.

Life Changes: All service recipients except caregivers will be asked this one question
about why they sought out services.

13.

National Family Caregiver Support Program Assessment: Caregivers who receive
caregiver support services through the National Family Caregiver Support Program
will be surveyed as part of the 15th NSOAAP. This module has questions on services
offered to caregivers through the National Family Caregiver Support Program, and the
impact of those services. There are also questions about services the care recipient
receives and satisfaction with and impact of those services; support the caregiver
receives, either as part of a formal support group or from other relatives and friends;
and what kinds of other information the caregiver would find valuable. The survey
asks about the type of help the caregiver provides for the care recipient, the amount of
time they provide care, benefits caregiving provides them (companionship, a sense of
accomplishment, etc.), drawbacks of caregiving (financial burdens, lack of private
time, etc.), and demographic and health information on the care recipient. Three of the
questions for this module were adapted from an AARP survey, Caregiving in the U.S8

14.

Demographic information of the respondent: Demographic information about the
respondent will be collected, including type of area of residence (urban, suburban, or
rural), Zip Code, education level, race, gender, living arrangements (living alone, with
spouse, or with others), and income level. This module will be administered to all
participants. The caregiver survey already includes some demographic questions about
the care recipient, but the demographic information on the caregiver will be gathered
using this demographic module.

15.

COVID-19 Module: For 2021 only, all service recipients and caregivers will be asked
questions about the effect COVID-19 has had on their access to and use of Older
Americans Act programs and services during the past year.

Many of the national survey questions come from such commonly used vehicles as the Survey of Income
and Program Participation (SIPP), (e.g., the ADL and IADL questions), the Behavioral Risk Factor
Surveillance System (BRFSS) surveys conducted within each state using HHS/CDC standard questions,
and other existing surveys. As mentioned previously, other questions were developed under the POMP
contract, as well as with input from an advisory panel of aging experts under a Redesign contract. The
survey instrument for the 15th NSOAAP is virtually the same as the instrument used for the previous 14th
NSOAAP.
8

National Alliance for Caregiving and AARP (2004, April). Caregiving in the U.S. Appendix C, pp. 16-17 retrieved from AARP Web site:
http://assets.aarp.org/rgcenter/il/us_caregiving.pdf

2-12

B.3

Methods to Maximize Response Rates and Deal with
Nonresponse

Procedures for Eliciting Cooperation and Maximizing Response Rates among AAAs
Westat will use a similar procedure to select respondents for the 15th National Survey of OAA
Participants as it did in the previous seven surveys, which proved very successful. As part of the
recruitment procedures, Westat initially contacts the AAAs by sending via email an introductory letter
from ACL and an information package about the survey. A copy of these materials is also sent to each
State Unit on Aging that had AAAs sampled for the survey. (See Appendix G for the ACL letter sent to
the State Units on Aging; Appendix H for the ACL letter sent to the AAAs; and Appendix I
instructions/information package sent to the AAAs.) Following up by email and telephone, the Westat
research team works closely with each participating AAA to generate numbered lists of clients (using
client ID numbers) by service for the client sample frame. The Westat research team uses the numbered
lists of client ID numbers for the random selection of the respondents to be interviewed. To complete the
random sampling process, Westat research team members enter the total numbers of participants by
service into a computer sampling program. The sampling program randomly selects line numbers from
the numbered lists of clients. The number of clients to select per service is already entered into the
program. Westat informs the AAAs of the selected line numbers. The AAAs then provide the participant
names and telephone numbers associated with those line numbers to Westat.
Westat research assistants serving as recruitment specialists will encourage the participation of all
selected agencies by establishing rapport with contacts within each agency, coaching them on how to
generate their client lists, and assuring them that the time involved for them to complete the participant
selection procedures will be minimal. For agencies that refuse to participate, Westat will send them a
refusal conversion letter (already developed for the previous surveys), and call them one more time to try
to gain their cooperation. Once an agency refuses a second time, Westat will not try to contact them
again. Westat will also work with ACL to communicate with ACL Regional Administrators for help in
persuading reluctant AAAs to participate. Additionally, to promote agency participation, we plan to gain
the endorsement and support of the National Association for Agencies on Aging (N4A), as well as the
Advancing States [formerly known as the National Association of States United for Aging and
Disabilities (NASUAD)].
To reduce the burden for the AAAs, Westat works with software vendors of commercial client tracking
software programs commonly used by AAAs to develop step-by-step instructions for creating numbered
lists of client ID numbers by service. By using agency-assigned client ID numbers to generate numbered
lists of clients for the participant sample frame, Westat is able to screen the lists for duplicate client
entries. Additionally, the use of agency-assigned client ID numbers helps to decrease the amount of
personally identifiable client contact information collected by Westat during the survey.
Numbered lists will be developed for the following services: home delivered meals, congregate meals,
transportation, case management, and homemaker services, as well as caregivers who are served by the
National Family Caregiver Support Program.

2-13

To ensure a high participant response rate, Westat will send participants who are eligible for the telephone
survey a letter before they are contacted by an interviewer. The letters will be on each ACL’s letterhead,
with toll-free phone numbers provided to answer participants’ questions. Westat will attempt to contact
participants at different times of the day and different days of the week to maximize the possibility of
contact. Westat is also experienced in refusal conversion procedures, having achieved a refusal
conversion rate of 40 percent for the earlier surveys. (See Appendix F)

B.4

Tests of Procedures or Methods to Be Undertaken

As discussed in earlier sections, the individual service modules and the modules on physical functioning,
quality of life, and demographics have all been field tested and validated by the POMP participants. For
example, for each module the POMP grantees drew samples of service recipients, administered the
modules, and analyzed the data. The POMP grantees revised the items on the modules based on the
results of the field tests and validity studies. Some new questions added for the 14th NSOAAP (Life
Changes, Social Integration, Falls) were developed by an advisory panel of aging experts and cognitivelytested under a separate NSOAAP Redesign contract.
The majority of the items on the survey instrument for the 15th National Survey are from the previous
survey instruments. Over the years, several items have been removed from the survey instrument because
of ambiguity of the wording or in cases where the results of the item showed no variation across response
options.

2-14

B.5

Individuals Consulted on Statistical Aspects and
Individuals Collecting and/or Analyzing Data

The use of statistical sampling methods is critical to this survey. Under the supervision of ACL,
Westat is responsible for selecting the sample, conducting the interviews, data weighting and data
analysis. Below are the names and contact information of individuals responsible for the
statistical aspects of the study and individual collection and/or analysis of the data.

Administration for Community Living/Administration for Aging Personnel Responsible for
Deliverables
Susan Jenkins, Ph.D.
Director, Office of Performance and Evaluation
Administration for Community Living
U.S. Department of Health and Human Services
300 C Street, SW, Rm 1243
Washington, DC 20201
Tel: 202-795-7369
[email protected]

Westat Staff
Dwight Brock, Ph.D. – Role: Senior statistician overseeing study design, development of the
sampling plan, weighting, and data analysis
Senior Statistician
Westat
1600 Research Blvd.
Rockville, Maryland 20850
301-517-4026
[email protected]
Joanne Campione – Role: Project Director
Senior Study Director
Westat
2530 Meridian Pkwy, Suite 200
Durham, NC 27703
919-474-3506
[email protected]
Katie Hubbell– Role: Senior systems analyst with involvement in sampling data weighting, data
analysis, and reports.
Senior Systems Analyst
Westat
1600 Research Blvd.
Rockville, Maryland 20850
301-294-2020
[email protected]
Robin Ritter – Role: Recruitment Director with involvement in design, sampling, AAA and
respondent recruitment, and data analysis

Senior Study Director
Westat
1600 Research Blvd.
Rockville, Maryland 20850
301-240-314-5804
[email protected]
Jacqueline Severynse – Role: Design of the sampling plan, weighting, and data analysis.
Senior Statistician
Westat
1600 Research Blvd.
Rockville, Maryland 20850
301-517-8058
[email protected]
Lena Watkins – Role: Telephone Operations Manager
Westat
1600 Research Blvd.
Rockville, Maryland 20850
301- 294-3938
[email protected]