Supporting Statement A revised per OMB guidance 12-21-12

A. Justification
A.1. Circumstances that Make the Collection of Information Necessary
Background
The Aging and Disability Resource Center (ADRC) Program is a collaborative effort of the
Administration for Community Living (ACL) (formerly the Administration on Aging (AoA))1 and
the Centers for Medicare & Medicaid Services (CMS). ACL and CMS envision ADRCs as highly
visible and trusted places available in every community across the country where people of all
ages, incomes and disabilities go to get information on the full range of long-term support
options. ADRCs target services to the elderly and individuals with physical disabilities, serious
mental illness, and/or developmental/intellectual disabilities. The ultimate goal of the ADRCs is
to serve all individuals with long-term care needs regardless of their age or disability.
The ADRC program encourages States’ efforts to integrate the full range of long-term supports
and services (LTSS) into a single, coordinated system. By simplifying access to long-term care
systems, ADRCs are serving as the cornerstone for long-term care reform in many states.
The ADRC initiative was launched in 2003 and ADRCs are specifically defined in the Older
Americans Act of 1965 as amended in 20062 as entities established by a State as part of the
State system of long-term care to provide:
‘‘(A) comprehensive information on the full range of available public and private longterm care programs, options, service providers, and resources within a community,
including information on the availability of integrated long-term care;
(B) personal counseling to assist individuals in assessing their existing or anticipated
long-term care needs, and developing and implementing a plan for long-term care
designed to meet their specific needs and circumstances; and
(C) consumers access to the range of publicly-supported long-term care programs for
which consumers may be eligible, by serving as a convenient point of entry for such
programs. (PUBLIC LAW 109–365—OCT. 17, 2006 Title I, Section 101(44)”
As of 2012 there are over 200 ADRC sites across the nation. Since the inception of this initiative,
ADRCs have had over 4.8 million contacts with consumers, caregivers, providers and
professionals.

1

As of April 2012 the Administration on Aging, the Office on Disability and the Administration on Developmental
Disabilities were combined into a single agency, the Administration for Community Living (ACL) that supports both
cross-cutting initiatives and efforts focused on the unique needs of individual groups, such as children with
developmental disabilities or seniors with dementia. For more information see:
https://federalregister.gov/a/2012-9238 or http://hhs.gov/acl
2

http://www.aoa.gov/AoARoot/AoA_Programs/OAA/oaa_full.asp Downloaded 6-30-11;
http://www.doleta.gov/seniors/other_docs/PublicLaw109-365.pdf Downloaded 7-1-11.

Page 1 of 12

Legal and Administrative Requirements
The statutory authority for the ADRC grant program is contained in Titles II and IV of the Older
Americans Act (OAA) (42U.S.C. 3032), as amended by the Older Americans Act Amendments of
2006, P.L. 109-365. (Catalog of Federal Domestic Assistance 93.048, Title IV Discretionary
Projects).3
Title II Section 202 Subsection b of the OLDER AMERICANS ACT AMENDMENTS OF 2006
(PUBLIC LAW 109–365—OCT. 17, 2006 )4 specifically authorizes the Assistant Secretary for
Aging to “To promote the development and implementation of comprehensive, coordinated
systems at Federal, State, and local levels that enable older individuals to receive long-term
care in home and community-based settings, in a manner responsive to the needs and
preferences of older individuals and their family caregivers, the Assistant Secretary shall,
consistent with the applicable provisions of this title” including implement in all States Aging
and Disability Resource Centers—
(A) to serve as visible and trusted sources of information on the full range of long-term
care options, including both institutional and home and community-based care, which
are available in the community;
(B) to provide personalized and consumer-friendly assistance to empower individuals to
make informed decisions about their care options;
(C) to provide coordinated and streamlined access to all publicly supported long-term
care options so that consumers can obtain the care they need through a single intake,
assessment, and eligibility determination process;
(D) to help individuals to plan ahead for their future long-term care needs; and
(E) to assist (in coordination with the entities carrying out the health insurance
information, counseling, and assistance program (receiving funding under section 4360
of the Omnibus Budget Reconciliation Act of 1990 (42 U.S.C. 1395b–4)) in the States)
beneficiaries, and prospective beneficiaries, under the Medicare program established
under title XVIII of the Social Security Act (42 U.S.C. 1395 et seq.) in understanding and
accessing prescription drug and preventative health benefits under the provisions of,
and amendments made by, the Medicare Prescription Drug, Improvement, and
Modernization Act of 2003;”
With regard to the evaluation of the ADRC program, Title II Section 206 of the Older Americans
Act of 1965 specifies that the Assistant Secretary for Aging “shall measure and evaluate the
impact of all programs authorized by this Act…Evaluations shall be conducted by persons not
immediately involved in the administration of the program or project evaluated.”5

Funding Opportunity Title: Aging and Disability Resource Centers: Empowering Individuals to Navigate
Their Health and Long Term Support Options; Funding Opportunity Number: HHS-2009-AoA-DR-0915;
Catalog of Federal Domestic Assistance (CFDA) Number: 93.048
4
Downloaded from http://www.doleta.gov/reports/pdf/pl_109-365.pdf on 12-5-11
3

5

Downloaded from http://infousa.state.gov/government/branches/docs/oldamact.pdf on 12/5/2011

Page 2 of 12

Although ADRC program grantees are required to include a local program evaluation in their
grant activities there has not been a national program evaluation since the program’s inception
in 2003. This new collection of information is necessary to determine the overall effect of
ADRCs on long term support and service systems and individuals as well as whether ADRCs are
meeting their mission.

A.2. Purpose and Use of the Information Collection
This evaluation is designed to:
1) Produce generalizable outcomes about the degree to which ADRCs are meeting their
goals;
2) Identify strategies that ADRCs can utilize to effectively integrate and streamline
access to LTSS;
3) Lead to program refinement and continuous quality management; and
4) Enable ACL to accurately report on the efficacy of the ADRC program to Congress, to
governmental agencies and to the public.
ACL specifically seeks to understand whether ADRCs are meeting the LTSS needs of the aging
and disability populations. With regard to the process evaluation, ACL will gather information
about whether ADRCs, as compared to other systems, are more personalized, consumerfriendly, streamlined, and efficient. With regard to the outcomes of consumers’ contact with
an ADRC ACL is interested in determining whether consumers who access ADRCs, as compared
to consumers who access other systems, report being more empowered to make informed
decisions about their care options, are better able to plan ahead for their future long-term care
needs, and have more understanding of, and access to, long term services and supports.
This data collection is directly related to the first strategic priority of AoA (now a program
within ACL) to “Empower older people, their families, and other consumers to make informed
decisions about, and be able to easily access, existing health and long-term care options.”6
Specifically, the evaluation will provide ACL with information about consumers’ ease of access
to the full range of health and long-term care options. The evaluation study will also gather
information important for later determination of the extent to which ADRCs contribute to the
second ACL strategic priority to “Enable seniors to remain in their own homes with high quality
of life as long as possible through the provision of home and community-based services,
including support for family caregivers.” Specifically the evaluation will analyze the relationship
between the range and mix of services received by ADRC consumers with their satisfaction with
6

ACL State Directors Leadership Training presented May 23-24, 2011 Sam Nunn Atlanta Federal Center 61 Forsyth
Street, SW #5M69 Atlanta, Georgia 30303-7600

Page 3 of 12

the services, perception of the extent to which those services helped them to remain in the
community (rather than enter a residential facility such as a nursing home). In order to
measure the important ADRC goal of increasing access to preventative health benefits and the
ACL goal of enabling consumers of long term services and supports (LTSS) to remain in their
homes (i.e., reduce avoidable hospitalizations, nursing home and other residential facility
placements) this evaluation will gather relevant information including gender, birthdate, zip
code, and the last four digits of the participants’ Social Security number to allow the collection,
through a later study, of information about health care utilization and nursing home and other
residential facility diversion through an analysis of Center for Medicare and Medicaid Service
(CMS) data. This second study is anticipated to start approximately 2-5 years after the current
study to allow for changes in participants’ residential status to be realized.
In addition, staff of the Administration on Aging’s Office of Nutrition and Health Promotion
Programs will use the information and recommendations resulting from the evaluation of the
Aging and Disability Resource Centers to both determine the value of the ADRC model and to
improve program operations. The ADRC model is based on extensive research about what
works to improve the service access of older Americans and people with disabilities as well as
methods for improving systems operations. Through the ADRC demonstration grants, this
model is being tested in the real world. This evaluation will help to determine both the extent
to which, and in what ways, the ADRC model improves LTSS systems and services for individuals
seeking LTSS.




The data from the process evaluation will be analyzed to determine the extent
to which ADRC grantees are providing the full range of high quality services (e.g.,
streamlining access to public programs, serving as a no wrong door/single entry
point ). This information will be used to target technical assistance to sites which
are not implementing all aspects of the ADRC model to help them improve their
service delivery and their consumer outcomes.
Data from the consumer-level outcome evaluation will be analyzed in aggregate
form to determine the overall impact of ADRCs on ADRC consumers as compared
to similar consumers accessing services in communities not served by an ADRC.
Outcomes for which the data are most positive will be highlighted in grant
announcements and other materials developed to encourage the continued
improvement of ADRCs. Outcome data will also be analyzed in conjunction with
the process evaluation data to determine whether there are certain types of
ADRC approaches that are most beneficial to consumers overall, best suited to
specific types or subpopulations of consumers or to certain community
characteristics. These findings will be incorporated into the technical assistance
provided to actual and potential ADRC sites.

A.3. Use of Improved Information Technology and Burden Reduction
This study will utilize a mix of electronic, telephonic, and paper tools.

Page 4 of 12

The process evaluation data will be collected from State-level and local-level Aging and
Disability Resource Centers (ADRCs) and Area Agencies on Aging (AAAs). The data will be
collected primarily using the web-based tool that is currently used by ADRC grantees for
submission of their Semi-Annual Grant Reports. This tool is HIPAA compliant. Responses will
be entered into a secure electronic data base (see item Section 10). The process evaluation
survey is presented in Attachment A.
The outcome evaluation involves the collection of two types of data. The first is data about
potential individual respondents gathered by the local-level Aging and Disability Resource
Centers (ADRCs) and Area Agencies on Aging (AAAs). Potential participants will be drawn from
all consumers who contact a participating ADRC or AAA during the study enrollment period
(approximately 6 months). For the purposes of this study potential participants may be either
be the older adult or person with disabilities who is seeking services, or their spouse, family
member, or other caregiver. Participants will be screened for study eligibility (e.g., age, whether
or not the consumer has a disability, determination that contact was made for more than a
telephone number or hours of operation; see Attachment C) and, if eligible, invited to
participate in the study. Consent will be obtained (see Attachment D) and the following data
elements will be collected by the site staff (see also Attachment E):
Date of contact*

Name and telephone number of client

Need for a Spanish language interpreter

Need for hearing enhancement technology

For whom contact was made*

Mode of contact (walk-in or telephone)*

New or repeat client

Reason for contact (need at the time of contact)*

Referral or result of contact*
*Indicates information routinely collected by sites


These data will be recorded either on paper or in an Excel spreadsheet, depending on the
preference of the site. The data will be forwarded to the research team via a secure server (for
electronic data collection) or pre-paid FedEx (for paper data collection).
The second type of data collected through the outcome evaluation is consumer-level data
which will be collected with a participant experience survey (PES; see Attachment B) using, for
most respondents, a Computer Assisted Telephone Interview (CATI) system. The evaluation
team believes that use of a telephone survey will result in a representative study sample.
Specifically, while consumers may contact ADRCs in one of three ways: 1) by telephone; 2) in
person; and 3) using the ADRC’s Web site, the ADRC Technical Assistance provider estimates
that 95% of contacts are made by telephone. The contractor conducting the survey interviews,
Abt SRBI Inc., will create a database and data entry protocol using its established system for
conducting telephone surveys with study participants. The use of a CATI system reduces the
data entry and cleaning burden for the contractor by performing a number of functions to
Page 5 of 12

reduce errors including:










Providing correct question sequences;
Automatically executing skip patterns based on prior question answers (which
decreases overall interview time and consequently the burden on respondents);
Recalling answers to prior questions and displaying the information in the text of
later questions;
Providing random rotation of specified questions or response categories (to
reduce response bias);
Ensuring that questions cannot be skipped; and
Rejecting invalid responses.

The CATI system lists questions and corresponding response categories automatically on the
screen, eliminating the need for interviewers to track skip patterns and flip pages. This allows
the interviewer to focus on interviewing and allows the instrument to be administered
efficiently, thus reducing burden on the respondent, interviewers, and analysts. Moreover, the
interviewers enter responses directly from their keyboards, and the information is
automatically recorded in the computer’s memory.
The CATI system also performs a number of critical assurance routines that are monitored by
survey supervisors, including tracking the average interview length, refusal rate, and
termination rate by interviewer, and performing consistency checks for inappropriate
combinations of answers.
Although this is expected to be a small number, individuals who are unable to respond by
telephone will be offered a self-administered web-based survey option—instead of the
telephone survey. This would include consumers with hearing, speech, or other physical
disabilities who are not able to communicate by telephone (voice or TRS). Information related
to this need will be collected through a question on the Client Screening Tool that asks for the
ADRC/AAA staff person’s observation regarding the consumers’ use of a computer or similar
device to communicate. For individuals who are unable to respond by telephone or web-based
survey, the research team will offer a self-administered paper form option.
A.4. Efforts to Identify Duplication and Use of Similar Information
The Administration on Aging (AoA) does not currently collect the proposed data. Further, a
comprehensive environmental scan, including a literature review, was conducted and found no
studies with a similar combination of objectives, design, setting and study participants. Hence,
the proposed data collection is not duplicative.
A.5. Involvement of Small Businesses or Other Small Entities
This project does not involve small business or other small entities.

Page 6 of 12

A.6. Consequences of Collecting the Information Less Frequently
Data for the process evaluation will be collected only one-time from the state program
directors and from contacts at the local programs. PES data for the outcome evaluation will be
administered via the CATI, and collected by research staff only one time per respondent. If
these data are not collected AoA will not be responsive to the requirement in Title II, Section
206 of the Older American Act of 1965 that all authorized programs should be evaluated. The
Act specifically authorizes the Assistant Secretary of Aging to establish ADRCs in every state and
to measure the impact of all programs authorized in the Act (The response A1 above contains
text from this section of the Act). Further, if the information is not collected, ACL and AoA will
lack important information needed for program improvement.
A.7. Special Circumstances Relating to Guidelines of 5 CFR 1320.5
This request is consistent with the general information collection guidelines of 5 CFR
1320.5(d)(2). No special circumstances apply.
A.8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the
Agency
A.8.a. Federal Register Notice
As required by 5 CFR 1320.8(d), notice was published in the Federal Register on October 14,
2011 Volume 76 number 199 for 60 days (see Attachment L). One set of comments was
received in response to the 60-day Federal Register Notice Published October 14, 2011. The
comments focused on both the evaluation design and the specific wording of the tools. There
were only 2 references to the cost or hour burden of the proposed information collection. The
complete set of comments with responses is available in Section 2 of the Attachment available
at http://www.aoa.gov/AoA_programs/Tools_Resources/docs/ADRC_Eval_Data_Collection.pdf.
Comments were addressed by both revising the data collection tools and through revamping
the evaluation approach which resulted in replacing consumers from Centers for Independent
Living with consumers from AAAs as a comparison group for the disability study.
A.8.b. Outside Consultations
Planning for data collection in this study has involved extensive consultation between ACL and
the IMPAQ International LLC/Abt Associates Inc. (the research team contracted to conduct the
ADRC evaluation). The design of the data collection schedule and methodology was informed
by the input of two outside consultants; Suzanne R. Kunkel, PhD, Scripps Gerontology Center,
Miami University, and Robert Newcomer, PhD, University of California, San Francisco. In
addition, valuable information was obtained during a series of conference calls with staff from
ADRCs, AAAs, and the Centers for Independent Living and during a meeting with a Technical
Expert Panel that contains experts in the fields of both Aging and Disability services. These
expert groups provided guidance on the feasibility of the research design and methodology
including the likelihood of site staff agreeing to recruit participants and participate in the webbased, process evaluation survey.

Page 7 of 12

A.9. Explanations of Any Payment or Gift to Respondents
This study does not include any payments or gifts to either organizational-level or consumerlevel respondents.
A.10. Assurance of Confidentiality
Individuals and organizations will be assured of the confidentiality of their replies under Section
934(c) of the Public Health Service Act, 42 USC 299c-3(c). They will be told the purposes for
which the information is collected and that, in accordance with this statute, any identifiable
information about them will not be used or disclosed for any other purpose. Respondents will
be informed in the survey introduction (process and outcomes surveys) that their answers will
be kept private and used only for statistical purposes and that participation in the survey is
voluntary (see Attachments D, F, and G). Identifying information such as names, addresses,
telephone numbers which allow the research team to contact consumer-level respondents will
be stored in a secure database separate from the database created through the CATI system
that will house consumers’ responses to the Participant Experience Survey (PES). Medicaid
and/or Medicare numbers will be obtained from respondents and recorded in the CATI system
with other survey data and kept in the Linux file system, a secure data system.
The IMPAQ/ABT team is very experienced in collecting and handling secure data. Abt
Associates, the contractor primarily responsible for the data collection and storage, is
committed to research practices conforming to basic ethical principles and Federal regulatory
requirements governing research involving human subjects. All research involving interactions
or interventions with human subjects that pose no more than minimal risk to those subjects is
within the purview of the Abt Associates IRB. Abt Associate’s Federal-wide Assurance (FWA)
with the Office for Human Research Protections of the U.S. Department of Health and Human
Services provides that the company will assure compliance with the Terms of Assurance for
Federally-supported research. The ABT Institutional Review Board reviewed and approved the
proposed data collection tools and privacy documents.
The ABT IRB granted approval for data collection protocols (see Attachments A – K) and data
management plans from Abt’s IRB for both the process and outcome evaluations (See
Attachment M).
A.11. Questions of a Sensitive Nature
This project includes questions that may be considered sensitive. The Agency Data Collection
Tool found in Appendix E requests respondents’ names and contact information to allow the
research team to contact them to administer the PES. The PES found in Appendix B requests
information about consumers’ level and type of disability, and demographic characteristics
including race and living status. These data will be used to both determine the extent to which
study sites are serving the intended populations as well as to allow for analysis of outcome data
by important subgroups. All data will be protected to the fullest extent possible by using
encrypted, password protected data files.

Page 8 of 12

A.12. Estimates of Annualized Burden Hours and Costs
Process Evaluation
This collection will involve a one-time request for information from a total of 463 organizations.
Specifically, the process evaluation survey will be conducted with all 43 State Units of Aging and
approximately 400 local ADRCs7 as well as the 20 AAA sites selected for participation in the
outcome evaluation. Organization directors will receive a letter of support from ACL (see
Attachment H) followed by a recruitment email (see Attachment I) from the research team.
The data will be collected using a web-based survey (see Attachment A) that will take
approximately one hour to complete. The burden estimate of one hour per site is based on a
comparison of the process evaluation survey with the existing semi-annual reporting
requirements.
Outcome Evaluation
Site Recruitment. The outcome evaluation will be conducted with 43 organizations (i.e., 23
ADRC programs and 20 AAA programs). Once sites have been selected as eligible for
participation (see Statement B for a description of site selection), they will be sent a letter of
support from ACL (see Attachment J). Recruitment will be conducted via a telephone
conversation with the director of each selected site (see Attachment K). Although the target
number of participating sites is 43, we estimate a response rate of approximately 70 percent
(approximately 60 recruitment calls). It is expected that each recruitment call will last fewer
than 10 minutes.
Participant Recruitment. For the outcome evaluation, staff at each organization will administer
a brief eligibility screening, and collect a minimal amount of data from consumers who agree to
participate in the study. There are two arms to the outcome evaluation study; an older adult
study (ages 60 and over) and a disability study (ages 18 and over with a disability). We estimate
that an average of 78 participants will be recruited at each site (see Statement B for sampling
description). Assuming a 60 percent participation rate, each site will need to screen
approximately 131 consumers, and we expect that eligibility screening and recruitment will
take approximately five minutes to complete (including the time to record the eligibility and
contact information). Overall, the outcome evaluation survey (PES; see Attachment B) will be
conducted with a total of 3,389 respondents. Approximately 70 of these respondents will be
included in a pilot test of the PES. The PES is estimated to take 20 minutes to complete.
Exhibit 1 shows the number of burdened hours for the process and outcome evaluations.
Exhibit 2 shows the estimated annual cost burden to the respondent, based on their time to
participate in this research. The annual cost burden is estimated to be $39,345.
Exhibit 1. Estimated annualized burden hours

7

This number may be slightly revised as the research team receives updated data from the ADRC Technical
Assistance Contractor.

Page 9 of 12

Number
of sites

Form Name
Process Evaluation
State and local-level ADRC Directors
and directors of AAAs
Outcome evaluation
Site telephone recruitment
Organization staff
screening/recruitment of PES
participants

Individuals
per site

Total
respondents

Hours per
response

Total burden
hours

463

1

463

1.00

463

60

1

60

.170

10

43

Approx. 131

5,650

.083

469

3,389

.330

1,118

PES

-

Total

-

Approx.
78
-

9562

2060

Exhibit 2. Estimated annualized cost burden

Staff Type

Number
of sites

Total burden
hours

Average
hourly wage
rate

Total cost
burden

Process Evaluation
State and local-level ADRC Directors
463
463
*45.03
$20,849
Outcome evaluation
Site telephone recruitment
60
10
*45.03
$450
Organization staff
screening/recruitment of PES
43
469
*23.65
$11,092
participants
PES
1,118
***6.22
$6954
Total
566
2,060
$39,345
*Based upon the mean of the average wages, National Occupational Employment and Wage Estimates,
U.S. Department of Labor, Bureau of Labor Statistics. May 2010. Average wages for medical and health
services manager ($45.03); health care social worker ($23.65).
**This is an average salary for one director and four organization staff at each site.
** *Based on average Social Security benefits from “Monthly Statistical Snapshot”, May 2011,
http://www.ssa.gov/policy/docs/quickfacts/stat_snapshot/

A.13. Estimates of Annualized Respondent Capital and Maintenance Costs
There are no direct costs to respondents other than their time to participate in the study.

Page 10 of 12

A.14. Estimates of Annual Cost to the Federal government.
The estimated cost of funding this data collection effort was $1,101,043 dollars (see Figure 3).
Process Evaluation:
Number of completed web-surveys (including 9 Pre-test)
Total estimated cost of conducting survey
Cost per completed interview

463
$15,578
$34.77

Outcome Evaluation:
Number of completed telephone interviews (including 70 pilot; 40 cognitive test) 5,248
Total estimated cost of conducting survey
$1,085,465
Cost per completed interview
$206.83

Figure 3: Estimated Annual Cost of Data Collection
Cost Component
Process Evaluation

Year 1

Year 2
$15,578

Year 3

Total
$15,578

Outcome Evaluation

$11,463

$413,533

660,469

$1,085,465

TOTAL

$11, 463

$429,111

$660,469

$1,101,043

This estimate is based on the total cost of the awarded survey contract divided by the specified
number of completed pretest/survey interviews.
A.15. Program Changes or Adjustments in Burden Hours or Costs
These are a new data collection efforts and do not build on a previous submission.
A.16. Plans for Publication
At the current time, there are no plans to publish the results of this study beyond producing a
final report that will be submitted to the ACL in September 2014. The final report will include a
description of the study design, data collection procedures, and results. It will also include
details about the sampling frame, the survey participation rate, the weighting procedures, and
copies of the questionnaires. Data collection is planned for April 1, 2013 – July 31, 2013
(process evaluation) and April 2013 to October 2013 (outcome evaluation). The overall project
began on September 30, 2010 and will end on September 29, 2014.
A.17. Exemption for Display of Expiration Date
ACL does not seek this exemption.
A.18. Exemptions to the certification statement
No exceptions to the certification statement are made.
Page 11 of 12

List of Attachments:
A. PE Survey
B. OE Participant Experience Survey
C. OE Eligibility Screening Tool
D. OE Study Description/Agreement to Participate
E. OE Organization Data Collection Form
G. OE Statement of Informed Consent
H. PE Letter of Support
I. PE Email Invitation
J. OE Letter of Support
K. OE Telephone Recruitment Script
L. 60-Day Federal Register Notice
M. IRB Approval of the data collection tools

Page 12 of 12