A. Justification
A.1 Circumstances Making the Collection of Information
Necessary
Introduction
This OMB package requests clearance to conduct a three-year longitudinal survey of Older Americans Act (OAA) service recipients. The baseline survey will be the sixth in a series of national surveys of OAA clients. The first five surveys provided important cross-sectional data on service recipients (e.g., consumer assessment of services, reported outcomes, physical functioning, quality of life, and demographic information). The longitudinal survey will continue to provide rich cross-sectional data as well as an opportunity to examine the predictors of nursing home placement and, most importantly, the relationship of the receipt of OAA services to the delay in nursing home placement.
AoA’s Strategy of Program Improvement
The Administration on Aging (AoA) has an ongoing strategy of program improvement through enhanced program performance measurement, in compliance with requirements of the Office of Management and Budget’s (OMB) program reviews, the Government Performance and Results Act (GPRA), and the OAA Section 202(f), by proposing to conduct further studies of program outcomes .
Previously, AoA conducted five cross-sectional surveys. The five surveys and their OMB control numbers are listed below:
Two pilot studies of Older Americans Act Title III Service Recipients in 2003 and 2004 (OMB control numbers 0985-0014 and 0985-0017);
Third National Survey of OAA Title III Service Recipients conducted in 2005 (OMB control number 0985-0020);
Fourth National Survey of OAA Title III Service Recipients conducted in 2008 (OMB control number 0985-0023); and
Fifth National Survey of OAA Title III Service Recipients conducted in 2009 (OMB control number 0985-0023).
The surveys have enabled AoA to establish baselines and performance targets for annual and long-term outcome measures required by OMB and incorporate new performance information in agency budget justifications and performance plans for FY 2005 and FY 2006. Further, the studies demonstrated that services provided under Title III:
Are effectively targeted to vulnerable populations
Are provided to individuals who need the services
Are highly rated by recipients (quality)
Provide assistance that is instrumental in enabling recipients to maintain their independence.
Performance Measurement Requirements
GPRA1 requires federal agencies to develop annual and long-term performance outcome measures and to report on these measures annually. Section 202(f) of the OAA2 requires AoA to work collaboratively with State agencies and area agencies on aging (AAAs) to develop performance outcome measures.
Since the passage of GPRA in 1993, AoA has accepted GPRA as an opportunity to document each year the results that are produced through the programs it administers under the authority of OAA. It is the intent and commitment of AoA, in concert with State and local program partners, to use the performance measurement tools of GPRA to continuously improve OAA programs and services for the elderly.
As described on AoA’s website: “In order to gather information on the performance of its program, the Administration on Aging surveys the participants in its Older Americans Act programs. These national surveys provide a portrait of who receives these services and how they assess the quality of the services received.” 3
Title III of the OAA establishes a home and community-based care program for older persons and their caregivers, to enable them to live as independently as possible for as long as possible. States and local agencies are given much latitude to design services tailored to the needs of their regions and communities. One challenge for AoA is to devise a means to improve the performance of the program nationally, while preserving and promoting the diversity of program design. AoA has chosen to work toward improved program performance throughout the Aging Services Network by working collaboratively with States and AAAs to develop performance outcome measurement tools. The tools identify elements of service quality so that states and AAAs can improve service systems at the local level. These same tools can also be employed by AoA to measure program performance at the national level.
Performance Outcomes Measures Project (POMP)
For the past 10 years, AoA has sponsored the Performance Outcomes Measures Project (POMP) demonstration, in which grants are awarded to states, who then work collaboratively to develop survey instruments that measure elements of service quality and consumer reported outcomes for various services provided under Title III of the OAA. Surveys have been developed for the following topics:
Service Domains:
Nutrition (including congregate and home-delivered meals)
Transportation
Information and Assistance
Homemaker/Housekeeper
Personal Care
Caregiver Support
Case Management
Senior Centers
Client Characteristics:
Physical Functioning
Demographics
Emotional Well-Being
Social Functioning
POMP is demonstrating the ability of states and AAAs to apply statistically sound sampling techniques to obtain numeric measures of program performance.
The survey instruments developed under POMP – along with various tools necessary for implementation – can be found at www.gpra.net. These performance measurement surveys have enabled some local agencies to obtain additional financial support and improve program management. Examples of uses of performance measurement at the state and local level follow:
The Hawkeye Valley Area Agency on Aging in Waterloo, Iowa compiled information on the level of client support and satisfaction with services and received additional funding from the United Way for exemplary programs.
The Area Agency on Aging in Cincinnati, Ohio expanded the use of Home Care Client Satisfaction Measure (HCSM) and incorporated it into an ongoing part of its case management process for all clients to improve service quality.
The Florida Department of Elder Affairs developed a computer simulation model that demonstrated the impact of home care programs on reducing nursing home admissions and showed the savings in Medicaid funds
Advanced POMP
A subgroup of POMP grantees participated in the Advanced POMP project, which focused on modeling the extent to which the receipt of OAA services is related to the time delay in nursing home placement. The grantees from North Carolina, Georgia, New York, Iowa, and Rhode Island supplied Westat with administrative datasets of AAA clients. The dataset contained information about the specific services that the clients received , measures of activities of daily living (ADLs), instrumental activities of daily living (IADLs), and demographics (e.g., age, race/ethnicity, presence/absence of a caregiver, and living arrangements). The datasets also contained the date the client started receiving services and the date the client stopped receiving services (if indeed the client did stop receiving services), and the outcome (e.g., nursing home placement, mortality, continue to receive services, other).
The contractor analyzed the data using a Cox proportional hazards regression model that not only examined the risk factors for nursing home placement, but examined the time in the community as a result of receipt of OAA services. The results across all states showed that the more services clients received (controlling for ADLs), the longer they remained in the community. The contractor repeated the analysis with a subset of respondents in the Health and Retirement Study (HRS) who had similar characteristics to those of the clients in the administrative datasets (e.g., age, race/ethnicity, receipt of services). The results of the analysis of the HRS were similar to the results of the OAA service recipients. The increase in the number of services received was related to a longer time in the community.
The proposed longitudinal survey will provide AoA with an opportunity to model the relationship of the receipt of services to a delay in nursing home placement on a national level with increased precision. It will also provide an opportunity to assess individual change in response over time with increasing age, and reduce bias due to differential selection or confounding factors. Finally, the longitudinal surveys will also provide rich datasets for cross-sectional analysis at more than one point in time.
The results of this information collection will be used to:
Report on FY 2011 performance targets as required by OMB.
Provide data on the extent to which the receipt of OAA services is related to a delay in nursing home placement.
Provide refined national benchmarks for use by states and AAAs.
Continue to explore the feasibility of substituting survey reporting for some of the NAPIS reporting requirements.
Provide secondary data for analysis of various Title III program evaluations.
Provide performance information for key demographic subgroups, geographical subregions, and different types of AAAs which will enable AoA to identify variations in performance and examine the need for additional targeted technical assistance.
The data will be used by the Assistant Secretary on Aging in testimony and presentations; it will be incorporated into the agency’s Annual Report; and it will be used by program staff to identify areas that may need attention at the national level. For example, the AoA nutritionist is interested in examining nutritional intake information by key population subgroups to identify potential areas for technical assistance initiatives.
Use of Client Tracking Software to Generate Client Lists for Sampling
The proposed procedures and materials requesting information from the agencies, as well as the telephone surveys of respondents, have been designed in a way that minimizes respondent burden.
To reduce the burden for the Area Agencies on Aging (AAAs), the contractor (Westat) has developed procedures for client sampling that utilize the same client tracking management information systems that are used by States and AAAs to create the required State Program Reports for AoA. Since the implementation of the fourth national survey in 2008, the contractor (Westat) has worked cooperatively with vendors of commercial off-the-shelf client tracking software programs most commonly used by the State and Area Agencies on Aging to develop step-by-step instructions for the AAAs to use to generate client lists by service to use for a sample frame. It is estimated that over 95% of the AAAs now have this technological capability and are able to follow the instructions to produce their client lists by service. We will provide similar instructions for the sixth, seventh, and eighth surveys.t tracking software system known as “SAMS.”
In specific states that have their own proprietary client tracking software, the contractor (Westat) has worked directly with an Information Technology Specialist at the State-level to generate electronic client lists for all of the AAAs selected for the national survey. This further reduces the burden for AAAs in states that have their own proprietary software.
Use of Survey Web Site
An AoA National Survey web site application has been developed to support and assist with data collection. For the fifth survey, the contractor (Westat) designed and utilized a secure website which the AAAs used to upload their lists of selected clients. That website will be updated and further refined for the sixth national survey.
The web site is divided into two major sections: the public and the restricted-access sections. The public section is accessible to the general public, without restrictions. It includes background information, endorsement letters from national and state associations for agencies on aging, frequently asked questions, results of previous AoA National Surveys, a copy of the survey instrument, and links to other relevant AoA web sites. The purpose of this section is to provide State and Area Agencies on Aging, professionals in the field of aging, and service recipients and their families with information about the data collection effort and uses of the data.
The restricted-access section of the web site houses an electronic records receipt system. Area Agencies on Aging have the option of submitting confidential client data to Westat via electronic files using the project web site. The web site was written in Active Server Pages (ASP), HTML, and JavaScript and uses the industry-standard SSL (Secure Socket Layer) encryption for secure File Transport Protocol (FTP) data submissions. Agencies choosing this option will receive usernames and passwords that enable their staff to sign on to the file upload utility on the web site. This system supports files in a large variety of file formats. Each agency's data file will be processed according to its structure and content.
Westat programming staff will manually map and convert the data items in each agency’s file to create standardized records for further processing. As each file is received, this system will log the source agency, date received, and file type.
Only agencies that have been selected to participate in the survey will have access to this area. Unique user IDs and temporary passwords will be assigned to each AAA at the time they are selected into the sample. The ID and password will be provided with other survey materials to the AAA.
Use of Computer Assisted Telephone Interviewing (CATI)
Westat (the contractor) will use computer-assisted telephone interviewing (CATI) technology to conduct the surveys of OAA service recipients and record the responses. Westat’s CATI capability includes customized software systems for scheduling, interviewing, and data handling and utilizes high-speed data networks and centralized voice and data monitoring. A single database is used to monitor and direct the interviewers. The Scheduler, a computerized survey control system, makes interviewer assignments, records the disposition of sample cases, and helps survey managers monitor performance.
Westat will attempt to contact each person in the sample, making multiple calls at different times and days when necessary. To reduce the burden for the respondents, Westat will schedule appointments for calls at times that are convenient for them. For Spanish-speaking respondents, Westat uses specially trained bilingual interviewers to conduct the interviews in Spanish. If other special arrangements are necessary (e.g., interpreter, proxy needed, mailout requested, interview needed to be conducted over several sessions), the respondent can be further accommodated.
Westat will take the AoA-approved finalized version of the survey instruments and program them into its CATI system. This involves:
Inserting specifications into the English version of the questionnaire;
Preparing the specifications for the CATI programmer;
Translating the questionnaire from the specifications into Spanish; and
Programming and testing both versions of the questionnaire into CATI.
Details of how skips will work in the questionnaire are included in the design document, as are the needed question variations. For example, some questions may need to be asked differently, depending on the answers to previous questions. In particular, if a respondent told us they live with others, the next question we would ask would be, “Do you live with your spouse?” However, if the respondent told us they lived alone, the follow-up questions will not be asked, and CATI will automatically skip to the next question.
The use of the CATI system in combination with Westat’s highly structured telephone interviewer training and procedures ensures that interviewers conduct the surveys in a professional, controlled, and consistent manner.
Every effort is being made to avoid duplication and minimize respondent burden. Over the last eight years, Westat conducted the First and Second National Surveys (pilot studies) of OAA Participants and the Third, Fourth, and Fifth National Surveys of OAA Title III Service Recipients. As a result of the information gathered, modifications have been made to the data collection procedures and to the survey instruments. We believe we have reduced agency and respondent burden to the minimum level possible to achieve the survey's objectives.
No small businesses will be involved in this study.
It is important to follow the respondents over a three-year period to determine the extent to which OAA services help clients remain the community. In the past, the survey instrument asked respondents about the extent to which the receipt of services helped them live at home longer than if they had not received the services at all. For all of the services, respondents indicated that the services did help them stay in the community longer. The longitudinal design of the Sixth National Survey will provide quantitative data to determine the extent to which the services do enable clients to remain in the community as measured in months and/or years.
Interviewers will ask respondents for the permission to conduct the telephone interview once each year for three years. Respondents that agree to participate in the longitudinal component of the study will receive reminder cards at 6-month intervals.
We believe that the collection of data at one-year intervals over a three-year period will provide sufficient information to measure change over time in physical functioning, consumer assessment of services, and self-reported outcomes. Most importantly, it will also provide an opportunity to collect information on those clients who no longer receive services for a variety of reasons, including placement in a nursing home or assisted living facility.
There are no legal obstacles to reduce the burden.
The data collection effort will be conducted according to the guidelines specified in 5 CFR § 1320.6. No special circumstances are known that would cause inconsistency with these guidelines.
A 60-day Federal Register Notice was published in the Federal Register on May 12, 2010, Volume 75, Number 91, page 26759 (see Appendix D). There were no public comments.
The survey instruments for this proposed information collection are based on those developed by AoA POMP grantees representing State Units on Aging and AAAs. POMP grantees who have worked on the survey instruments include state and local level representatives from Arizona, Florida, Georgia, Massachusetts, New York, North Carolina, and Ohio. The development of the survey instruments has been an iterative process. There were no areas of disagreement during the latest POMP revisions.
The POMP grantees tested the instruments with service recipients at the local AAA-level using several methods:
Field-tested the survey instruments with a sample of service recipients and revised the instruments based on their experience.
Conducted cognitive testing to ensure that the items on the survey instruments were interpreted as intended.
Conducted validity testing on the survey instruments.
Westat (the contractor) has also consulted representatives from different State Units on Aging to develop and test the instructions and procedures for generating client lists used for sampling. The state representatives who have reviewed the instructions and procedures include:
Robin Tofil, Connecticut Department of Social Services, Aging Services Division
Scott Wamsley, Michigan Office of Services to the Aging
Leonard Eshmont, Virginia Department for the Aging
No payments or gifts will be given to respondents.
Confidentiality and anonymity are important parts of the survey design. In response to this concern, Westat will ensure the anonymity of all individuals who provide data. A pledge of confidentiality and anonymity is a major positive incentive for potential respondents to participate in the survey. Its absence would be a significant deterrent and could create complications in implementing the survey.
Westat will take the following precautions to ensure the confidentiality and anonymity of all data collected:
All Westat project staff, including recruitment specialists, telephone interviewers, research analysts, and systems analysts, will be instructed in the confidentiality requirements of the survey and will be required to sign statements affirming their obligation to maintain confidentiality;
Only Westat staff who are authorized to work on the National Survey have access to client contact information, completed survey instruments, and data files.
Data files that are delivered will contain no personal identifiers for program participants; and
Analysis and publication of survey findings for the participant survey will be in terms of aggregated statistics only.
Westat, the contractor for administering the survey instrument and collecting the data, has extensive experience in protecting and maintaining the confidentiality of respondent data collected from surveys. To ensure confidentiality, Westat has drawn from its experience in designing the data collection procedures incorporated in this program. In addition to the confidentiality agreement, Westat has implemented several other procedures to protect confidentiality of survey participants.
Data is saved on secure network folders only accessible to authorized users. No data is ever stored on laptop computers. At the end of the survey, all confidential data is permanently deleted.
For the Sixth National Survey and subsequent follow-up surveys, AAAs will be instructed to submit confidential client data to Westat via electronic files using the project web site. This web site is written in Active Server Pages (ASP), HTML, and JavaScript and uses the industry-standard SSL (Secure Socket Layer) encryption for secure File Transport Protocol (FTP) data submissions. Agencies will receive usernames and passwords that enable their staff to sign on to the file upload utility on the web site. The passwords are created by a password generator which creates random passwords that are highly secure due to a combination of lower and upper case letters, numbers and punctuation symbols. The database containing the client survey data is not accessible via the Internet; it resides on a server inside the Westat firewall. Only Westat Data Collection Program staff members have access to the master survey database.
For AAAs that may experience problems with the survey website and wish to send client data electronically by email, we instruct the AAAs to password protect the file containing the data. Password protection of client data sent electronically by email is required not only for transmission between the AAA and Westat, but even internally within Westat. Additionally, we provide the AAAs with an email address to a secure dedicated project email box ([email protected]) which cannot be accessed remotely.
For the small number of AAAs that are not able to generate client records by service electronically, they can submit client information in a hard copy format (fax, FedEx, U.S. Postal Service). Hard copies of client information are stored in locked filing cabinets within a locked room. At the conclusion of the survey, all hard copies of client data are shredded.
A secure fax machine dedicated solely to this survey is used to receive faxes from AAAs that choose to transmit their data by fax. The fax machine is located within a locked project room. AAAs that wish to transmit their data by fax are asked to call to Westat staff to alert them to watch for and intercept an incoming fax. If the fax machine is busy, it does not roll over to any other fax machine.
All respondents in this data collection effort are assured of the confidentiality of their answers. Respondent data are aggregated and estimates are produced and published at the both at the national level and at the geographic regional or demographic sub-group level. No individual-level data are published, nor are they accessible or provided to anyone outside the Westat Data Collection Program staff.
A pre-notification letter mailed to potential respondents contains essential program information and assurances of confidentiality that enable the person to make an informed decision regarding his or her voluntary participation in the data collection effort. A sample of the pre-notification informational letter sent to potential survey participants appears in Appendix F as part of the information packet sent to the AAAs.
The AoA National Survey informs respondents that their responses to all questions are voluntary. We assure them that their survey responses will remain confidential. Respondents can refuse to answer any question, and the interviewer will move on to the next question on the survey instrument. Additionally, respondents are permitted to stop at any point and to continue the interview at a later time.
The physical functioning module contains questions on the ability of respondents to perform certain tasks, such as getting around inside and outside the home, getting in and out of a chair or a bed, getting to and using the toilet, etc, as well as questions asking about their health conditions. While these types of questions might be considered to be sensitive, most respondents do not object to answering these types of questions, especially when presented as part of a battery of physical functioning questions. In addition, analysis of Activities of Daily Living (ADL) and Instrumental Activity of Daily Living (IADL) limitations, in conjunction with outcomes and the type of services received, is an important outcome measure. This kind of information, along with responses to questions on health conditions, can tell us about the frailty of the respondents served by the nutrition (home-delivered and congregate meals), home care (homemaker/housekeeping), case management, and transportation services, and the people who are able to maintain their independence, rather than enter nursing homes, because of those services. Caregivers will also be asked about the health conditions and ADL and IADL limitations of their care recipients to allow for analysis of the frailty of care recipients whose caregivers are part of the National Family Caregiver Support Program. Again, respondents can opt to not answer questions that make them uncomfortable.
We estimated the respondent burden for the survey instruments based on the First and Second National Surveys of OAA Participants and the Third, Fourth, and Fifth National Surveys of OAA Title III Service Recipients. The cost to respondents who participate in the survey will be in terms of their time only. The Service Recipient survey instrument takes about 30 minutes (1/2 hour). Most of the Service Recipients are retired. The Caregiver survey will also take about 30 minutes. Based on the valuation of a participant's time at $20 per hour, the respondent burden for each participant will be $10 for both the Service Recipient and $10 for the Caregiver surveys and $80 for the agency respondent selection process (estimated at 4 hours of agency personnel time). Exhibit A-1 presents the estimated hour and annual cost response burden by respondent.
Exhibit A-1. Estimated Hour and Annual Cost Response Burden
Respondent/Data collection activity |
Number of respondents |
Responses per respondent |
Hours per response |
Annual burden hours |
Annual burden (cost) |
Area Agency on Aging: Respondent selection process
|
300 |
1 |
4.0 |
1200 |
$24,000 |
Service Recipients: National survey using questions from the Performance Outcome Measures Project
|
4000 |
1 |
.50 |
2000 |
$40,000 |
National Family Caregiver Support Program Clients: National survey using questions from the Performance Outcome Measures Project |
2000 |
1 |
.50 |
1000 |
$20,000 |
Total |
6300 |
1 |
5.00 |
4,000 |
$84,000 |
* It is important to note that not all of the individual respondents (6000 for the national survey) will be asked to complete all of the questionnaire modules (see Sampling Plan).
Total annual cost burden excluding wages of agency time and respondent time is zero (see Exhibit A-1).
The overall cost of this research to the Federal Government is presented in Exhibit A-2.
Exhibit A-2. Total Annualized Cost to the Federal Government [Based on Year 1]
Category |
Costs |
Personnel |
$398,865 |
Telephone (long-distance telephone survey) |
$23,821 |
Other direct |
$24,126 |
Total direct charges (per task order) |
$446,812 |
Indirect charges |
$551,720 |
Total |
$998,532 |
The change consists of minor adjustments to the survey instrument without increase in burden hours.
In this section, the range of analyses that will be conducted is described using the performance measurement data. The research team suggests several levels of analysis to first describe the characteristics of clients and the range of services provided by State Units on Aging (SUAs) and AAAs, noting similarities and differences in the size and scope of program design and operations. The research team will paint a descriptive profile of OAA clients, paying special attention to risk factors that prior research shows are highly correlated with quality-of-life outcomes for older persons. These risk factors include ADL and IADL limitations, as well as income, educational level, living arrangements, age cohort, gender, race and ethnicity, and area of residence by degree of urbanization. In conjunction with the type and severity of ADL/IADL limitations that clients report, these factors are indicative of risk for loss of independence and other adverse outcomes. For example, it is known that among the elderly, those living alone are at higher risk for institutional placement. Educational attainment, in conjunction with age, is the most powerful predictor of need for services and outcomes for older persons.4
In conjunction with the performance measure on service recipients, the survey will include questions on ADL limitations (e.g., difficulty with personal care activities such as bathing and dressing) and IADL limitations (e.g., difficulty with such home management activities as meal preparation, shopping, and housekeeping), and health conditions.
The timetable for the baseline data collection and the two follow-up data collections is shown in Exhibit A-3.
Exhibit A-3. Data Collection Timetable
Survey Cycle |
Data Collection Activity |
End dates |
Sixth National Survey/ Baseline Data Collection |
Telephone/email contact with agencies to draw sample |
1 month after OMB clearance |
Sixth National Survey/ Baseline Data Collection |
Telephone survey of participants |
5 months after OMB clearance |
Sixth National Survey/ Baseline Data Collection |
Data editing, coding and key entry, data analysis |
7 months after OMB clearance |
Sixth National Survey/ Baseline Data Collection |
Deliver data to AoA |
9 months after OMB clearance |
Sixth National Survey/ Baseline Data Collection |
Final report on baseline data collection |
15 months after OMB clearance |
Seventh National Survey/ Data Collection #2 |
Telephone/email contact with agencies to refresh sample |
13 months after OMB clearance |
Seventh National Survey/ Data Collection #2 |
Telephone survey of participants |
17 months after OMB clearance |
Seventh National Survey/ Data Collection #2 |
Data editing, coding and key entry, data analysis |
19 months after OMB clearance |
Seventh National Survey/ Data Collection #2 |
Deliver data to AoA |
21 months after OMB clearance |
Seventh National Survey/ Data Collection #2 |
Final report on data collection #2 |
27 months after OMB clearance |
Eighth National Survey/ Data Collection #3 |
Telephone/email contact with agencies to refresh sample |
25 months after OMB clearance |
Eighth National Survey/ Data Collection #3 |
Telephone survey of participants |
29 months after OMB clearance |
Eighth National Survey/ Data Collection #3 |
Data editing, coding and key entry, data analysis |
31 months after OMB clearance |
Eighth National Survey/ Data Collection #3 |
Deliver data to AoA |
33 months after OMB clearance |
Eighth National Survey/ Data Collection #3 |
Final report on data collection #3 |
39 months after OMB clearance |
AoA is not seeking an exemption from displaying the expiration date of OMB approval.
AoA is not requesting any exceptions from OMB Form 83-I.
1 P.L.103-62 [ http://thomas.loc.gov/cgi-bin/bdquery/R?d103:FLD002:@1(103+62) and http://www.whitehouse.gov/omb/mgmt-gpra/gplaw2m.html]
4Ficke, R.C. Digest of Data on Persons with Disabilities. Washington, DC: National Institute on Disability and Rehabilitation Research, 1992.
Page
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | 7420.01: OMB Package. Section A. Introduction |
| Author | MARKOVICH_L |
| File Modified | 0000-00-00 |
| File Created | 2021-02-02 |