CaregiverOutcomeEvaluation_PRA_SectionA_Passback_CLEAN 3-2-16 v2

Application for New Data Collection:

Supporting Statement for

Outcome Evaluation of

Older Americans Act Title III-E

National Family Caregiver Support Program

Section A

Revised per OMB guidance 3/1/2016

Submitted to: U.S. Administration for Community Living Administration on Aging 1 Massachusetts Avenue, NW Washington, DC 20001

Submitted by: WESTAT 1600 Research Boulevard Rockville, Maryland 20850-3195 (301) 251-1500

Table of Contents

Chapter Page

A. Justification 1-1

A.1 Circumstances Making the Collection of Information Necessary 1-1

A.2 Purpose and Use of the Information Collection 1-3

A.3 Use of Improved Information Technology and Burden Reduction 1-5

A.4 Efforts to Identify Duplication and Use of Similar Information 1-8

A.5 Impact on Small Businesses or Other Small Entities 1-9

A.6 Consequences of Collecting the Information Less Frequently 1-9

A.7 Special Circumstances Relating to the Guidelines of

5 CFR 1320.5 1-9

A.8 Comments in Response to the Federal Register Notice and

Efforts to Consult Outside the Agency 1-10

A.9 Explanation of Any Payments or Gift to Respondents 1-17

A.10 Assurances of Privacy Provided to Respondents 1-17

A.11 Justification for Sensitive Questions 1-18

A.12 Estimates of Annualized Burden Hours and Costs 1-19

A.13 Estimates of Other Total Annual Cost Burden to

Respondents and Record Keepers 1-21

A.14 Annualized Cost to the Federal Government 1-21

A.15 Explanation for Program Changes or Adjustments 1-22

A.16 Plans for Tabulation and Publication and Project Time

Schedule 1-22

A.17 Reason(s) Display of OMB Expiration Date is Inappropriate 1-24

A.18 Exception to Certification for Paperwork Reducation

Act Submissions 1-24

B. Collection of Information Employing Statistical

Methods 2-1

B.1 Respondent Universe and Sampling Methods 2-1

B.2 Procedures for the Collection of Information 2-6

B.3 Methods to Maximize Response Rates and Deal with

Nonresponse 2-9

B.4 Test of Procedures or Methods to be Undertaken 2-10

B.5 Individuals Consulted on Statistical Aspects and Individuals

Collecting and/or Analyzing Data 2-10

Table of Contents
(continued)

Exhibits Page

A-1 Estimated Hour and Annual Cost Response Burden 1-20

A-2 Total Annualized Cost to the Federal Government 1-21

A-3 Data Collection Timetable 1-23

Tables Page

A-1 60-Day Federal Register Comments and ACL Responses………… 1-12

B-1 Margin of error for the difference between two estimates

and estimates of target characteristics 2-3

B-2 Average frame counts of Caregivers among sampled AAAs 2-4

Table of Contents
(continued)

Appendices

A Pertinent Legislation

B Caregiver Survey Overview and Research Questions Crosswalk

C Instructions for Creating Numbered Client Lists for Sampling

D Federal Register Notice Published by ACL for the Proposed

Information Collection

E Westat Assurance of Data Security Agreement

F Caregiver Pre-Interview Notification Letter & Sample Six-Month Reminder Card

G-P Survey Instruments [see separate zip file/document]:

G NFCSP Caregiver Participant Group Survey: Baseline

H NFCSP Caregiver Participant Group Survey: 6-month follow-up

I NFCSP Caregiver Participant Group Survey: 12-month follow-up

J Caregiver Comparison Group Survey: Baseline

K Caregiver Comparison Group Survey: 6-month follow-up

L Caregiver Comparison Group Survey: 12-month follow-up

M NFCSP Care Recipient Survey: Baseline

N NFCSP Care Recipient Survey: 12-month follow-up

O Comparison Group Care Recipient Survey: Baseline

P Comparison Group Care Recipient Survey: 12-month follow-up

A. Justification

A.1 Circumstances Making the Collection of Information

Necessary

Introduction

This OMB package requests clearance to conduct an outcome evaluation of the National Family Caregiver Support Program (NFCSP) under Title III-E of the Older Americans Act (OAA). The NFCSP outcome evaluation includes the conduct of baseline interviews with two follow-ups after 6 months and 1 year. This evaluation will provide the Administration for Community Living (ACL) with a better understanding of the NFCSP client service outcomes associated with a comprehensive and coordinated caregiver services and support system.

ACL Program Evaluation

Consistent with requirements of the Government Performance Results Modernization Act (GPRMA), ACL integrates its strategic priorities and plans with performance measurement criteria. ACL has three major performance measures: improve program efficiency, improve client outcomes and improve effective targeting to vulnerable elders. However, simply measuring performance at a macro level does not provide the level of detail required to understand emerging trends, systems issues and program innovation at the community level. ACL also seeks a better understanding of key programs, in this case, the Older Americans Act Title III-E National Family Caregiver Support Program. In fact, the OAA requires that ACL evaluate all programs authorized through this act (Title II Section 206(a)) to determine “their effectiveness in achieving stated goals in general, and in relation to their cost, their impact on related programs, their effectiveness in targeting for services under this Act...” Further, the requirements stipulated under 202(f) of the OAA direct ACL (through the Office of Performance and Evaluation (OPE)) to ensure the most relevant data are available to policy makers; that programs demonstrate value to the taxpayer; that programs have a track record of results; and that programs warrant continued or additional resources. Citations for this legislation are found in Appendix A.

Title III-E Older Americans Act Programs: Technical and Contractual History

The Older Americans Act (OAA) National Family Caregiver Support Program (NFCSP) statutory authority is contained in Title III Part E Section 373 of the Older Americans Act (OAA) (42U.S.C. 3032), as amended by the Older Americans Act Amendments of 2006, P.L. 109-365). The NFCSP is an important part of comprehensive home-and community-based services.

When OAA was reauthorized in 2000, legislation created NFCSP as a new program aimed at supporting family caregivers. Older Americans Act Title III, Part E provides grants to States and Territories under approved State Plans for the establishment and operation of the Program. Section 373 authorizes grants to provide a multifaceted system of support services to family caregivers and grandparents or older family members caring for related children. Supportive services include five core services for family caregivers, including:

• Information to caregivers about available services;

• Assistance to caregivers in accessing supportive services;

• Individual counseling, support groups, and caregiver education/training* to assist caregivers in making decisions and solving problems relating to their caregiving roles;

• Respite care* to temporarily relieve caregivers from their caregiving responsibilities; and

• Supplemental services, on a limited basis, to complement the care provided by caregivers.

* The outcome evaluation described in this supporting statement will focus on these services.

The NFCSP was designed to stimulate development of a multifaceted system that spans and integrates the five NFCSP services, other OAA services, and other relevant home and community-based services (HCBS) programs. Ultimately, the program is intended to help family caregivers experience a seamless process for getting connected to information and services that best meet their needs and preferences that will enhance caregiving to the greatest extent possible.

Three Phases of NFCSP Evaluation

This outcome evaluation will represent the third phase of ACL’s current evaluation of the NFCSP. This three-phase evaluation represents the first evaluation of the NFCSP since the program’s inception.

1. The first phase was conducted under a contract (GS23F984OH) awarded in 2008 to develop the approach to the evaluation.

2. The second phase, under the Process Evaluation of Older Americans Act Title III-E National Caregiver Support Program (NFCSP) contract HHSP23320095639WC, began in late September 2012, to finalize the evaluation research methodologies and conduct the process evaluation research, analysis and reporting. The Phase 2 contract and the process evaluation will be completed March 14, 2016 and the final report will be available on the ACL website in April 2016. National Family Caregiver Support Program Process Evaluation OMB control number: 0985-0038; ICR Reference Number:201408-0985-002.

3. The objective of this third phase study is to implement an outcome evaluation of the NFCSP.

The overall goal of this NFCSP evaluation is to contribute to what ACL terms an “integrated” evaluation of NFCSP. Taken together the previous phases (Phases 1 and 2), that included the development of the evaluation approach and the process evaluation, and this phase (phase 3), which includes the outcome evaluation, should result in an overarching evaluation that combines process, outcome, and impact evaluation information.

A.2 Purpose and Use of the Information Collection

The purpose of this third phase is the implementation of an outcome evaluation of the Older Americans Act (OAA) Title III-E National Family Caregiver Support Program.

This outcome evaluation will provide a better understanding of the NFCSP client service outcomes associated with a comprehensive and coordinated caregiver services and support system, including (but not limited to) stress reduction and maintenance of the care recipient in the community rather than placement in a nursing home. This outcome evaluation will also serve to determine the extent to which NFCSP caregivers, as compared with non-NFCSP caregivers, can continue to provide home-based caregiving as needed, provide information on service quality and stress/stress reduction, and benefit from the cost-effective services and maintain their level of health and well-being. Finally, this outcome evaluation will identify any NFCSP resources, organizational characteristics, and implementation practices that appear to contribute to positive outcomes for caregivers receiving the key NFCSP services of respite and/or caregiver training/education and for their care recipients.

The study aims to assess a series of target outcomes by comparing NFCSP participants to non-participants. It will ascertain the impact of services on the ability of caregivers to continue to provide caregiving as needed, and include, for example, an examination of the relationship between the self-reported measures of physical and mental well-being of program participants and the amount of caregiver services received.

Additionally, this evaluation will promote an understanding of the relationship between NFCSP client service outcomes and the organizations providing NFCSP services by examining the Phase 2 Process Evaluation data regarding:

• Aging Network capacity, caregiver program maturity, structural characteristics (type of entity, budget, number served, urban/rural)

• Quality assurance and improvement activities

• Partnerships/relationships between NFCSP and other home- and community-based service (HCBS) programs

• Use of effective/innovative models, activities, and procedures (e.g., caregiver assessments, self-directions, prioritization and cost containment policies, care management, whether or not direct services are provided to caregivers).

While this outcome evaluation will focus on the NFCSP’s impact on caregivers, the evaluation will also include a short set of questions for the care recipients of the study’s sample of caregivers. Such a survey would confirm the extent to which care recipients are aware that their caregivers are receiving caregiver services and the care recipients’ views on the benefits of these services.

The study consists of the following components:

• Select a sample of local Area Agencies on Aging (AAA) from which to develop a sample of approximately 1,250 NFCSP caregivers and a matched comparison group of approximately 1,250 caregivers not receiving NFCSP services.

• A comparison study focusing on NFCSP participants and non-participants, which will include data collection of the number of all care recipients and amount of time spent on caregiving, demographics of the caregiver and primary care recipient, an inventory of caregiving tasks, additional caregiving helpers, knowledge and use of formal services, perceptions of the extent to which the services were helpful, identification of the most useful type of services, perceptions of delayed institutionalization, and the personal impact of caregiving on employment, health, family relationships. Data will be collected from members of the participant and comparison groups via telephone interviews, in a manner that takes into account their special characteristics and encourages the highest response rates. Please note: Appendix B presents an overview of the caregiver survey and a crosswalk of the research questions to the survey instrument modules.

The study will also include a short survey of caregivers’ care recipients to ascertain the awareness and perceived impact of services on their caregivers.

This evaluation is being implemented to help ACL answer the following questions:

1. In conjunction with information from the process evaluation (conducted in Phase 2), what types of organizational structures and/or approaches for NFCSP services are associated with the best participant-level outcomes;

2. Are services reaching the groups targeted by the OAA, including caregivers serving older adults with greatest social or economic need;

3. To what extent do NFCSP services participants also receive other home- and community-based long-term support and services and what is the relationship among these services (e.g., do clients receive other services as result of initiating NFCSP services, do clients of other services become aware of and initiate NFCSP services based on receipt of the other HCBS, etc.); and

4. To what extent do NFCSP services program participants realize outcomes such as those associated with a comprehensive and coordinated caregiver support system, compared to caregivers not receiving NFCSP services.

In summary, this evaluation is essential for ACL to gain insight into a core OAA program.

A.3 Use of Improved Information Technology & Burden Reduction

Use of National Survey of Older Americans Act Participants (NSOAAP) to Generate Sample of Potential Respondents

To reduce the burden on the sampled agencies and to improve the efficiency of the evaluation, the contractor (Westat) is proposing to utilize the same stage-one sample of Area Agencies on Aging (AAAs) selected for the 11th ACL National Survey of Older Americans Act Participants (NSOAAP). The NSOAAP is also referred to as the National Survey of OAA Title III Service Recipients, OMB Control Number 0985-0023. Additionally, the contractor will also conduct the stage-two respondent sampling for NFSCP caregivers, and comparison non-NFSCP caregivers and care recipients in tandem with the 11th NSOAAP. AAAs will only have to provide client lists for sampling potential respondents at one point in time, and the client lists will be used to select separate samples for both the 11th NSOAAP and for the NFCSP outcome evaluation.

To reduce the burden for the Area Agencies on Aging (AAAs), the contractor (Westat) has developed procedures for client sampling that utilize the same client tracking management information systems that are used by States and AAAs to create the required State Program Reports for ACL. Since the implementation of the fourth NSOAAP in 2008, the contractor (Westat) has worked cooperatively with vendors of commercial off-the-shelf client tracking software programs most commonly used by the State and Area Agencies on Aging to develop step-by-step instructions for the AAAs to use to generate client lists by service to use for a sample frame. It is estimated that over 95% of the AAAs now have this technological capability and are able to follow the instructions to produce their client lists by service. Appendix C contains an example of instructions created for agencies which use a commercial client tracking software system known as “PeerPlace.”

In specific states that have their own proprietary client tracking software, the contractor (Westat) has worked directly with an Information Technology Specialist at the State-level to generate electronic client lists for all of the AAAs selected for the national survey. This further reduces the burden for AAAs in states that have their own proprietary software.

Use of Survey Web Site

As mentioned above, during the 11^th NSOAAP, AAAs will submit client lists that will be used to select separate participant samples for both the 11^th NSOAAP and for the NFCSP Outcome Evaluation. To protect the privacy of any personally identifiable information, the AAAs will upload client lists to be used for sampling for both studies to a dedicated website application https://aoasurvey.org/default.asp which has been developed to support and assist with respondent sampling and data collection. For the NSOAAP, the contractor (Westat) designed and utilized a secure website which the AAAs used to upload their lists of clients for sampling. That website is being updated and further refined for the 11th NSOAAP, as well as incorporating additional functionality for the NFCSP outcome evaluation.

The website is divided into two major sections: the public and the restricted-access sections. The public section is accessible to the general public, without restrictions. It includes background information, frequently asked questions, and links to results of previous ACL/AoA National Surveys. The purpose of the public section is to provide State and Area Agencies on Aging, professionals in the field of aging, and service recipients and their families with information about the data collection effort and uses of the data.

The restricted-access section of the website houses an electronic records receipt system. Area Agencies on Aging have the option of submitting private personally identifiable client data to Westat via electronic files using the project web site. The website was written in Active Server Pages (ASP), HTML, and JavaScript and uses the industry-standard SSL (Secure Socket Layer) encryption for secure File Transport Protocol (FTP) data submissions. Agencies choosing this option will receive usernames and passwords that enable their staff to sign on to the file upload utility on the web site. This system supports files in a large variety of file formats. Each agency's data file will be processed according to its structure and content. Westat programming staff will manually map and convert the data items in each agency’s file to create standardized records for further processing. As each file is received, this system will log the source agency, date received, and file type.

Only agencies that have been selected to participate in the survey will have access to this area. Unique user IDs and passwords will be assigned to each AAA at the time they are selected into the sample. The ID and password will be provided with other survey materials to the AAA.

Use of Computer Assisted Telephone Interviewing (CATI)

The contractor (Westat) will use computer-assisted telephone interviewing (CATI) technology to conduct the surveys of NFCSP caregivers, care recipients, and the non-NFCSP caregiver comparison group and record the responses. Westat’s CATI capability includes customized software systems for scheduling, interviewing, and data handling and utilizes high-speed data networks and centralized voice and data monitoring. A single database is used to monitor and direct the interviewers. The Scheduler, a computerized survey control system, makes interviewer assignments, records the disposition of sample cases, and helps survey managers monitor performance.

Westat will attempt to contact each person in the sample, making multiple calls at different times and days when necessary. To reduce the burden for the respondents, Westat will schedule appointments for calls at times that are convenient for them. For Spanish-speaking respondents, Westat uses specially trained bilingual interviewers to conduct the interviews in Spanish. If other special arrangements are necessary (e.g., interpreter, interview needed to be conducted over several sessions), the respondent can be further accommodated.

Westat will take the ACL-approved finalized version of the survey instruments and program them into its CATI system. This involves:

• Inserting specifications into the English version of the questionnaire;

• Preparing the specifications for the CATI programmer, including an interpreter version;

• Translating the questionnaire from the specifications into Spanish; and

• Programming and testing both versions of the questionnaire into CATI.

Details of how skips will work in the questionnaire are included in the design document, as are the needed question variations. For example, some questions may need to be asked differently, depending on the answers to previous questions. In particular, if a respondent told us they live with others, the next question we would ask would be, “Do you live with your spouse?” However, if the respondent told us they lived alone, the follow-up questions will not be asked, and CATI will automatically skip to the next question.

The use of the CATI system in combination with Westat’s highly structured telephone interviewer training and procedures ensures that interviewers conduct the surveys in a professional, controlled, and consistent manner.

A.4 Efforts to Identify Duplication & Use of Similar Information

Every effort is being made to avoid duplication and minimize respondent burden.

The overall objective of this NFCSP Caregiver Outcome Evaluation is to analyze the client-level outcomes of the NFCSP such as a reduction in the emotional, physical and financial toll of caregiving, reduced caregiver stress, and a continued willingness to care for loved ones at home for as long as possible. The research will include an analysis of the relationship between program structure and costs (both collected through the Phase 2 evaluation), caregiver services received, and participant-level outcomes. The phase 3 research will include original data collection from NFCSP participants and a comparison group of caregivers.

This is the first impact evaluation of the National Family Caregiver Support Program. The evaluation employs a mixed method approach combining survey data with qualitative data (approved under a previous submission) designed to identify organizational structures of AAAs and providers that deliver caregiver services, as well as the proposed longitudinal survey of NFCSP caregivers and a control group. The evaluation will address the extent to which NFCSP caregivers use services, seek services from non OAA sources, and the impact of the services on their well-being and ability to provide care. The control group of caregivers will provide similar information about services sought and received, as well as their caregiving burden and well- being.

The unique characteristics of the evaluation design are that Westat will survey the caregivers at three points in time (at baseline and at two six-month data collection points) and that the survey will yield outcome data, such as the effect of the program on caregiver physical and emotional well-being as well as the extent to which the services have helped them over time. Westat will also model the effect of the program on caregivers’ ability to continue providing care and for any increase in the time that care recipients remain in the community.

The National Survey of Older Americans Act Program Participants (NSOAAP) is another survey that ACL conducts that includes NFCSP caregivers. However, the purposes of this NFCSP evaluation and that of the NSOAAP are distinctive. The goal of the NFCSP evaluation is to determine the extent to which the outcomes of program participants differ from those of non-participants for both caregivers and care recipients and the factors that contribute to the outcomes, such as the type and/or amount of services received with a focus on respite care and services that provide counseling, training, support groups and education. The primary focus of the NSOAAP is to measure the extent to which home delivered meals, congregate meals, homemaker services, transportation, and caregiver services target the appropriate groups, the extent to which service recipients use services, and their self-reported outcomes.

Utilizing the NFCSP caregiver evaluation as a means to compare OAA service recipients to a nationally representative sample of caregivers can help ACL better understand its program participants.

A.5 Impact on Small Businesses or Other Small Entities

No small businesses will be involved in this study.

A.6 Consequences of Collecting the Information Less Frequently

It is important to follow the respondents over three data collection points over a 12-month period (baseline, 6-month follow-up, and 12-month follow up) to determine the extent to which NFCSP services help care recipients remain in the community. The longitudinal design of the outcome evaluation will provide quantitative data to determine the extent to which the services enable care recipients to remain in the community as measured in months and/or years.

Interviewers will inform the respondents that we would like to conduct a telephone interview again in six-month intervals for two subsequent follow-up interviews. We believe that the collection of data at six-month intervals over a 12-month period will provide sufficient information to measure change over time in physical functioning, emotional stress and caregiver burden, and self-reported outcomes. Most importantly, it will also provide an opportunity to collect information on those clients and care recipients who no longer receive services for a variety of reasons, including placement in a nursing home or assisted living facility.

A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

The data collection effort will be conducted according to the guidelines specified in 5 CFR § 1320.6. No special circumstances are known that would cause inconsistency with these guidelines.

A.8 Comments in Response to the Federal Register Notice & Efforts to Consult Outside the Agency

Comments in Response to the Federal Register Notice

A 60-day Federal Register Notice was published in the Federal Register on November 20, 2013, Volume 78, No. 224, pages 69683-69684 (see Appendix D).

ACL received six sets of comments between November 20, 2013 and January 23, 2014. Every submission included more than one comment, for a total of 33 comments/suggestions. Each of the comments/suggestions is summarized, along with the proposed ACL responses or actions are detailed in Table A-1 below.

Sources of Comments to Federal Register Notice:

State Units on Aging: Three sets of comments

Area Agencies on Aging: Two set of comments

Alzheimer’s Association

Key Comments and Proposed Responses:

Five of the six commenters said that the estimated 45 minutes on average was too long for the baseline caregiver survey. The ACL proposed response acknowledges this concern and notes that specific questions have been eliminated and that, at the convenience of the caregiver, the survey may be conducted over multiple telephone contacts, allowing breaks at the discretion of the caregiver, similar to the process used in other surveys. The revised caregiver participant survey and that of the caregiver comparison group will take about 40 and 35 minutes, respectively. These time periods are comparable to the time required for the outcome evaluation survey being used with Title III-C consumers and which was approved under OMB Control Number: 0985-0037.

Other suggestions included eliminating specific questions (adopted), using definitions from other surveys (not adopted), and making the caregiver survey available in readable print (not adopted due to mode bias and complexity of CATI-programmed survey instrument but the telephone interview will allow for the use of interpreter for hearing impaired or other special circumstances). Caregiver directors from Massachusetts and New York and the Alzheimer’s Association all expressed support for the study. The Alzheimer’s Association stated that “The data ACL proposes to collect can be incredibly valuable if it is collected and published consistently.”

The following table summarizes the comments that were received by ACL. ACL reviewed each of the comments and proposed responses for each comment are detailed below. For ease of review, responses are grouped by topic.

Table A-1 60-Day Federal Register Comments and ACL Responses

Topic/Issue	Comment	ACL Response
Definition of caregiver	“The term caregiver is not clearly defined. Surveys seem to imply a caregiver could be someone who calls long distance once a month. In the FCSP a caregiver is someone who provides direct care of the participant on a daily or regular basis.”	ACL agrees. For purposes of the NFCSP evaluation, ACL will reference language from the Older Americans Act (OAA), Section 302(3), which reads: “The term ‘family caregiver’ means an adult family member or another individual, who is an informal provider of in-home and community care to an older individual or to an individual with Alzheimer’s disease or a related disorder with neurological and organic brain dysfunction.” Since no timeframe is specified in the OAA, the survey has been modified to remove the words “at least a couple of hours a month” from the caregiver survey’s description of caregiving.
Definition of caregiver	“Update questions and/or adopt BRFSS definition of ‘caregiver’ and reword survey instruments to align with language used in the CDC’s Behavioral Risk Factor Surveillance System (BRFSS) Caregiver Module.	ACL proposes no change. ACL will use the definition of caregiver specified in the criteria for program eligibility. As this evaluation did not include the kinship care program, those criteria are: Adult family members or other informal caregivers age 18 and older providing care to individuals 60 years of age and older; Adult family members or other informal caregivers age 18 and older providing care to individuals of any age with Alzheimer’s disease and related disorders. http://www.aoa.acl.gov/AoA_Programs/HCLTC/Caregiver/ The definition of caregiver in the ATUS does not specify that the care recipient must be 60 years old or older. “The ATUS defines an eldercare provider as someone who has provided unpaid care or assistance more than one time in the 3-4 months prior to the interview day to a person who needed help because of a condition related to aging.” http://www.bls.gov/tus/tuquestionnaire.pdf The definition of caregiver in the BRFSS caregiver module does not specify that the care is for an older individual: People may provide regular care or assistance to a friend or family member who has a health problem or disability. http://www.vdh.virginia.gov/livewell/data/surveys/brfss/documents/2015%20BRFSS%20Call%20for%20Proposal.pdf
Demographic information asked in survey questionnaire	“Some of the information included in the initial part of the caregiver survey is collected by the AAA, such as demographics, how long they have been providing care, income, and are they providing care to others. Is there a way to get that information from the AAA to shorten the amount of time for the caregiver survey? AAAs ask these questions when doing an assessment and enter the information into the database.”	ACL proposes no change. Because not all AAAs may be able to provide these specific types of information this information will be collected directly from survey respondents. This will increase the likelihood of complete, accurate, and timely information. There is a separate effort at ACL, working with Regional Office staff and State Units on Aging to improve the quality of data submitted by ACL grantees.

Topic/Issue	Comment	ACL Response
Terminology & Categorization	“Change the title of one of the Categories listed as ‘Using the Toilet’ to “Toileting’.”	ACL proposes no change. ACL will maintain use of the more person-centered terminology “Using the Toilet”, which is applicable to all aspects of toileting, including lowering clothes and undergarments, getting on/off the toilet, cleansing, etc. Person-centered language is one of the cornerstones of the ACL mission-“ [ACL] manages programs and initiatives that promote the use of self-directed and person-centered service models and advance the development of health and long-term care services and support systems that are responsive to the needs and preferences of older adults, persons with disabilities, caregivers, and families.” https://www.federalregister.gov/articles/2015/06/02/2015-13351/statement-of-organization-functions-and-delegations-of-authority-administration-for-community-living. The NCHS now also uses the term “using the toilet” so there is no longer a discrepancy-- National Health Interview Survey http://www.cdc.gov/nchs/data/series/sr_10/sr10_229.pdf
Use of Social Security Numbers	“Asking for Social Security Number (SSN) of caregiver and care recipient is problematic since everyone is warned to not give out SSN. To track consumers, use name and birthdate or ask for only last four numbers of SSN.”	ACL agrees. The survey questionnaires, as well as the client contact information collected from the AAAs, have been changed and do not ask for Social Security Number (SSN).
Eliminate duplicative questions	“Streamline the Baseline Caregiver Intervention Survey by eliminating ‘duplicative questions’ such as questions 57 and 58 related to strained relationships.”	ACL agrees. The requested change has been made to the survey instrument.
Eliminate cumbersome open-ended questions	“The Baseline Caregiver Intervention Survey’s open-ended questions like #21a are helpful, but #24c questions on helpfulness of caregiver education and training are cumbersome.”	ACL agrees. The requested change has been made to the survey instrument.
Concern over unknown phone caller	“Caregivers may hesitate to speak to any caller who is not familiar to them or recommended to them by a trusted source. We recommend that AAAs or local providers known to the caregivers conduct outreach before the survey.”	ACL Agrees. ACL is planning to conduct comprehensive outreach efforts which include advance letters for potential respondents, fact sheets, brochures, and FAQs. We will also work with AAAs and LSPs in advance of the surveys to conduct outreach with their caregivers and recipients.

Topic/Issue	Comment	ACL Response
Privacy concerns	“Privacy concerns: The six-month caregiver survey begins with ‘when we last spoke you were receiving – list services.’ This might appear as the interviewer relating specific initial survey responses to the individual caregiver they are calling.”	ACL proposes no change. It is appropriate and helpful to the caregiver for the interviewer to list the services discussed during the initial interview. The list of services is a useful introduction to the first of the screening questions: “Are you receiving services?” All follow-up surveys will be conducted only after the interviewer has confirmed that she /he is talking with the same caregiver who was initially interviewed.
Complex terminology	The tool has many terms that the typical caregiver will not understand; it is not written on the 6th grade level which many recommend.	ACL proposes no change. The surveys were pilot tested and underwent cognitive testing as well. Additional cognitive testing will be done to identify if respondents are likely to need additional definitions or examples to make the question easier to understand and respond appropriately. Also, the surveys will be conducted over the telephone by a team of highly skilled interviewers with appropriate levels of training. The interactive nature of this exchange will allow caregivers to seek clarification when needed.
Font size	“The tool should have a larger font; easier on the eye.”	ACL proposes no change. The surveys will be conducted over the telephone. In certain circumstances (e.g., a hearing or language impairment of the respondent), the survey will be administered through an interpreter. Therefore, font size is irrelevant.
Non-telephone options	“An electronic option might be useful, particularly for working caregivers.”	ACL proposes no change. The primary methodology will be by telephone. To accommodate working adults, the survey can be administered in short modules and the CATI system will “bookmark” where the respondent left off. Additionally, the survey can be administered during the evening and by appointment.
Topic/Issue	Comment	ACL Response
Opting out of survey and follow-up	“In the situation of a death of a care recipient – the caregiver should have the option to continue or not with questions even if they are brief.”	ACL agrees. Caregivers will be informed that they may opt out of the survey process at any time.
Follow-up with caregivers	“The one-year time frame to conclude survey is significant in the life of a caregiver of an older adult. It may not be possible to reach a significant number of caregivers at the 6-month and 12-month point	ACL proposes no change. For this national evaluation to identify the long-term impacts of receipt of NFCSP services across multiple domains, it is important to follow caregivers over an extended period of time (one year) to identify long-term programmatic impacts.
Follow-up with caregivers	“Three contacts with caregivers over a year is excessive in determining if the program is meeting its goal and mission.”	ACL proposes no change. For this national evaluation to identify the long-term impacts of receipt of NFCSP services across multiple domains, it is important to follow caregivers over an extended period of time (one year) to identify long-term programmatic impacts.
Dissemination of results	ACL should consistently collect and publish these data to promote continuous improvement, especially the community needs assessment questions in the SUA and AAA surveys.	ACL agrees. The results of all ACL program evaluations will be disseminated.
Partnerships	ACL should work with public health partners to coordinate and implement strategies to improve service delivery to caregivers.	ACL agrees and supports partnerships at the federal, state, and local levels.
Support	We support the concept of conducting a national evaluation of the NFCSP and are prepared to work with ACL to ensure a successful project.	ACL greatly appreciates this support and welcomes ongoing feedback.
Support	“We understand the need for such a study and look forward to participating.”	ACL greatly appreciates this support and welcomes ongoing feedback.

Topic	Comments	ACL Response
Length of survey	Six comments were received stating that the caregiver survey was too long.	ACL understands the need to shorten the length of the caregiver surveys. The initial surveys underwent both pilot testing and cognitive testing. We are taking these comments into account and have eliminated some questions. Please see the text immediately below this table for details about which questions were removed. Surveys may be completed over several calls to accommodate the needs of caregivers and the schedules of working caregivers. Interviews will be scheduled at a time convenient for each caregiver.

Caregiver Interview: All timeframes

During Westat’s development of the Survey, several caregiver burden and impacts which were duplicative questions were removed. The interview will move more quickly for comparison group caregivers and during follow-up, as some static characteristics will only be asked at baseline. Also, we decided to use validated, scale-response short forms (PROMISE 10-item and Zarit Burden 4-item interview) to best capture health and burden change from baseline.

We DID, however, decide to keep the IADL list at the end of the survey (I11-I24) to better determine care recipients’ abilities to perform some common activities of everyday life and whether Care Recipient needs assistance performing these activities. These will be simple YES/NO responses and require minimal time.

Reference documents: NFCSP_CaregiverPart_Baseline_VarNames_022916.docx

Caregiver Survey Item	Ref DOC Page & item #s	detail	Comments	Suggestion / Response
Participant only. Variable = PEOTHB Something else from <AGENCY>?	Page 8 / item A17g	Have you received any other education, training, counseling or support group services from <AGENCY> in the past 6 months ?	This was asked to know “what else” besides educ., training, counseling, support that they were getting from the agency.”	JC: Team feels as though we can remove / delete this question. This question not essential for analysis.
Participant only. Var = PEOFTB If yes, how often?	Pg 8 / item A17h	How often did you receive these services? Would you say…		This item can be deleted.
Part and comparison: OEOTHB and OEOFTB “any other edu. & training from any other source”	Page 10 / A22g and A22h	Have you received any other education, training, counseling or support group services from any other source in the past 6 months? If yes, how often.	I think we should drop this question.	Team suggests DELETING these 2 items. Similar to items PEOTHB and PEOFTB, this item will be hard for them to answer and does not add value for analysis. It also muddies the response to item OEDHELPB. RR: Participants will be confused about any other organizations that they get counseling or training from—those additional services may or may NOT be funded by OAA Title IIIE.
C3a if they said YES to receiving a voucher or budget. SPENDB.	Page 15 / item C3A	How did you spend the money? Purchase supplies Pay for a service [transportation, meals, etc.] Hire a person to assist with caregiving activities or tasks Purchase Respite Services Other (please describe: __________________) Refused g. Don’t know	CA: for me who is reading it for the first time, this item has a somewhat of a harsh tone. Respondent is being questioned about the money. Consider softening the tone or removing.	Team: We suggest deleting this question. Too personal and harsh of a question.
Participant only. MSTHELPB.	Page 16 / C7	Ask if previous question shows that a “CORE SERVICE” IS NOT MOST IMPORTANT Of the following caregiver services that you receive from <AGENCY>, which service is most helpful for you? Would you say …	Core services are: a.Information b. accessing supportive services c. Individual counseling, support groups, education/traig d.Respite care e.Supplemental services	The team suggests that we delete / remove this question. We’ve since deemed the question not necessary about a non-core service and “core” may be hard to determine for the interviewer.

Efforts to Consult Outside the Agency

The survey instruments for this proposed information collection are based on questions from the following sources:

• National Health and Aging Trends Study (NHATS)/National Survey of Caregiving (NSOC)

• NIH PROMIS Measures

• National Survey of Older Americans Act Participants (survey questions developed under ACL/AoA’s Performance Outcomes Measurement Project)

• Zarit Burden Interview (ZBI)

The contractor (Westat) has also consulted representatives from different State Units on Aging and different companies representing the commercially available client software packages used by AAAs. The consultation was for purposes of developing and testing the instructions and procedures for generating client lists used for sampling for this outcome evaluation in tandem with the NSOAAP. The representatives who have reviewed the instructions and procedures include:

• Susan Deaver, Georgia Division of Aging Services

• Daniel Chalender, Oklahoma Aging Services Division

• Lisa Beauregard, Massachusetts Executive Office of Elder Affairs

• Leigh Hayden, North Carolina Division of Adult and Aging Services

• Jim Burd, Pennsylvania Department of Aging

• Lisa Sherman, Harmony Information Services

• Carrie Frey, PeerPlace Networks

• Alana Hawkins, RTZ GetCare

A.9 Explanation of Any Payment or Gift to Respondents

No payments or gifts will be given to respondents.

A.10 Assurance of Privacy Provided to Respondents

Privacy and anonymity are important parts of the survey design. In response to this concern, the contractor (Westat) will ensure the anonymity to the general public and ACL of all individuals who provide data. A pledge of privacy and anonymity is a major positive incentive for potential respondents to participate in the survey. Its absence would be a significant deterrent and could create complications in implementing the survey. Please see the Privacy Impact Assessment for more information about data security and the protection of individuals’ PII.

Westat will take the following precautions to ensure the privacy and anonymity of all data collected:

• All Westat project staff, including recruitment specialists, telephone interviewers, research analysts, and systems analysts, will be instructed in the privacy requirements of the survey and will be required to sign statements affirming their obligation to maintain privacy;

• Only Westat staff who are authorized to work on the NFCSP Caregiver Outcome Evaluation have access to client contact information, completed survey instruments, and data files.

• Data files that are delivered will contain no personal identifiers for program participants; and

• Analysis and publication of survey findings for the participant survey will be in terms of aggregated statistics only.

Appendix E presents the internal corporate “Assurance of Data Security Agreement” all Westat project staff must sign. This agreement requires the signer to protect any and all information about individual respondents to which they may gain access. Any Westat employee who violates this agreement is subject to dismissal and to possible civil and criminal penalties.

Westat, the contractor administering the survey instrument and collecting the data, has extensive experience in protecting and maintaining the privacy of respondent data collected from surveys. To ensure privacy, Westat has drawn from its experience in designing the data collection procedures incorporated in this program. In addition to the corporate Assurance of Data Security Agreement, Westat has implemented several other procedures to protect privacy of survey participants.

1. Data is saved on secure network folders only accessible to authorized users. No data is ever stored on laptop computers. At the end of the survey, all private data is permanently deleted.

2. For the 11th National Survey of Older Americans Act Participants (from which the sample of NFCSP caregivers will be drawn) and for the NFCSP Outcome Evaluation, AAAs will be instructed to submit private personally identifiable client data to Westat via electronic files using the secure survey web site. This web site is written in Active Server Pages (ASP), HTML, and JavaScript and uses the industry-standard SSL (Secure Socket Layer) encryption for secure File Transport Protocol (FTP) data submissions. Agencies will receive usernames and passwords that enable their staff to sign on to the file upload utility on the web site. The passwords are created by a password generator which creates random passwords that are highly secure due to a combination of lower and upper case letters, numbers and punctuation symbols. The database containing the client survey data is not accessible via the Internet; it resides on a server inside the Westat firewall. Only Westat Data Collection Program staff members have access to the master survey database.

3. For AAAs that may experience problems with the survey website and wish to send client data electronically by email, we instruct the AAAs to password protect the file containing the data. Password protection of client data sent electronically by email is required not only for transmission between the AAA and Westat, but even internally within Westat. Additionally, we provide the AAAs with an email address to a secure dedicated project email box ([email protected]) which cannot be accessed remotely.

4. For the small number of AAAs that are not able to generate client records by service electronically, they can submit client information in a hard copy format (fax, FedEx, U.S. Postal Service). Hard copies of client information are stored in locked filing cabinets within a locked room. At the conclusion of the survey, all hard copies of client data are shredded.

5. A secure fax machine dedicated solely to this survey is used to receive faxes from AAAs that choose to transmit their data by fax. The fax machine is located within a locked project room. AAAs that need to transmit their data by fax are asked to call to Westat staff to alert them to watch for and intercept an incoming fax. If the fax machine is busy, it does not roll over to any other fax machine.

All respondents in this data collection effort are assured of the privacy of their answers. Respondent data are aggregated and estimates are produced and published at the both at the national level and at the geographic regional or demographic sub-group level. No individual-level data are published, nor are they accessible or provided to anyone outside the Westat Data Collection Program staff.

A pre-notification letter mailed to potential respondents contains essential survey information and assurances of privacy that enable the person to make an informed decision regarding his or her voluntary participation in the data collection effort. A sample of the pre-notification informational letter which will be sent to potential survey participants appears in Appendix F.

A.11 Justification for Sensitive Questions

The caregiver and care recipient survey instruments inform respondents that their responses to all questions are voluntary. We assure them that their survey responses will remain private. Respondents can refuse to answer any question, and the interviewer will move on to the next question on the survey instrument. Additionally, respondents are permitted to stop at any point and to continue the interview at a later time.

As noted by HHS’s Office of Minority Health and supported by additional research, persons who identify as lesbian, gay, bisexual, or transgender (LGBT) are more likely to suffer disparities in health and access to human service organizations. Therefore, information about the LGBT population is valuable to ACL, as well as to ACL/AoA’s Aging Network, since providers of Older Americans Act services can benefit from a greater understanding of this commonly underserved population; a population that may face unique health challenges and limited access to health care and social services.

The Institute of Medicine (IOM) in its March 2011 report, The Health of Lesbian, Gay, Bisexual, and Transgender People: Building a Foundation for Better Understanding, suggested the need for collection of sexual orientation data on federally supported surveys. Moreover, Kathleen Sebelius, then Secretary of the Department of Health and Human Services (HHS), announced on June 29, 2011 that HHS would begin in 2013 to collect data about respondents’ sexual orientation as part of the National Health Interview Survey (NHIS).

A standard set of questions on sexual identity was created and tested for the NHIS. This series of sexual identity questions was approved by OMB (OMB 0920-0222) for use in the NHIS fielded by the National Center for Health Statistics (NCHS). This included a primary question (“Which of the following best represents how you think of yourself?”).

This same set of questions was also approved by OMB (OMB 0985-0023) for inclusion in ACL/AoA National Survey of Older Americans Act Participants. ACL is proposing to include the primary question about sexual identity as part of the baseline interviews for the NFCSP Caregivers and for the comparison group caregivers.

A.12 Estimates of Annualized Burden Hours and Costs

To estimate the respondent burden, the contractor’s (Westat) Telephone Research Center tested the survey instruments by playing the roles of interviewer and respondent and timing the administration of each instrument several times. The cost to respondents who participate in the survey will be in terms of their time only. The caregiver participant survey and that of the caregiver comparison group will take about 40 and 35 minutes (.67 hour; .583 hour) respectively. The care recipient survey instruments take about 10 minutes (0.16 hour).

The cost burden for participants was calculated based on estimates that a salary of $20 per hour equates to approximately $40,000 per year. Data from the 10th NSOAAP survey show that almost 40% of those receiving NFCSP services make $40K or more annually. Therefore, $20 per hour was selected as an average burden level. The annual respondent burden for baseline and applicable follow-up surveys will be as follows:

• $40.00 for the NFCSP caregivers

• $35.00 for the comparison group caregivers

• $6.60 for the care recipients of NFCSP caregivers

• $6.60 for the comparison group of care recipients.

Exhibit A-1. Estimated Hour and Annual Cost Response Burden

Respondent/Data Collection Activity		Number of respondents	Responses per respondent	Number of responses	Hours per response	Annual burden hours	Approx. Annual burden (cost)
National Family Caregiver Support Program Clients– Baseline		1250	1	1250	.67	837.5	$16,750
National Family Caregiver Support Program Clients – 6 month follow-up		1250	1	1250	.67	837.5	$16,750
National Family Caregiver Support Program Clients – 12 month follow-up		1250	1	1250	.67	837.5	$16,750
Care Recipients of NFCSP Clients – Baseline		1250	1	1250	.166	207.5	$4,000
Care Recipients of NFCSP Clients 12 month follow-up		1250	1	1250	.166	207.5	$4,000
Comparison group non-NFCSP family caregivers – Baseline		1250	1	1250	.583	728.75	$14,575
Comparison group non-NFCSP family caregivers– 6 month follow-up		1250	1	1250	.583	728.75	$14,575
Comparison group non-NFCSP family caregivers– 12 month follow-up		1250	1	1250	.583	728.75	$14,575
Comparison group of care recipients – Baseline		1250	1	1250	.16	200	$4,000
Comparison group of care recipients – 12 month follow-up		1250	1	1250	.16	200	$4,000
	Subtotals by Respondent Group Over an 12-month Data Collection Period:
Subtotal: NFCSP Caregivers		1250	3	3,750	.67	2512.5	$50,250
Subtotal: Care Recipients of NFCSP Clients		1250	2	2,500	.166	415	$8,300
Subtotal: Comparison group non-NFCSP family caregivers		1250	3	3,750	.583	2186	$43,720
Subtotal: Comparison group of care recipients		1250	2	2,500	.16	400	$8,000
Total		5,000	varies	12,500	1.579	5,514	$110,270

A.13 Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

Total annual cost burden excluding respondent time is zero (see Exhibit A-1).

A.14 Annualized Cost to the Federal Government

The overall cost of this research to the Federal Government is presented in Exhibit A-2.

Exhibit A-2. Total Annualized Cost to the Federal Government [Based on Year 1]

Category	Costs
Personnel (T&M including staff & indirect costs)	$786,476
Telephone (long-distance telephone survey)	$17,847
Other direct	$ 22,116
Total direct charges (per task order)	$ 39,963
Indirect charges	$6,890
Total	$833,329

A.15 Explanation for Program Changes or Adjustments

This is a new collection of information.

A.16 Plans for Tabulation & Publication and Project Time Schedule

The Contractor (Westat) will clean data, impute and create variables as needed, and prepare all data documentation including codebooks. Westat will also prepare to statistically analyze the data gathered during the Phase 2 evaluation in connection with the data collected from clients under this Phase 3 outcome evaluation for possible associations and include appropriate descriptive and inferential statistics. Data files and documentation will be submitted with a summary report of the outcome evaluation implementation. The summary report will include challenges and lessons learned to assist ACL in the improvement of program evaluation activities.

Westat will draft a complete final report that analyzes the client–level data collected under this contract separately, as well as with the process data collected under Phase 2. The report will address all of the research questions and topics included in the original statement of work as well as others developed throughout the course of the contract.

Additionally, Westat will deliver a complete database that includes comprehensive documentation. This database will provide data disaggregated at the client level that has been sanitized so that it does not include Personally Identifiable Information (PII) such as name, street address, or Medicare number. The final data file will either include the linked process data or be easily linkable with those data.

The timetable for the baseline data collection and the two follow-up data collections is shown in Exhibit A-3.

Exhibit A-3. Data Collection Timetable

Phase 3 Cycles	Data Collection Activity	End dates
Finalize CATI Instruments	Produce final data collection tools	3 weeks after OMB clearance
Sampling of Caregiver Participants	Sampling of caregiver participants from client lists submitted during 11th NSOAAP	4 weeks after OMB clearance
Baseline Data Collection: NFCSP Caregivers	Contact AAAs agencies to confirm sample of NFCSP caregiver clients	6 weeks after OMB clearance
Baseline Data Collection	Telephone surveys of client and comparison caregivers and their care recipients	2 months after OMB clearance
6-month Data Collection	Follow-up telephone surveys of client and comparison caregivers and their care recipients	8 months after OMB clearance
Conduct data processing	Data editing and cleaning, coding and key entry, data analysis (can start before all data rounds collected)	10 months after OMB clearance
Develop analytical models	Develop variables and model designs for multivariate analyses according to each of the four research questions; test with preliminary 6-month outcomes.	12 months after OMB clearance
12-month Data Collection	Follow-up telephone surveys of client and comparison caregivers and their care recipients	14 months after OMB clearance
Conduct final data processing	Data editing and cleaning, coding and key entry, data analysis	15 months after OMB clearance
Analyze study data	Conduct descriptive and multivariate analyses according to each of the four research questions	16 months after OMB clearance
Conduct data documentation	Prepare technical documentation and derived variables and tabulations	19 months after OMB clearance
Finalize data	Deliver data and documentation to ACL	20 months after OMB clearance
Prepare final report	Submit final results in a report format	21 months after OMB clearance

A.17 Reason(s) Display of OMB Expiration Date is Inappropriate

ACL is not seeking an exemption from displaying the expiration date of OMB approval.

A.18 Exceptions to Certification for Paperwork Reduction Act Submissions

ACL is not requesting any exceptions from OMB Form 83-I.

Appendix A

Pertinent Legislation

Pertinent Legislation

Legal Authority: The legal authority to collect this information is found in the Older Americans Act:

• Title II Section 206(a) of the OAA which establishes the authority to measure and evaluate the impact of all programs authorized by the OAA

Title II Section. 206.

(a) The Secretary shall measure and evaluate the impact of all programs authorized by this Act, their effectiveness in achieving stated goals in general, and in relation to their cost, their impact on related programs, their effectiveness in targeting for services under this Act unserved older individuals with greatest economic need (including low-income minority individuals and older individuals residing in rural areas) and unserved older individuals with greatest social need (including low-income minority individuals and older individuals residing in rural areas), and their structure and mechanisms for delivery of services, including, where appropriate, comparisons with appropriate control groups composed of persons who have not participated in such programs. Evaluations shall be conducted by persons not immediately involved in the administration of the program or project evaluated.

Source: http://www.aoa.gov/AOA_programs/OAA/oaa_full.asp#_Toc153957641

• Title III Part E Section 373 of the Older Americans Act (OAA) (42U.S.C. 3032), as amended by the Older Americans Act Amendments of 2006, P.L. 109-365) authorizes the National Family Caregiver Support Program.

Source: http://www.aoa.gov/AOA_programs/OAA/oaa_full.asp#_Toc153957712

Additional Pertinent Legislation:

• GPRA Modernization Act of 2010 (GPRAMA): http://www.whitehouse.gov/omb/performance/gprm-act

http://www.gpo.gov/fdsys/pkg/BILLS-111hr2142enr/pdf/BILLS-111hr2142enr.pdf

Appendix B

Caregiver Survey Overview and Research Questions Crosswalk

Overview of the Caregiver Survey for the Administration for Community Living’s

Outcome Evaluation of the National Family Caregiver Support Program

The caregiver survey will be fielded at baseline, 6-month follow-up, and 12-month follow-up.

Research Questions

1. What types of organizational structures and approaches for delivering NFCSP services are associated with optimal participant-level outcomes?

2. To what extent are services targeted to those in the greatest economic and social need?

3. To what extent do NFCSP caregivers receive other home- and community-based support and services? What is the relationship among NFCSP services and other services that caregivers and care recipients received? How do caregivers who receive NFCSP support gain access to these non-NFCSP services?

4. To what extent do NFCSP caregivers’ outcomes differ from those of caregivers who do not receive support and services from the NFCSP?

Analytical Approaches

1. Test of differences in means and distributions of the receipt of caregiver services, organizational factors and caregiver/care recipient characteristics, both cross-sectionally and longitudinally.

2. Measure the difference-in--differences in outcomes between NFCSP-participant caregivers and the comparison-group caregivers over time, while controlling for the caregiver’s propensity to need support services and for other caregiver/care-recipient characteristics.

Questions to Determine Use of NFCSP Services for Respite and Counseling / Education / Support

The items in this section are similar to the items in the Administration on Aging (AoA) National Survey of Older Americans Act Participants. The section starts with a screener question; if the person is no longer the primary caregiver, we attempt to learn why. The non-screener items in this section are asked to understand exactly which respite services and education/training services a caregiver is receiving from NFCSP, from other organizations, or informally. These items address the third research question about the extent of NFCSP caregivers’ use of other home- and community-based support and services. Four other items ask about the “helpfulness” of services provided by NFCSP and other organizations. These items are outcome-measures for the first research question.

Caregiving Tasks, Frequency, and Intensity

This section asks about caregiving activities, as listed in the AoA national survey and the National Study of Caregiving (NSOC) survey. However, to better measure change in outcomes longitudinally, we replaced YES/NO responses with an ordinal scale. For example, the responses to the caregiving activities are: 1=daily, 2 = several times a week, 3= once a week, 4=several times a month, 5=once a month, 6=do not provide this help. These scaled response levels can make it easier to detect a statistically-significant change in intensity of caregivers’ activities across time.

The items in this section will also be used to address the extent to which services are targeted to those in need (RQ.2). In our statistical modeling to answer research questions 1, 3, and 4, we will use recipient-frailty and caregiving-intensity as independent variables and/or stratifiers. More specifically, this rich array of caregiver and care-recipient characteristics will allow us to develop propensity-score models to control for selection bias in caregivers’ service-use, as well as to investigate whether key subgroups’ wellbeing responds strongly (or weakly) to program activities.

Knowledge and Use of Formal Services Available

This section includes questions about caregivers’ receipt of support services from any paid agency, caregiver-need for services (RQ2), and caregivers’ perceptions about whether services help (RQ1). Many items are similar to those that were asked in the previous NFCSP evaluation survey of caregivers – developed by Lewin. The items in this section, as well as those in Section A, will help us gauge (1) the extent of caregivers’ receipt of NFCSP services and of other services and (2) the association between receipt of NFCSP services and receipt of other services (RQ3). As a result, we will be able to estimate the effect of changes in services received on changes in caregiver stress, caregiver physical and mental health (section E) and care-recipient health (section I and recipient survey.)

Caregiving Satisfaction and Other Aspects

Several items in this section originated from NSOC. In contrast, the items about satisfaction and confidence as a caregiver draw on the Lewin caregiver survey. This section includes two items in order to learn about the caregiver’s family beliefs with respect to of caregiving.¹ This information about cultural norms is worth including because these norms may influence the receipt of caregiver support services, caregivers’ satisfaction with them, and their effect on outcomes.² In this section, we include four items from the Zarit Burden interview, which incorporate standardized scoring and allow for comparison of this survey’s scores to those of other populations.³ In addressing RQ.4, levels of caregiver stress will be used as outcomes.

Impact of Caregiving (Health, Social and Financial)

The first nine questions are drawn from the Adult Patient Reported Outcomes Measurement Information System (PROMIS) Global Short Form (SF) v1.1. PROMIS, funded by the National Institutes of Health, is a system of highly reliable, precise self-reported measures of health and well–being. The PROMISE SF uses a 5-reponse scale (from excellent to poor) and provides standardized scoring for mental and physical health. This section also includes questions about caregiver employment – mainly derived from either the Lewin survey or NSOC. The items in this section will address RQ.4 (caregivers’ outcomes) and RQ.1 (optimal participant-level outcomes). The financial questions will assist in addressing RQ.2.

Delayed Institutionalization & Continued Caregiving

This section asks the caregiver whether (a) the services that he/she has received from NFCSP have enabled him/her to provide care to the recipient for a longer amount of time than if the caregiver hadn’t received those services and (b) the services have influenced whether the care recipient has been able to remain home longer than if they had not been provided. Answers to these questions will furnish the outcomes needed to address RQ1 and RQ4.

Caregiver and Household Demographics

This section includes basic demographic questions about the caregiver. They will be most useful in addressing RQ2. However, the race/ethnicity, income, and years of education data collected in this section will be important as covariates for propensity-score modeling and sub-population analysis.

Caregiver Health Status and Healthcare Utilization

This section asks the caregiver about hospitalizations, emergency-department visits, nursing-facility use, and services from a hospital outpatient department or ambulatory surgical center during the past 6 months. Another question asks about his/her volume of physician visits. The third item (from PROMIS) asks the caregiver about his/her overall health (excellent, very good, good, fair, or poor). These items can be used as covariates or as outcomes for addressing RQ1 and RQ4. Caregiver

Report of Recipient’s Demographics, Health, and Function

In this section, the caregiver is asked about the care recipient’s demographics, activities of daily living (ADL) functionality, chronic-disease conditions, and health-care utilization. This section employs the same list of ADL and instrumental ADLs found in the AoA survey and in NSOC’s chronic disease list. This information about the care recipient is necessary for modeling propensity to need services and for analyzing sub-populations (e.g., caregivers caring for recipients with Alzheimer’s disease). (RQ.1, RQ.2, and RQ.4)

CONCLUSION

Westat and ACL designed the ACL longitudinal caregiver outcome survey to provide the information needed to estimate the effect of NFCSP caregiver-support services on caregiver outcomes and on the care-recipient’s ability to remain in the community. Respondents to the survey will be an intervention group – caregivers who receive respite services and educational support from the NFCSP – and a comparison group – caregivers not receiving those services from NFCSP. With results from the survey items, the NFCSP evaluation’s statistical analysis of caregiver outcomes will be able to control for factors that affect the caregiver’s propensity to need services (e.g., lack of informal support, CR health) and other caregiver-characteristics. The survey includes standardized items from the Zarit Caregiver Burden instrument and from the NIH PROMISE Global Health instrument, which permit us to compare our caregiver sample’s perceived burden and health to national benchmarks. Westat, with ACL’s guidance, chose questions and responses to extract the most benefit from the longitudinal survey-design in detecting change in caregiver-outcomes.

Appendix C

Instructions for

Creating Numbered Client Lists for Sampling

[Using PeerPlace Software]

Please note: The following page is an example of instructions on how to create client lists for sampling developed for one of the commonly used client tracking management information systems.

Instructions for

ACL/AoA National Survey of Older Americans Act Participants

Survey Summary: Westat is ACL/AoA’s contractor for National Survey of Older Americans Act Participants (NSOAAP). This is an annual telephone survey of about 6000 clients who receive OAA services from 312 AAAs that were randomly selected. The AAAs selected for the NSOAAP are asked to electronically submit client lists by service (as Excel files generated by their client tracking system) from which the select a random sample of about 80 clients per AAA to receive the actual survey.

In order to assist our clients in getting their data easily, PeerPlace has created a View Builder View for you to pull your data. The instructions are as follows:

1. Go to the PeerPlace program you want to pull the data for.

2. Run the “Served Clients with ID” View Builder View for the designated timeframe.

3. Press the “Export to Excel” button to transfer the data into an Excel spreadsheet.

4. Save the complete file to your desktop so you have the data set. We recommend saving the file with the Program Name you ran the View Builder View in.

   1. If doing the one-step approach, you have the information you need.

   2. If doing the two-step approach, you can now do a “Save As” and remove the client name and any other data fields not needed by AoA.

Note: Repeat this for each program as Westat requests a separate file for each program.

5. Upload the file(s) to Westat’s secure National Survey website.

Note: If doing the two-step approach, you will have the original data files saved on your desktop based on step 4 above and you can then provide the client detail based on the random ID’s they selected.

“Served Client with ID” View Builder View

C-1

Appendix D

Federal Register Notice

Published by the

Administration for Community Living

For the Proposed Information Collection

D-1

D-2

Appendix E

Westat Assurance of Data Security Agreement

S

Westat, Inc.

Employee or Contractor’s Assurance of Data Protection of Survey Data

Statement of Policy

Westat is firmly committed to the principle that the individual data obtained through Westat surveys must be protected. This principle holds whether or not any specific guarantee of protection was given at time of interview (or self-response), or whether or not there are specific contractual obligations to the client. When guarantees have been given or contractual obligations regarding data protection have been entered into, they may impose additional requirements that are to be adhered to strictly.

Procedures for Maintaining Data Protection

1. All Westat employees and field workers shall sign this assurance of protection. This assurance may be superseded by another assurance for a particular project.

2. Field workers shall keep completely private the names of respondents, all information or opinions collected in the course of interviews, and any information about respondents learned incidentally during field work. Field workers shall exercise reasonable caution to prevent access by others to survey data in their possession.

3. Unless specifically instructed otherwise for a particular project, an employee or field worker, upon encountering a respondent or information pertaining to a respondent that s/he knows personally, shall immediately terminate the activity and contact her/his supervisor for instructions.

4. Survey data containing personal identifiers in Westat offices shall be kept in a locked container or a locked room when not being used each working day in routine survey activities. Reasonable caution shall be exercised in limiting access to survey data to only those persons who are working on the specific project and who have been instructed in the applicable data protection requirements for that project.

Where survey data have been determined to be particularly sensitive by the Corporate Officer in charge of the project or the President of Westat, such survey data shall be kept in locked containers or in a locked room except when actually being used and attended by a staff member who has signed this pledge.

5. Ordinarily, serial numbers shall be assigned to respondents prior to creating a machine-processable record and identifiers such as name, address, and Social Security number shall not, ordinarily, be a part of the machine record. When identifiers are part of the machine data record, Westat’s Manager of Data Processing shall be responsible for determining adequate protection measures in consultation with the project director. When a separate file is set up containing identifiers or linkage information which could be used to identify data records, this separate file shall be kept locked up when not actually being used each day in routine survey activities.

6. When records with identifiers are to be transmitted to another party, such as for keypunching or key taping, the other party shall be informed of these procedures and shall sign an Assurance of Data Protection form.

7. Each project director shall be responsible for ensuring that all personnel and contractors involved in handling survey data on a project are instructed in these procedures throughout the period of survey performance. When there are specific contractual obligations to the client regarding data protection, the project director shall develop additional procedures to comply with these obligations and shall instruct field staff, clerical staff, consultants, and any other persons who work on the project in these additional procedures. At the end of the period of survey performance, the project director shall arrange for proper storage or disposition of survey data including any particular contractual requirements for storage or disposition. When required to turn over survey data to our clients, we must provide proper safeguards to ensure data protection up to the time of delivery.

8. Project directors shall ensure that survey practices adhere to the provisions of the U.S. Privacy Act of 1974 with regard to surveys of individuals for the Federal Government. Project directors must ensure that procedures are established in each survey to inform each respondent of the authority for the survey, the purpose and use of the survey, the voluntary nature of the survey (where applicable) and the effects on the respondents, if any, of not responding.

PLEDGE

I hereby certify that I have carefully read and will cooperate fully with the above procedures. I will keep completely private all information arising from surveys concerning individual respondents to which I gain access. I will not discuss, disclose, disseminate, or provide access to survey data and identifiers except as authorized by Westat. In addition, I will comply with any additional procedures established by Westat for a particular contract. I will devote my best efforts to ensure that there is compliance with the required procedures by personnel whom I supervise. I understand that violation of this pledge is sufficient grounds for disciplinary action, including dismissal. I also understand that violation of the privacy rights of individuals through such unauthorized discussion, disclosure, dissemination, or access may make me subject to criminal or civil penalties. I give my personal pledge that I shall abide by this assurance of data protection.

________________________________________

Signature

_________________________________________

Printed Name

_________________________________________

Date

Appendix F

Caregiver Pre-Interview Notification Letter

and

Sample Six-Month Reminder Card

Sample Letter for NFCSP Caregivers Selected to be Interviewed

Tri-County Area Agency on Aging

123 Anywhere St.

Anywhere, XX 12345

<DATE>

Dear Mr./Mrs. ______________________________ ,

We are writing to ask for your help. <<INSERT AAA NAME>> is taking part in a study for the Administration for Community Living (ACL), part of the U.S. Department of Health and Human Services. ACL wants to know more about caregivers around the country and the support they get.

You have been chosen to be part of this national study. An interviewer from Westat, a research firm working for ACL, will call you in the near future to ask you some questions about the caregiver respite and/or counseling and training services that you receive from <<INSERT AAA NAME & PROVIDER NAME>>. Your answers will be kept private. Your name won’t be given to anyone else.

Your answers will be combined with answers from other clients and will be part of a report to ACL about how well these services are meeting the needs of older Americans. ACL and Congress will use this information to help them decide how well the programs are working and what improvements to make. Your answers will be very important in helping ACL and Congress decide what to do.

I hope you will be part of this important study. It’s your choice. Whatever you decide, it won’t affect the services you get now or in the future. If you do not want to take part in this study, please call Westat toll-free at 1-888-204-0046 by <<INSERT DATE>>.

If you have any questions about this study, please call Westat at 1-888-204-0046 or the Administration on Aging toll-free at 1-888-204-0271. If you have any questions about the services you get, please call our office at <<INSERT AAA PHONE NUMBER>>.

Thank you for your help and support.

Sincerely,

______________________________,

Executive Director

An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number. The OMB control number for this information collection is xxxx-xxxx. Public reporting burden for this information collection is estimated to average 40 minutes per response. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to the Administration for Community Living, Washington, DC 20201 Attn: Alice-Lynn Ryssman.

Sample Six-Month Reminder Card for Follow-Up Interviews

<DATE>

Dear <NAME OF RESPONDENT>:

Thank you very much for participating in a telephone interview about the National Family Caregiver Support Program. We would like to call you back in approximately ___ (weeks) [6 months from previous interview] for a follow-up interview. At that time, an interviewer will call to ask if there are any changes in the caregiver services that you receive or any changes to your health or that of the person you provide care for.

We urge you to continue your participation in this important study about the caregiver services that you receive from the <NAME OF AGENCY/PROVIDER>. Your participation will help us determine how to best meet the needs of caregivers caring for elderly family members.

If your telephone number has changed, please complete the enclosed postcard and return it to Westat.

If you have any questions about the study, please call <NAME OF CONTACT PERSON> at <TOLL-FREE NUMBER>. She will be glad to answer any questions.

Again, thank you very much for your participation in this important study.

Yours truly,

NAME OF PARTICIPANT___________________________________________.

My phone number has changed. Please call me at ________________________.

[Please insert new phone number]

1 Rozario and DeRienzis. Familism beliefs and psychological distress among African American women caregivers. The Gerontologist. 2008;48(6):772-780.

2 Aranda MP and Knight BG. The influence of ethnicity and culture on the caregiver stress and coping process: a sociocultural review and analysis. Gerontologist. 1997; 37(3):342-54.

3 Bedard, M., Molloy, D. W., Squire, L., Dubois, S., Lever, J. A., & O’Donnell, M. The Zarit Burden Interview: A new short version and screening version. The Gerontologist. 2001;41, 652-657.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Modified	0000-00-00
File Created	2021-01-22