OMB Supporting Statement A May2_final (1)

National Family Caregiver Support Program (NFCSP) Process Evaluation

Supporting Statement

Table of Contents

Part A 3

A.1. Circumstances Making the Collection of Information necessary 3

A.1.1. Authorizing Legislation 3

A.1.2. Background on the National Family Caregivers Support Program 3

A.1.3. Need to Evaluate the Title III-E Program (NFCSP) 6

A.1.4. Evaluation Objectives 6

A.2. Purpose and Use of the Information Collection 9

A3. Use of Improved Information Technology and Burden Reduction 10

A4. Efforts to Identify Duplication and Use of Similar Information 10

A.5. Impact on Small Businesses or Other Small Entities 12

A.6. Consequences of Collecting the Information Less Frequently 12

A.7. Special Circumstances Relating to Guidelines of 5 CFR 1320.5 12

A.8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency 12

A.9. Explanations of Any Payment or Gift to Respondents 13

A.10. Assurance of Confidentiality Provided to Respondents 13

A.11. Justification for Sensitive Questions 14

A.12. Estimates of Annualized Burden Hours and Costs 15

A.13 Estimates of Other Total Annual Cost Burden to Respondents and Record Keeper 15

A.14 Annualized Cost to the Federal Government 15

A.15 Explanation for Program Changes or Adjustments 15

A.16 Plans for Tabulation and Publication and Project Time Schedule 16

A.16.1 Tabulation and Statistical Analysis 16

A.16.2 Publication and Schedule for Project 20

A.17 Reason(s) Display of OMB Expiration Date is Inappropriate 20

A.18 Exceptions to Certification for Paperwork Reduction Act Submissions 20

Part B 21

B.1. Respondent Universe and Sampling Methods 21

B.1.1. Universe of Potential Respondents 22

B.1.2. Stratified Sampling 22

B.1.3. Sample Development and Selection of AAAs and LSPs 23

B.1.4. Recruitment Strategy 24

B.2. Procedures for the Collection of Information 25

B.2.2. Estimation Procedures 25

B.2.4. Data Collection Methods 25

B.3. Methods to Maximize Response Rates and Deal with Nonresponse 27

B.3.1. Process Surveys 27

B4. Tests of procedures or methods to be undertaken 27

B5. Individuals Consulted on Statistical Aspect and Individuals Collecting and/or Analyzing Data 28

ATTACHMENTS 30

Attachment 1-1. 60 Day Federal Register Notice 31

Attachment 1-2. National Family Caregiver Support Program (NFCSP) Process Evaluation Comments in Response to the 60-day Federal Register Notice 33

Attachment 2: Technical Advisory Group Members 36

Attachment 3.1. LSP Survey IRB Documentation 37

Attachment 3.2. AAA Survey IRB Documentation 38

Part A

A.1. Circumstances Making the Collection of Information necessary

A.1.1. Authorizing Legislation

The Older Americans Act (OAA) National Family Caregiver Support Program (NFCSP) statutory authority is contained in Title III Part E Section 373 of the Older Americans Act (OAA) (42U.S.C. 3032), as amended by the Older Americans Act Amendments of 2006, P.L. 109-365). The NFCSP is an important part of comprehensive home-and community-based services.

The 2000 reauthorization of the OAA created NFCSP as a new program aimed at supporting family caregivers. Older Americans Act Title III, Part E provides grants to States and Territories under approved State Plans for the establishment and operation of the Program. Section 373 authorizes grants to provide a multifaceted system of support services to family caregivers and grandparents or older family members caring for related children. Supportive services include information to caregivers about available services; assistance to caregivers in gaining access to the services; individual counseling, organization of support groups, and caregiver training to assist the caregivers in the areas of health, nutrition, and financial literacy, and in making decisions and solving problems relating to their caregiving roles; respite care to enable caregivers to be temporarily relieved from their caregiving responsibilities; and supplemental services, on a limited basis, to complement the care provided by caregivers.

This evaluation is targeted to provide the first review of this Program’s impact on family caregivers and the persons for whom they care. The Older Americans Act requires AoA to conduct evaluations of OAA programs. The requirements stipulated under 206(a) of the OAA direct AoA to “…measure and evaluate the impact of all programs authorized by this Act, their effectiveness in achieving stated goals in general, and in relation to their cost, their impact on related programs, their effectiveness in targeting for services under this Act unserved older individuals with greatest economic need (including low-income minority individuals and older individuals residing in rural areas) and unserved older individuals with greatest social need (including low-income minority individuals and older individuals residing in rural areas), and their structure and mechanisms for delivery of services, including, where appropriate, comparisons with appropriate control groups composed of persons who have not participated in such programs. Evaluations shall be conducted by persons not immediately involved in the administration of the program or project evaluated.”¹

A.1.2. Background on the National Family Caregivers Support Program

Family caregivers—including relatives, friends, neighbors, and others who provide unpaid support—perform immensely valuable work, helping older adults with chronic disabilities get the help they need at home, rather than entering a facility. In a 2009 study conducted by the National Alliance for Caregiving, the researchers estimate that there are at least 43.5 million caregivers (age 18+) providing unpaid care to an adult family member or friend who is 50 years or older.² AARP reported that, in 2009, 61.6 million people provided care at some time during that year. The regular and intermittent caregiving is valued at approximately $450 billion³.

Although caregivers have always been an implicit responsibility of the Aging Network under the Older Americans Act (OAA), the OAA Amendments of 2000 made this responsibility explicit through the authorization of Title III-E – the National Family Caregiver Support Program (NFCSP).⁴ Title III-E constituted the first major nationwide program initiative under the OAA since the 1970s. Through this program, the national aging services network under the Older Americans Act—–including the U.S. Administration for Community Living (ACL), State Units on Aging (SUAs), Area Agencies on Aging (AAAs), and local service providers (LSPs)—helps to meet the immediate needs of caregivers and care recipients.⁵ In addition, by assisting caregivers of older adults, the program helps states and communities meet longer-term goals of restructuring long-term care (LTC) service systems so they allow for individuals to age in place at home and in their communities.

The NFCSP acknowledges and encourages the vital role caregivers play in the country’s Home and Community Based Services (HCBS) system. It seeks to support family caregivers in a coordinated and responsive manner. The program calls for all states and tribes, working in partnership with AAAs and LSPs, to offer five core services for family caregivers, including:

1. Information to caregivers about available services;

2. Assistance to caregivers in accessing supportive services;

3. Individual counseling, support groups, and caregiver training to assist caregivers in making decisions and solving problems relating to their roles;

4. Respite care to temporarily relieve caregivers from their responsibilities; and

5. Supplemental services, on a limited basis, to complement the care provided by caregivers.

ACL, in developing the program design for the NFCSP legislation, included these service components based on the preponderance of research evidence that indicated they would best meet the range of caregivers’ needs and provide the necessary flexibility through the supplemental service category. ACL also drew on the experience of several state-funded caregiver support programs created in the 1980s and 1990s to identify the services that might constitute an effective multifaceted system of support services for caregivers. Program implementation was the responsibility of federal, state, and local communities of the national Aging Services Network.

When the NFCSP was enacted, the OAA identified the following two caregiver population groups as eligible for services:

• Family caregivers of individuals 60 years of age or older, including family members as well as non-relatives; and

• Grandparents and other older relatives who are sole caregivers of children 18 years of age and younger, if the child is 1) their grandchild, or 2) a child with intellectual or developmental disabilities. The caregivers must be related to a child by blood or marriage, be over age 60, live with the child, act as the primary caregiver of the child, and have a legal relationship to the child or raise the child informally.

The 2006 OAA reauthorization enacted several changes to expand eligibility:

• Family caregivers of a person with Alzheimer’s disease or a related dementia may be served regardless of the age of the person with dementia.

• Grandparents and other older relative caregivers providing care to children (under age 18 years) may receive services at 55 years of age and older;

• Grandparent or older relative caregivers, providing care for adult children with a disability, who are between 19 and 59 years of age, can now be served under the NFCSP as follows: 

• Caregivers must be age 55 years and older; 

• Priority is given to caregivers providing care for an adult child with severe disabilities; and

• Services provided to these caregivers are not counted against the 10 percent ceiling for grandparents and other caregivers providing care to children under the age of 18.

The statute also requires states to give priority consideration to persons in greatest social and economic need, with particular attention to low-income older individuals and older relatives caring for children or older adults (age 60 and older) with ID/DD. At least ninety percent of the funds appropriated for the NFCSP are to be used to provide support services to caregivers caring for older adults, while up to 10 percent of the funds can be used to serve grandparents and other older relative caregivers of children.

By creating the NFCSP, ACL envisioned that each of the five service components ultimately would become available to caregivers in every state. Congress appropriated $125 million for FY 2001, $141.5 million for FY 2002, and $155.2 million for FY 2003 for the NFCSP. Since then, funding remained same through FY 2012 ($154 million) until a funding decrease to $146 million in FY 2013 and FY2014. Under the OAA, SUAs receive the greatest proportion of these funds, allocated through a congressionally mandated formula based on a proportionate share of the over-70 population, while providing a minimum to each state. States, in turn, allocate funds to the AAAs based on intrastate funding formulas. AAAs provide the NFCSP services to family caregivers or contract with LSPs for their provision. States have the flexibility to determine the funding allocated the five categories of services authorized. States are required to match 25 percent of their allocation with nonfederal dollars. As is the case with other OAA services, the Aging Network is expected to leverage existing resources and coordinate with community agencies and voluntary organizations to best serve family caregivers.

The NFCSP also was designed to stimulate development of a multifaceted system that spans and integrates the five NFCSP services, other OAA services, and other relevant HCBS programs. While the Aging Network is required to use the NFCSP funds to provide at a minimum the five core services, they are strongly advised to leverage resources with other programs to build sustainable systems of support across all of the recommended services rather than using all the funds for a particular service, such as respite support. Ultimately, the program is intended to help family caregivers experience a seamless process for getting connected to information and services that best meet their needs and preferences that will enhance caregiving to the greatest extent possible.

A.1.3. Need to Evaluate the Title III-E Program (NFCSP)

As indicated above in Section A.1.1, the Older Americans Act (OAA) requires AoA to conduct evaluations of OAA programs, as stipulated under 206(a) of the OAA. Particularly relevant to this outcome evaluation is the stipulation that AoA evaluate the effectiveness of “…their structure and mechanisms for delivery of services…”. This process evaluation of the Title III-E program will provide the first documentation on how family caregiver support systems operate at state and local levels. It is important to examine the program processes such as community needs assessment, caregiver screening and assessments, and services wait listing, to assess the extent to which the processes to ensure that NFCSP services are reaching the clients that need them most. Knowing more about the diversity and similarities of the organizations administering caregiver support programs and providing NFCSP services to family caregivers will provide the context for a future NFCSP outcome evaluation.

A.1.4. Evaluation Objectives

The overall evaluation aims to answer the question, “How effective and efficient is the OAA Title III-E Program at addressing the Program’s goal of helping family caregivers experience a seamless process for getting connected to information and services that best meet their needs and preferences that will enhance caregiving to the greatest extent possible?” The process evaluation of the Title III-E Program has three broad objectives which are to:

1. Provide information to support program planning, including an analysis of program operations;

2. Develop information about program efficiency and costs; and,

3. Gauge program effectiveness in assessing community and client needs, targeting and prioritizing, and providing services to family caregivers.

Thus, the process evaluation will analyze the various organizations and operational processes established to meet the NFCSP goals in terms of program efficiency and costs, as well as the effectiveness of targeting, prioritizing, and meeting the needs of family caregivers. A future outcomes evaluation will focus on the impact of the program at the individual (family caregivers), program, and long-term care system levels; the outcomes study will be conducted as a separate data collection submitted to OMB later in the year.

A.1.5 Approach of current study

The evaluation framework includes assessment of processes. In terms of processes, the evaluation will document the extent to which the program is doing what it was funded to do and assess similarities and differences in the structure and focus of the caregiver programs at the state and local levels. The study combines a variety of data sources to understand program impacts from multiple perspectives and on different levels. To accomplish this, we will survey the:

• Universe of SUAs, to understand program implementation and outcomes at the state level

• Universe of AAAs and a subgroup of LSPs to examine implementation and outcomes at the local level.

To guide the evaluation work, ACL developed a program logic model, which allows us to readily link program environmental/contextual factors and program resources with program activities and process outputs, outcomes, and impacts. The logic model is particularly useful when studying diverse programs as it provides a common framework for describing programs along the same dimensions, providing a simple way to compare and contrast different sites’ program goals, activities, and outcomes. The logic model is presented in Exhibit 1.

Exhibit 1: NFCSP Logic Model

A.2. Purpose and Use of the Information Collection

The overall purpose of this evaluation is to understand and document the extent to which and how NFCSP goals are being met and if the investment is producing a high quality, cost-effective program. The proposed methodology aligns with the identified research questions, with an emphasis on understanding the program’s contribution to long-term care system reform and identifying effective program models. It will promote a better understanding of program impacts at multiple levels—i.e., on long-term care (LTC) policy and home- and community-based services (HCBS) systems/programs (state and local levels). The evaluation will also identify program strengths and weaknesses and opportunities for improvement.

Information gained from the evaluation will greatly enhance efforts to improve the quality of Aging Network caregiver programs. Supporting family caregivers becomes even more critical as certain socio-demographic changes unfold—a growing older adult population with long-term care needs, smaller family sizes, increased participation of women in the workforce, and geographic dispersion. Supporting family caregivers is an important part of ACL’s goal of furthering HCBS options, independence, choice and consumer-directed care.

The process evaluation focuses on three broad research questions:

• How does the program meet its goals (legislative, state, local, and provider level)? Do caregivers have easy access to a high quality, multi-faceted system of support and services that meets caregivers’ diverse and changing needs and preferences? What systems need to be in place in order to achieve this?

• Has the program contributed to LTC system efficiency? How is the NFCSP integrated or coordinated with other long-term care programs and what is the effect?

• What implementation methods and approaches to service delivery have been the most successful, for whom and under what circumstances?

These broad research areas can be further subdivided into eight major research domains that address the measurable inputs, outputs, and outcomes represented in the logic model:

1. Program funding and spending

2. Characteristics of program recipients and care recipients

3. Program consistency and variability

4. Aging Network capacity

5. Quality assurance and improvement

6. Relationship of NFCSP with other LTC programs and role of NFCSP in broader LTC reform

7. Caregiver populations served

8. Effective/innovative models

A3. Use of Improved Information Technology and Burden Reduction

The study strives to comply with the E-Government Act of 2002 (Public Law 107-347, 44 U.S.C. Ch 36) by using web-based surveys with programmed skip patterns and consistency and data range checks that minimize data entry error.

For the surveys of SUAs, AAAs and LSPs, each agency/provider director will receive e-mail notification of the survey. In addition to a web-based format, a PDF version of each survey will be available for any respondent who does not have the capacity to complete a web-based survey or who prefers a paper version. The PDF version will be a form that respondents can complete electronically and save, or print in order to complete it with pen and paper if they prefer. Respondents will be able to submit the form electronically (as an email attachment or mailed disk) or by fax or mail. To enhance response rates and minimize respondent burden, we will also offer proxy web survey completion via telephone-assistance for respondents who prefer to schedule a time to complete the survey.

A4. Efforts to Identify Duplication and Use of Similar Information

The ACL sought to avoid duplication of effort in both design and data by trying to identify existing instruments and data sets relevant to the study. It was concluded that no existing data sources can provide data needed to answer the study’s research questions. This study will be the first to collect nationally representative policy and process information from SUAs, AAAs, and LSPs about how the NFCSP operates at the state and local levels. This study will build on the Title III-E national program summary of caregiver characteristics and circumstances, amount and types of assistance provided to caregivers, self-reported strain, and self-reported benefits based on NFCSP recipients’ responses in 2009 to the Fifth National Survey of OAA Program Participants.⁶

Where possible, existing state-level data collected by ACL through the State Program Reports (SPRs) and disseminated through ACL’s AGing Integrated Database (AGID) will provide state-reported information on NFCSP costs and the numbers of clients served. The following SPR data elements will supplement the state-level data collected for this study:

• Section I.E. Summary Characteristics of Caregivers Serving Elderly Individuals (NFCSP – Title III-E)—Gender, age, ethnicity, race, relationship to care recipient

• Section I.F. Summary Characteristics of Grandparents and Other Elderly Caregivers Serving Children (NFCSP – Title III-E)—Gender, age, ethnicity, race, relationship to care recipient (grandparent or other elderly relative), total children receiving care

• Section II.B. Title III-E Utilization, Expenditure, and Program Income Profile for Caregivers Serving Elderly Individuals—Federal expenditures, total expenditures, program income received, number of caregivers served, units of service, and number of providers for each of the five types of caregiver service (1. counseling/support groups/ caregiver training, 2. respite care, 3. supplemental services, 4. access assistance, and 5. information services)

• Section II.C. Title III-E Utilization, Expenditure, and Program Income Profile for Grandparents and Other Elderly Caregivers Serving Children—Federal expenditures, total expenditures, program income received, number caregivers served, units of service, and number of providers for each of the five types of caregiver service (1. counseling/support groups/ caregiver training, 2. respite care, 3. supplemental services, 4. access assistance, and 5. information services)

These data exist primarily for program monitoring purposes and are not sufficient to evaluate and determine the impact of the program. Therefore, the design incorporates the use of secondary data to the extent possible and augments these secondary data with additional client and program data.

In addition to the SPR administrative program information on state-level NFCSP costs and clients available through ACL’s online AGID system (http://www.agid.acl.gov/), this study will use the annual national survey results from the National Survey of OAA Participants for the national perspective on NFCSP services as reported by program participants.

Other key secondary sources of information on caregiving include the following:

National Aging Network Survey: 2013 Results (formerly the National Survey of Area Agencies on Aging): This survey, conducted every two or three years, provides a wide range of information on AAAs; however, the data are only on the aggregate level and not available for individual AAAs.

National Study on Caregiving, National Health and Aging Trends Study (NHATS): The National Study of Caregiving (NSOC) is a part of NHATS and was funded by the Assistant Secretary for Planning and Evaluation (ASPE). Interviewers spoke with helpers (i.e., caregivers) of NHATS participants who received assistance with self-care, mobility, and medical or household activities. The components of the survey were caregiving activities, duration and intensity of care, support services sought and used by caregivers, effect on caregiver participation in activities including work, and demographic questions.

In addition to the information available on AGID, the evaluation team also reviewed early findings on the NFCSP from a 2004 study for background information about the program.⁷ These early findings will provide background context for assessing progress over time.

To gather the additional data needed for a comprehensive evaluation of impacts on caregivers, the outcomes evaluation research (to be conducted under a separate contract and later OMB PRA submission) will survey a large sample of caregivers who have received NFCSP services, as well as a comparison group who have not received services, using a pre/post design.

A.5. Impact on Small Businesses or Other Small Entities

We plan to survey AAAs and local service providers. The AAAs and local service providers will include government-run and private non-profit organizations, ranging in size from small to large in terms of number of participants and size of budget. The information requested from respondents is the minimum required to meet the study objectives, and the burden to organizations has been minimized as much as possible.

A.6. Consequences of Collecting the Information Less Frequently

Collectively, this is a one-time data collection activity. In the current climate of heightened accountability, if these data are not collected, ACL will have to make important decisions regarding the effect of the NFCSP with limited program information. In 2011, the Government Accountability Office released a report on the Older Americans Act about how more should be done to measure the extent of unmet need for services. The report recommended that the Department of Health and Human Services study the effectiveness of cost-sharing and definitions and measurement procedures for need and unmet need.”⁸ They suggest that taking these measures may include suggesting legislative changes to OAA, supporting the need for research to make important decisions on this program. Finally, if these data are not collected ACL will not be responsive to the requirement in Title II, Section 206 of the Older American Act of 1965 that all authorized programs should be evaluated.

A.7. Special Circumstances Relating to Guidelines of 5 CFR 1320.5

This request is consistent with the general information collection guidelines of 5 CFR 1320.5(d)(2). There are no special circumstances.

A.8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

Federal Register Notice

An announcement was published in the Federal Register on November 20, 2013 (65 FR 66760). The Federal Register notice (see Attachment 1-1) soliciting public comments received comments from six individuals and/or organizations. The responses to these comments, and resulting changes to the surveys and methodology, are described in Attachment 1-2.

Outside Consultations

The project team also consulted outside of the agency through a Technical Advisory Group (TAG). The TAG (Attachment 2 lists the names and affiliations of TAG members) was appointed in 2009 to provide expert advice and guidance for the initial study period. Specifically, the TAG provided input on the specification of the study objectives and research questions, the development of the evaluation design, and other issues that arose during the course of the study.

Recent consultations on the evaluation include meetings during the past year with such key stakeholders as AARP, NASUAD, and the National Association of Area Agencies on Aging. (N4A).

A.9. Explanations of Any Payment or Gift to Respondents

There will be no payment made to the respondents of this data collection.

A.10. Assurance of Confidentiality Provided to Respondents

All data collection activities will be performed within the guidelines specified in the Privacy Act. Project staff will, during all phases of data collection and analysis, engage in practices designed to ensure the confidentiality of all respondents. As part of the research team’s letters introducing and following up on the surveys, respondents will be informed that participation in the study is voluntary and will receive information about confidentiality protections. All individual respondents will be contacted through their professional/public addresses, email addresses, and telephone numbers.

The following safeguards will be routinely employed to carry out confidentiality assurances:

• Access to information collected from respondents through the online surveys, by telephone and email, and from those preferring to provide written responses will be limited to those who have direct responsibility for entering and analyzing the results. Access to the file linking sample ID numbers to respondent identification and contact information is limited to a small number of individuals who need to know this information.

• Access to hard copy documents is strictly limited. Documents are stored in locked files and cabinets. Discarded material is shredded.

Computer network resources are secured and protected using file level Access Control Lists network authentication via Active Directory, strong password management and anti-virus /anti-malware scanning. Computer security is maintained via localized firewalls, mandatory password-protected screen savers, FIPS compliant hard-disk encryption and removable media protection. The network perimeter is secured and protected using an Enterprise-level Firewalls, client Virtual Private Networking (VPN), Enterprise-level Anti-Virus/Anti-Malware protection, two-form-factor authentication and Intrusion Detection Systems (IDS). Additionally, security updates and anti-virus definitions are automatically deployed to all nodes to minimize risk due to vulnerabilities.

The process evaluation contractor, The Lewin Group, is located in an office building with physical security systems in place to prevent unauthorized entry and access to both computer systems and hard copies of files. The office has a key pass entry system with a receptionist on duty during working hours. During non-working hours, the office is accessible to key holders only. Additionally the building is patrolled by a security officer throughout the day and remotely monitored by video cameras in the elevators and other building entrances. The Lewin Group maintains two locked shredding bins that are picked up monthly for secure off-site document destruction.

Institutional Review Board

The process evaluation subcontractors, University of Connecticut (UConn) Health Center and Scripps Gerontology (University of Miami, Ohio), prepared and submitted requests for approval to a recognized Institutional Review Board (IRB) for Research Involving Human Subjects. IRB approval is not needed for the SUA survey; this data collection is part of the NFCSP grantee responsibilities of all states.

• The University of Connecticut Health Center received approval for the LSP survey on March 7, 2014. This was provided by the UConn Health Center IRB Office (Attachment 3-1).

• Scripps Gerontology received an exemption to IRB certification on February 20, 2013 for the AAA survey. This was received from Miami University’s Research Compliance Office (Attachment 3-2).

Any updates to the LSP study materials and instruments for the program participants will be submitted to and approved by the UConn IRB. These updates will not need to go through Miami University for the AAA survey as they received exemption from IRB review.

A.11. Justification for Sensitive Questions

The written surveys of SUAs, AAAs, and LSPs do not contain questions of a sensitive nature.

A.12. Estimates of Annualized Burden Hours and Costs

Exhibit 2 provides a summary of the total number of respondents by type, estimated response time per interview, and the total response time for the different respondents. Based on a small number of test interviews, the surveys of SUAs, AAAs, and LSPs are expected to take about 20 minutes to 2 hours each.

Exhibit 2: Estimated Hours and Hourly Cost of Burden

Respondent Type	Number of respondents	Responses per respondent	Average burden per response (hrs.)	Total average annual burden (hrs.)	Average hourly wage9	Total average hourly cost
All SUAs	56	1	1.5	84	35.82	$3,008.88
All AAAs	618	1	2	1,236	35.82	$44,273.52
Stratified sample of LSPs	1,000	1	0.33	330	35.82	$11,820.60
Total	1,674			1,650

A.13 Estimates of Other Total Annual Cost Burden to Respondents and Record Keeper

There are no capital, start-up, or annualized maintenance costs associated with this data collection for respondents.

A.14 Annualized Cost to the Federal Government

The cost to the Federal government for all the tasks associated with the Title III-E Program process evaluation is $919,915 This figure includes designing and administering the surveys of SUAs, AAAs, and local service providers; processing and analyzing the data; and preparing reports summarizing the results. This expense also includes the costs associated with the partial salary of the assigned ACL project officer.

A.15 Explanation for Program Changes or Adjustments

This is a new collection of information. The estimated total amount of burden for this data collection is 1,650 hours.

This is the second phase of an existing project, originally from 2008-2011. During the former project period, The Lewin Group and subcontractors developed the preliminary survey instruments and the proposed evaluation methodology for both a process and outcome evaluations. In 2012, ACL released an RFP to complete the process evaluation component of this project and begin new collection of information. The Lewin Group will be completing the process evaluation, along with subcontractors Miami University, Scripps Gerontology Center and the University of Connecticut, Center on Aging.

A.16 Plans for Tabulation and Publication and Project Time Schedule

This section contains 1) plans for tabulating and analyzing results, and 2) the publication plans and time schedule for completing the project.

A.16.1 Tabulation and Statistical Analysis

This study uses national surveys at the program (SUA, AAA, and local service provider) level. This approach will maximize the capacity of ACL to address research questions regarding NFCSP implementation, outcomes, and impacts. The evaluation team will analyze survey results and compare results with key findings from national and state-level program data from ACL’s online data system (AGID), a national survey of SUAs conducted separately from the present study, and other key literature, as well as other available information on the political, legal, legislative, and regulatory environments and state and local economic climates of the programs included in the study. AAAs and LSPs will be grouped by their organizational/structural characteristics (e.g., caregiver program maturity, size of program in terms of budget and number of caregivers served, whether or not the program provides direct services to family caregivers, and whether or not they use comprehensive assessments specifically for caregivers). Data will be reported in the aggregate only.

SUA Surveys

To collect information about the unique development of each state’s NFCSP organization over time and identify key similarities and differences across states, we will survey all 56 State Units on Aging (SUAs). The SUA survey will provide information on program and policy trends.

Exhibit 3. SUA Organizational Structure Related to NFCSP and Caregiver Support and Services

Characteristic
Type of SUA
Boundaries of AAAs
Full time NFCSP program manager
Development of State NFCSP organization
Caregiver program before NFCSP
Caregiver services before NFCSP
Statewide identify for NFCSP
Statewide policy/standardized eligibility criteria and assessment tools for caregivers as clients
Level of integration with other HCBS for elderly and disabled
Level of coordination between NFCSP and separate state caregiver program
Coordination with other state agencies to implement NFCSP
Single point of entry for HCBS including/excluding NFCSP
Non-OAA programs administered by SUA
Current separate state caregiver program (including minimum age and functional status requirements)
Staff/Volunteer Training
Requirements for AAA training of caregiver staff
State-provided training
Program Administration
Community Needs Assessment
Targeting
Caregiver Intake
Caregiver Screening
Caregiver Assessment
Waiting Lists
Prioritization of Services
Quality Assurance
Monitoring and Evaluation
Self-directed Care/Consumer Direction
State requirements for and caps on NFCSP Services
Funding Sources of NFCSP Services
Basis for Funding Allocation Decisions for NFCSP Services
Funding by Specific Sources
Policy/Regulations on Caregiver Fee-for-Services
Process for meeting federal match by state/PSAs
Long-term and future issues
Status of long-term issues
Significant issues for the future of NFCSP
Needed NFCSP improvements
State best practice/s

AAA Surveys

The national surveys of AAAs will be administered online and will be available in electronic format. Once data collection is complete, the surveys will be processed, coded, and tabulated to facilitate analysis. Initial data examination will also include data cleaning and adjustment for missing data.

The three overarching research questions and 8 research domains (listed in Section A.2), in addition to the logic model (Section A.1), will provide the basic analytic framework for the analysis of the survey data. Information obtained from AAAs will be used to better understand the range of program experience at the AAA level and to form a typology of program models. The surveys will include questions about general aspects of the NFCSP program, including how the program operates (i.e., stand alone, integrated with state caregiver program, etc.) and whether or not a uniform caregiver assessment is used. The surveys will also solicit information about program models, operations and the procedures, role of consumer direction/self-directed support, policies for prioritization and cost containment, the role of care management, and activities undertaken in service delivery and quality assurance. Information obtained from the AAA surveys, summarized below in Table A.16.2, will be used to better understand the range of program experience and to form a typology of program models.

Exhibit 4. AAA Organizational Structure Related to Caregiver Support and Services

Characteristic
Type of AAA
Governance
Description and boundaries of service area
Aging and Disability Resource Center (ADRC) status
Title VI program in PSA
Program History
Caregiver program before NFCSP
Caregiver services before NFCSP
NFCSP implementation impact on AAA
Level of integration with other Home- and Community-Based Services (HCBS) for elderly and disabled
Level of coordination between NFCSP and separate caregiver program
Non-OAA programs administered by SUA
Current separate caregiver program (i.e., minimum age, income eligibility, and functional status requirements) Flexibility of separate caregiver program compared to NFCSP
Staff and Volunteers
Number of staff; number of volunteers/ volunteer hours during past year
Staff dedicated to caregiver services/shared time with other programs Minimum qualifications for three key staff positions
Staff/Volunteer Training
Requirements for training of specific caregiver staff positions
Specific types of caregiver training during past year
Program Administration
Community Needs Assessment
Targeting
Caregiver Intake
Caregiver Screening
Caregiver Assessment
Referrals
Waiting Lists
Prioritization/Service Access
Quality Assurance
Monitoring and Evaluation
Systems Development and Integration Self-directed Care/Consumer Direction
Medicaid Waiver Programs for the Elderly
Types of Provided Services
Caregiver eligibility criteria Provider pool changes over time
Caregiver Education and Training Respite Access Assistance Assistive Technology
Local Service Providers (LSPs) Number of LSPs
Type of payments to NFCSP providers
Proportion of “not-for-profit” NFCSP providers
Funding Sources of NFCSP Services
Operating Budgets for NFCSP Services by Type of Service
NFCSP Funding by Specific Sources
Process for meeting federal match by state/PSAs
Long-term and future issues
Needed NFCSP improvements
Specific Local Organizations Providing Caregiver Services

Frequencies and cross-tabulations of closed-ended responses (e.g., yes/no; number of participants) will be run using a statistical program, SAS. Cross-tabulation analyses will also be performed to examine differences and similarities in responses by program organizational/structural characteristics, including caregiver program maturity, type of governance, size of budget and number of caregivers served, whether or not direct services are provided to caregivers, and whether or not comprehensive assessments are used for family caregivers. Chi-square significance tests will be used to determine the statistical significance of differences between groups.

Local Service Provider (LSP) Surveys

The relatively brief LSPs survey will be administered online to a sample of 1,000 LSPs selected from a sample of 200 AAAs. Only LSPs providing either respite services and/or caregiver training/education to NFCSP caregivers will be considered for inclusion in the survey. The survey will collect general characteristics on the types of caregivers served, services provided, and length of operation. The focus of the survey will be the staffing, volunteers, clients served, and processes in place to provide caregivers with caregiver respite and education/training.

Once data collection is complete, the surveys will be processed, coded, and tabulated to facilitate analysis. Initial data examination will also include data cleaning and adjustment for missing data.

As shown in the example below, analyses of comparisons between different groups of Family Caregiver Support Programs will be prepared to provide new information about the types of services provided to NFCSP caregivers at the local levels.

Exhibit 5: Example Analyses of AAA and LSP Responses

Question: Does your program support NFCSP caregiver clients with care transitions of their loved ones between any of the following settings? (n=___ for AAAs; n=___ for LSPs)

	AAAs		LSPs
	Number	Percent	Number	Percent
Hospital discharge to nursing home or assisted living
Hospital discharge to home
Nursing home or assisted living discharge to the community
Placement of the care recipient into a nursing facility or assisted living
None of the above

A.16.2 Publication and Schedule for Project

The analysis of the data collected from this evaluation will be synthesized in a final report, prepared by The Lewin Group, will be available on ACL website , in accordance with the ACL information quality guidelines.¹⁰ As noted in ACL’s information quality guidelines, as required by the Data Quality Act, analyses of program results, including evaluations, may be developed and used for many purposes.¹¹ Highlights of evaluations and analyses may be on the ACL website and used in testimonies and speeches by ACL officials. To assist in efforts to collaborate with the Aging Network, the target audience for the report will include SUAs, AAAs, LSPs, consumer and advocacy groups, researchers, and state and federal policy makers. The timetable for data collection, analysis and publication is presented in Exhibit 6.

Exhibit 6. Timetable for Data Collection, Analysis, and Publication

Activity/Deliverable	Expected Date of Completion (month after OMB approval)
Data Collection	1-7 months
Data Processing	8-9 months
Data Analysis	9-11 months
Report Preparation	11-13 months

A.17 Reason(s) Display of OMB Expiration Date is Inappropriate

ACL does not seek this exemption. All data collection instruments for the ACL Evaluation of the Title III-E Program will display the OMB approval number and expiration date.

A.18 Exceptions to Certification for Paperwork Reduction Act Submissions

Exception to the certification statement is not requested.

Part B

Part B of the Justification for this information collection activity, the Evaluation of Older Americans Act Title III-E National Family Caregiver Support Program, addresses the five points outlined in Part B of the OMB guidelines.

B.1. Respondent Universe and Sampling Methods

The universe of potential respondents includes all State Units on Aging (SUAs) and Area Agencies on Aging (AAAs)¹². This will ensure that this first process evaluation of the National Family Caregiver Support Program (NFCSP) will be based on a comprehensive set of results for analysis of the similarities and differences across all states and localities. Based on the AAA survey results, a phased, stratified sample of 200 AAAs will be selected, stratified based on organizational size and geography. The additional criterion for selection will be that the AAA provides, at a minimum, respite and/or caregiver training/education services; the AAA may provide other services as well. Each of the selected 200 AAAs will be asked for a list of local providers of respite and/or caregiver training/education, from which five local providers will be randomly selected for the local provider survey. More information is provided in Section B.1.1-B.1.3.

State Unit on Aging (SUA) Survey

The web-based SUA survey will be administered to the universe of SUAs (N=56) providing NFCSP services and is available for review at: http://stageaoa.acl.gov/AoARoot/Program_Results/docs/Surveys/NFCSP_SUA_Survey.pdf.

ACL will send SUA Director a letter providing an overview of the study, followed by an e-mail notification of the survey. The Directors will be encouraged to share the survey solicitation with the most appropriate person(s) in the agency for completion. We will use web-based format and we will also prepare a PDF version of the survey for individuals who do not have the capacity to complete a web-based survey or who prefer a paper version. The research team will employ various methods to maximize survey participation, which could include telephone follow-up, postcard follow-up, email follow-up, etc.

Area Agencies on Aging (AAA) Survey

The AAA survey will be administered to the universe of 618 AAAs providing NFCSP services. All 618 of the AAAs will receive a single in-depth survey. The AAA survey is available for review at: http://stageaoa.acl.gov/AoARoot/Program_Results/docs/Surveys/NFCSP_AAA_Survey.pdf.

This approach will produce the maximum number of responses from AAAs, which will provide greater ability to compare results across different groupings of AAAs.

The comprehensive AAA survey will also be offered in a PDF version for respondents preferring to complete a paper version of the survey. The survey will solicit information about program models, operational procedures, role of consumer direction/self-directed support, policies for prioritization and cost containment, the role of care management, and activities undertaken in service delivery and quality assurance.

LSP Surveys

The LSP survey will be administered as a web-based survey by UCHC Center on Aging, with an optional PDF version. A total of 1,000 LSPs will be sampled. The LSP survey is available for review at: http://stageaoa.acl.gov/AoARoot/Program_Results/docs/Surveys/NFCSP_LSP_Survey.pdf

The rest of Section B.1 covers the sampling approach for the LSP surveys:

• Universe of Potential Respondents

• Stratified Sampling

• Sample Development and Selection of AAAs and LSPs

B.1.1. Universe of Potential Respondents

At the local service provider (LSP)¹³ level, the universe of potential respondents includes LSP organizations delivering the key NFCSP-funded services of respite and/or caregiver training/education. Together, these two types of NFCSP services account for more than half of the program funding¹⁴ and serve almost 200,000 clients annually. In Fiscal Year 2011, there were 11,309 total providers across all types of Title III-E services. Of the total providers nationally, as many as 85 percent provided either respite (n=8,368) or caregiver training/education and (n= 1,247). Thus, the results of the LSP survey will be generalizable to other programs providing these services and represent a substantial majority of all LSPs.

B.1.2. Stratified Sampling

The sample of the LSPs will be based on a stratified sample of all AAAs that both complete the AAA survey and provide either respite and/or caregiver training/education (from AAA survey question #19e and #19b, respectively). From the AAA survey, we will draw a sample of 200 AAAs based on their responses to the provider module on the AAA survey. We will stratify the sample of AAAs by the following:

• Size of AAA budget (small, medium, and large from AAA survey question#100);

• Population density of region served by the AAA (from AAA survey question #4)

  • Rural (Predominantly rural; predominantly remote or frontier)

  • Urban/suburban (Predominantly urban; predominantly suburban, mix of urban and rural; mix of suburban and rural; mix of urban, suburban and rural)

B.1.3. Sample Development and Selection of AAAs and LSPs

The sampling frame for selecting AAAs for the LSP survey will be the AAAs completing the AAA surveys which confirm that the AAAs provides either respite and/or caregiver training/education services. Based on an expected response rate of 80 percent, there will be completed surveys from 494 AAAs. Of these AAAs completing the AAA survey, a conservative estimate is that at least 80 percent of these 494 AAAs will confirm that they provide either respite and/or caregiver training/education services. Thus, we estimate a sampling frame of 396 AAAs. We will use a sample of 200 AAAs to serve as Primary Sampling Units (PSUs) for selecting samples to collect LSP data.

The initial sample of 200 AAAs will be selected as a stratified random sample. As indicated in Section B.1.2, the sample will be stratified to ensure that the variety of AAAs by a measure of size (MOS) defined as total operating budget and population density will be adequately represented.

We will solicit each of the 200 selected AAAs for a list of LSPs with contracts to provide respite and/or caregiver training/education. We will randomly select 5 LSPs from each AAA-provided list for a sample of 1,000 LSPs.

We will follow up with non-responders first by email, and then by telephone contact.

Considerations for the LSP sampling frame include:

1. The LSPs included on the AAAs’ contact lists must provide respite and/or training and education among their core services;

2. Both AAAs that provide direct services and their contracted service providers will be included in the lists to be randomized for survey completion;

3. LSPs will be asked to provide information about clients served, client assessments, interventions, financing, and use of consumer satisfaction assessments;

4. The sample will represent providers of services to the major caregiver population group which is caregivers of adults 60 years and older.

Of Phased Approach to LSP Sampling

The AAA survey will be in the field for 6 months. Drawing a sample of 200 AAAs from the AAA survey respondents (to get the LSP lists) will be done in three phases:

• Phase 1: After the first 2 months, a proportionate sample of AAAs will be drawn from the group of AAA respondents, to get lists of LSPs.

• Phase 2: After the 4^th month, an additional sample of LSPs will be drawn from the group of new AAA respondents.

• Phase 3: After the 6^th month, the remaining sample of LSPs will be drawn from those respondents.

For each phase, we will sample a number of AAAs proportional to the responses received. From Scripps prior experience surveying the AAAs, we expect that this will be around 120 AAAs in Phase 1, 40 AAAs in Phase 2, and 40 AAAs in Phase 3.

B.1.4. Recruitment Strategy

To enhance the likelihood of a high participation rate in the process evaluation surveys at each organizational level, we have developed a contact/recruitment strategy that includes advance mailings from the NFCSP funding agency ACL to the SUA grantees and AAA subgrantees. As well, written promotional materials have been prepared, and we plan to prepare online videos to introduce the survey to SUAs, AAAs, and LSPs. Finally, we will obtain, where possible, endorsements from other relevant organizations such as the National Association of States United for Aging and Disabilities (NASUAD). The outline of the NFCSP process evaluation outreach is provided in the “NFCSP Data Collection Activities” under Study Recruitment Materials at: http://stageaoa.acl.gov/AoARoot/Program_Results/Program_survey.aspx. The documentation details the advance contacts, invitations to participate in the survey, and follow ups to be conducted with the SUA, AAA, and LSP surveys.

B.1.5. Sample Size and Response Rates

The sample sizes for each respondent group are discussed in subsections B.1.2 through B.1.3. Exhibit 1 summarizes the planned census/sample sizes and the expected number of responses. No nonresponse is anticipated at the SUA level. For the AAAs and sample of LSPs, we expect 80 percent cooperation.

Exhibit 1 Summary of Sample Sizes

Respondent Group	Census/ Sample Selected	Number of Responses
State and Territorial Units on Aging (census)	56	5615
Area Agency on Aging (census)	618	494 a
Area Agency on Aging (stratified sample)	200	200
Local Service Provider (random sample)	1000	800a

^aApproximate.

The investigators will use a number of proven methods to maximize participation in the SUA, AAA, and LSP surveys. These include:

• Use of survey instruments with clear instructions for completion;

• Limited number of open-ended questions in the survey instruments;

• Flexibility about the time and administration mode (e.g., online, fax, e-mail, mail) for the SUA, AAA and LSP surveys; and,

• Tracking responses and conducting follow-ups with non-respondents via email, phone, and postcard mailings.

B.2. Procedures for the Collection of Information

Procedures for the collection of information addressed below include:

• Statistical methodology for stratification and sample selection

• Estimation procedure

• Degree of accuracy needed for the purpose described in the justification

• Unusual problems requiring specialized sampling procedures

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden

B.2.1. Statistical Methodology for Stratification and Sample Selection

This is described in subsections B.1.1 through B.1.6.

B.2.2. Estimation Procedures

Analysis weights will be prepared for the LSP survey data file and then merged onto the data files. Most of this work will occur after the data collection period has ended, so that the weights can reflect adjustments for nonresponse. Weights are needed for each of the planned analysis levels. For this process evaluation, weights are needed only for the LSP data gathered through a stratified sample.

We will select the initial sample of AAAs and the subsample of LSPs using software that can directly compute the sampling weights. After data collection is complete and the initial weights are computed, we will conduct an analysis to assess the response patterns. The analysis will consist of univariate and bivariate cross-tabulations. Where appropriate, the analysis will also include multivariate analysis to detect patterns that interactive effects may mask.

B.2.4. Data Collection Methods

SUA Process Survey

The ACL task order officer will elicit the support of the ACL Regional Offices. Before telephone contact, we will send an outreach package by Federal Express to the ACL regional contacts and make a courtesy telephone call to seek their support. After we have established contact with the 10 ACL regions, we will send the 56 SUA outreach packages by FedEx and begin recruitment calling. We will enlist the support of the SUA director and request name and contact information of the designated respondent if it is someone other than the director. If an alternate proxy respondent is identified, we will request that the SUA director give the survey materials to that person, and a survey specialist will attempt to contact that person.

The Outreach Packages Will Include:

1. A cover letter

2. A brochure

3. A survey preparation worksheet

These materials for the SUA Survey Outreach Package are described in detail in subsection B.3.1 and provided in the “NFCSP Process Evaluation Activities” documentation under the heading of Study Recruitment Materials at: http://stageaoa.acl.gov/AoARoot/Program_Results/Program_survey.aspx.

After the respondent is identified, we will contact them and urge them to complete the survey. In some instances, we will complete the survey with the respondent by telephone. Other respondents will complete the paper survey and return it to the contractor.

AAA and LSP Process Surveys

The AAA and LSP process surveys will be web-based. Web surveys offer maximum flexibility to respondents and minimize errors associated with data entry of hard-copy surveys, although a PDF version of each survey will be made available to any organization requesting this option. High response rates are achievable when support is available to help respondents during the field period. For this purpose, AAAs and LSPs not responding to the initial mailing will be contacted to (1) identify appropriate respondent(s), (2) provide technical assistance to complete the survey, and (3) monitor completion. Reminder emails will be sent to encourage timely submission of completed surveys.

Recruitment for the AAA data collection will be similar to that for the SUA process survey; however, it will be more reliant on electronic communication. Recruitment materials will be sent by email that includes (1) a brochure; (2) a list of frequently asked questions (FAQ) about the study’s purpose, the role of the AAAs and LSPs and information on whom to contact with questions; and (3) a letter inviting their participation in this study of the National Family Caregiver Support Program. AAAs selected for the sample of LSPs will be asked to make the first contact with the five LSPs through an email drafted by the contractor.

These materials for both the AAA Survey Outreach Package and the LSP Survey Outreach Package are described in detail in subsection B.3.1 and are provided in the “NFCSP Process Evaluation Activities” documentation under the heading of Study Recruitment Materials at: http://stageaoa.acl.gov/AoARoot/Program_Results/Program_survey.aspx

B.3. Methods to Maximize Response Rates and Deal with Nonresponse

To maximize response rate for this study, we will develop multimode data collection systems that ensure high quality data collection while minimizing burden on respondents. Exhibit 1 summarizes the data collection mode and number of responses for each survey. We will encourage greater participation through contact and recruitment materials that are relevant to each sample group. Here, we present our strategies for maximizing response rates by survey.

B.3.1. Process Surveys

The process survey will examine the strategies, activities and resources of the Title III-E organizations at each of the three levels of the Aging Network: (1) SUAs, (2) AAAs, and (3) LSPs. We will initiate the contacts at the ACL region level and proceed to the SUA level and from there to the AAAs and LSPs. At each level in the Aging Network, we will not only request endorsement for the next level but also ask the respondent to directly communicate that support to the next level in the Aging Network.

A key element in a high recruitment success rate will be the recruitment materials. Dillman (2000) showed that clear, well-written, and persuasive survey materials assist in higher response rates. Recruitment materials include a cover letter, project brochure, and survey worksheet.

Cover letter. The cover letter will explain the purpose of the NFCSP evaluation and will contain endorsements from other agencies or individuals that support the evaluation.

Brochure. The trifold brochure will contain information on the purpose and importance of the study, key components of the study, contact information for the sponsoring and contracting agencies, and responses to frequently asked questions, with a toll-free number and email address in case the recipient has additional questions.

Survey preparation worksheet. The survey preparation worksheet will list the specific types of information that should be gathered, e.g., prior annual report(s) on clients and budgets. By knowing in advance the types of information we seek, the respondents will be able to identify the best respondent for the survey (him- or herself or another staff member). This will provide time for the selected respondent to prepare for the survey and help reduce the burden on the respondents.

B4. Tests of procedures or methods to be undertaken

Instruments to be used in this study were pre-tested on small numbers of appropriate respondents (2 SUA representatives, 9 AAA representatives, and 5 LSP representatives). Time burden estimates were derived and refinements made to question wording and ordering based on this pilot testing.

In developing the study protocol, every attempt was made to replicate or adapt existing instrumentation.

The results of the three pretests are summarized below:

• SUA. The feedback on pre-testing the SUA survey was generally positive. Both pre-testers preferred taking the survey online, but would like to have the paper survey on hand when completing the survey. The pre-testers suggested changes to specific questions to minimize confusion, which were incorporated into the final SUA survey.

• AAA. The respondents’ opinion of the survey was positive overall. A number of changes were recommended. These included reordering questions so that more efficient skip patterns could be used, adding skip patterns or “do not apply” responses, adding definitions, clarifying terms, by adding examples or changing terminology. Most of these changes are straightforward and will assist other AAA directors when they complete the final survey. These changes were made in the survey, along with some recommendations to change specific questions and reduce the length of the survey.

After incorporating the changes to the AAA survey after the first round of pre-testing, Scripps completed a smaller pre-test for the edited survey. The individuals were from the same pool that pre-tested the survey the first time. General comments about the new version were generally positive and respondents reported that the changes “made it easier to complete” and it was “better this time.”

• LSP. The majority of the survey was well-received by pre-testing participants. They felt that the information requested would be available to the person being surveyed or in collaboration with colleagues. There were a few areas where questions appeared unclear or where the order of questions could be improved. This feedback was incorporated into the survey.

B5. Individuals Consulted on Statistical Aspect and Individuals Collecting and/or Analyzing Data

The investigators chosen by ACL to conduct this study include the following individuals:

• Lisa Alecxih, Senior Vice President, The Lewin Group, 703-269-5542, Project Director

• Cindy Gruman, Vice President, The Lewin Group, 703-269-5506, Program Manager

• Ashley Tomisek, Research Consultant, The Lewin Group, 703-269-5632, Research Assistant

• Jane Straker and Suzanne Kunkel, Scripps Gerontology Center

• Julie Robinson and Noreen Shugrue, UConn Health Center

In addition, Greg Link, the program manager for the National Family Caregiver Support Program in the Administration on Aging/Administration for Community Living (ACL), provided extensive expertise and helpful review in the development of the study design and instruments.

The ACL task order officer for this study is:

Alice-Lynn Ryssman

Office of Performance and Evaluation, Center for Disability and Aging Policy

Administration for Community Living, US Department of Health and Human Services

1 Massachusetts Avenue, NW, Washington, DC 20201

Telephone-202.357.3491

Fax-202.357.3549

Email: [email protected]

List of Attachments

Attachment	Type of Documents	Attachment Number
1	Federal Register Notice of Study Responses to Federal Register Notice Comments	1-1 1-2
2	Technical Advisory Group (TAG) members	2
3	IRB Approval UConn – IRB Approval Form Scripps – IRB Exemption Letter	3-1 3-2

ATTACHMENTS

Attachment 1-1. 60 Day Federal Register Notice

Agency Information Collection Activities: Submission for OMB Review; Comment Request; OAA Title III-E Evaluation

Action: Notice.

Summary: The Administration for Community Living (ACL) is announcing an opportunity to comment on the proposed collection of certain information by the agency. Under the Paperwork Reduction Act of 1995 (the PRA), Federal agencies are required to publish notice in the Federal Register concerning each proposed collection of information, including each proposed extension of an existing collection of information, and to allow 60 days for public comment in response to the notice. This notice solicits comments on the information collection requirements relating to Older Americans Act (OAA) Title III-E Evaluation.

DATES: Submit written or electronic comments on the collection of information by January 21, 2014.

ADDRESSES: Submit electronic comments on the collection of information to: [email protected]. Submit written comments on the collection of information to Alice-Lynn Ryssman, U.S. Administration for Community Living, Washington, DC 20201.

FOR FURTHER INFORMATION CONTACT: Alice-Lynn Ryssman, 202-357-3491.

SUPPLEMENTARY INFORMATION: Under the PRA (44 U.S.C. 3501-3520), Federal agencies must obtain approval from the Office of Management and Budget (OMB) for each collection of information they conduct or sponsor. “Collection of information” is defined in 44 U.S.C. 3502 (3) and 5 CFR 1320.3 (c) and includes agency request or requirements that members of the public submit reports, keep records, or provide information to a third party. Section 3506(c)(2)(A) of the PRA (44 U.S.C. 3506 (c)(2)(A)) requires Federal agencies to provide a 60-day notice in the Federal Register concerning each proposed collection of information, including each proposed extension of an existing collection of information, before submitting the collection to OMB for approval. To comply with this requirement, ACL is publishing notice of the proposed collection of information set forth in this document. With respect to the following collection of information, ACL invites comments on: (1) Whether the proposed collection of information is necessary for the proper performance of ACL's functions, including whether the information will have practical utility; (2) the accuracy of ACL's estimate of the burden of the proposed collection of information, including the validity of the methodology and assumptions used; (3) ways to enhance the quality, utility, and clarity of the information to be collected; and (4) ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques when appropriate, and other forms of information technology.

The OAA Title III-E National Family Caregiver Support Program (NFCSP), with statutory authority contained in Title III sections 302, 372, and 373 of the Older Americans Act (OAA) (42 U.S.C. 3032), as amended by the Older Americans Act Amendments of 2006, Pub. L. 109-365), funds a range of comprehensive home- and community-based services supports that assist family and informal caregivers to care for their loved ones at home for as long as possible. ACL is directed under 206(a) of the OAA to conduct evaluations of OAA programs. Thus, this data collection will conduct an evaluation of the NFCSP to fulfill this requirement and understand how well this program is meeting its goals and mission.

The evaluation design is comprised of two primary components:

1. A process study, which examines the strategies, activities, and resources of the program at each level of the Aging Network—State Unit on Aging (SUA), Area Agency on Aging (AAA), and Local Service Provider (LSP); and

2. A client outcome study, which examines the health and social effects of the program on participants compared to non-participants. This study examines the health and social effects on caregivers and also tracks the health outcomes of the care recipients.

The process study will include all 56 SUAs, all of the AAAs (N = 618), a sample of local service providers (N = 1,000), and a sample of program participants (1,250) and non-participants (N = 1,250). The table below provides the information ACL used to estimate the burden of this collection of information:

Respondent Type	Number of Respondents	Responses per respondent	Average burden per response (hrs.)	Total average annual burden (hrs.)
All SUAs	56	1	1.5	84
All AAAs	618	1	2	1236
Stratified sample of LSPs	1,000	1	0.33	330
Family caregivers participating in NFCSP	1,250	3	0.58	2,175
Family caregivers not participating in NFCSP	1,250	3	0.58	2,175
Total	4,174			6,000

The proposed data collection tools may be found on the ACL Web site at http://www.aoa.gov/AoARoot/Program_Results/Program_survey.aspx.

Dated: November 15, 2013.

Kathy Greenlee,

Administrator and Assistant Secretary for Aging.

Attachment 1-2. National Family Caregiver Support Program (NFCSP) Process Evaluation Comments in Response to the 60-day Federal Register Notice

OVERVIEW

Six sets of comments were received between November 20, 2013 and January 23, 2014. Every submission included more than one comment, for a total of 33 comments/suggestions. Eight of those comments related to the process surveys. Each of the comments/suggestions is summarized, along with the proposed ACL response or action below.

COMMENTS SOURCES

Representatives from three State Units on Aging (Massachusetts, New York, and Tennessee) and two representatives from the Pennsylvania and Tennessee Area Agencies on Aging commented on the NFCSP evaluation materials. In addition, the Alzheimer’s Association provided a list of suggestions and comments.

KEY COMMENTS AND PROPOSED RESPONSES TO COMMENTS

Key suggestions on the process evaluation surveys included using different terminology and using administrative data from other data sources. Other comments concerned the caregiver surveys, which will be part of the outcome evaluation. In total, 3 suggestions were implemented to revise one or more of the surveys; 5 were not implemented.

The following table summarizes the process evaluation-related comments received by ACL. ACL reviewed each of the comments and proposed responses for each comment are detailed below. For ease of review, responses are grouped by topic/type of survey.

Comment	ACL Response
Applicable to Multiple Surveys (Implemented=2; Not implemented=2)
Per the DSM-V last May 2013, the term Dementia has been replaced with the term Neurocognitive Disorder. Since surveys are for professionals, the term Dementia should be removed and replaced with the professionally recognized terms like Neurocognitive Disorder or disorders like Alzheimer’s.	ACL proposes no change. ACL will reference language from the Older Americans Act (OAA), Section 302(3): “an individual with Alzheimer’s disease or a related disorder with neurological and organic brain dysfunction.”
Although the survey includes “repairs" as a service which can be provided, "repairs" are not a covered service in FCSP.	ACL proposes no change. ACL will maintain use of the word “repairs” in the surveys. Repairs are an acceptable use of NFCSP Supplemental Funds – though not specifically included or excluded in the OAA. State and local policies may place a limitation on this; however, this is not a policy limitation within the Act itself.
In the SUA and AAA surveys, include "caregivers of individuals with dementia" as a special population of caregiver in the "Targeting" sections.	ACL agrees. To be consistent with language in OAA Section 302(3), the changes will reflect the following language: “caregivers of individuals with Alzheimer’s disease or related disorders with neurological and organic brain dysfunction.”
Add "Dementia training" to list of options under State and Volunteer Trainings in SUA and AAA surveys.	ACL agrees. To be consistent with language in OAA Section 302(3), the additional training topic will reflect the following language regarding training on: “Alzheimer’s disease or related disorders with neurological and organic brain dysfunction.”
State Unit on Aging (SUA) Survey (Implemented =NA; Not implemented =1)
The SUA survey may take more than 90 minutes including program and fiscal staff time to review questions, gather recommended lists of information, and determine accurate responses. The recommendation is that ACL use data captured by the SPR.	ACL proposes no change. ACL will use information from the State Program Report (SPR) where appropriate in the evaluation’s data analysis. Given the importance of obtaining current information reflecting NFCSP operations at the time of the NFCSP evaluation, the annual SPR is not a timely or optimal source of the data needed for this evaluation.
Area Agency on Aging (AAA) Survey (Implemented =1; Not implemented =1)
Add "voluntary health and/or advocacy organization" to list of possible referral sources in AAA survey since the Alzheimer's Association 24/7 Help Line and Chapters refer constituents to aging service providers.	ACL agrees.
Additional program and fiscal staff time will be necessary to review the AAA questions, gather the recommended list of information, and determine accurate responses, so ACL should use data captured by the SPR from required reports submitted by AAAs.	ACL proposes no change. The time estimates provided were based on pilot testing. State Program Report (SPR) data submitted to ACL only provides data aggregated at the state level, for the purposes of this evaluation, current information will be collected directly from AAAs.
Local Service Provider (LSP) Survey (Implemented =NA; Not implemented =1)
More than the estimated time of 20 minutes may be needed to complete LSP survey, so use data captured by the SPR to the greatest extent possible.	ACL proposes no change. The time estimates provided were based on pilot testing. State Program Report (SPR) data submitted to ACL only provides data aggregated at the state level, for the purposes of this evaluation, current information will be collected directly from LSPs.
General Feedback
ACL should consistently collect and publish these data to promote continuous improvement, especially the community needs assessment questions in the SUA and AAA surveys.	ACL agrees. The results of all ACL program evaluations will be disseminated.
ACL should work with public health partners to coordinate and implement strategies to improve service delivery to caregivers.	ACL agrees and supports partnerships at the federal, state, and local levels.
We support the concept of conducting a national evaluation of the NFCSP and are prepared to work with ACL to ensure a successful project.	ACL greatly appreciates this support and welcomes ongoing feedback.
We understand the need for such a study and look forward to participating.	ACL greatly appreciates this support and welcomes ongoing feedback.

Attachment 2: Technical Advisory Group Members

• Ann Bannes, Vice President, Home & Community Based Services, St. Andrew’s At-Home Services

• Kay Carter, Director, Gateway Area Agency on Aging

• Nellie Garay, Caregiver Specialist, Alabama Area Council of Governments, Bexar Area Agency on Aging

• Kathy Greenlee, Administrator, Administration for Community Living and Assistant Secretary for Aging (At the time, Assistant Secretary Greenlee was the Secretary of Aging for the Kansas Department on Aging )

• Hilari Hauptan, Kinship and Family Caregiver Program Manager, Washington Department of Social & Health Services

• Greg Johnson, Creator of the Care for the Family Caregiver Program, Assistant Director of Community Based Activities, Emblem Health

• Arlene Kershaw, Director, New Hampshire Easter Seals Senior Services

• Joan Klein, Former Director, Family Caregiver Support Program, Philadelphia Corporation for Aging

• John Peterson, Program Manager, Snohomish County Department of Human Services, Long Term Care & Aging

• Gail Schwersenska, Office Director, Bureau of Aging and Disability Resources, Wisconsin Department of Health and Family Services

• Laura Trejo, General Manager, City of Los Angeles Department of Aging

• Sue Vaeth, Administrator, Howard County Office on Aging

• Kay Vanags, Family Caregiver Specialist, Aging Resources of Central Iowa

• Jean Wood, Director, Aging and Adult Services, Minnesota Department of Human Services

• Donna Yee, Chief Executive Officer, Asian Community Center of Sacramento Valley

• Sonnie Yudell, Program Manager, Utah Caregiver Support Program

Attachment 3.1. LSP Survey IRB Documentation

Attachment 3.2. AAA Survey IRB Documentation

1 Older Americans Act Amendments of 2006. (2006). Section 206: Evaluation. Retrieved from: http://www.ACL.gov/ACL_programs/OAA/oaa_full.asp#_Toc153957641

2 NAC. 2009. “A Focused Look at Those Caring for Someone Age 50 or Older: Caregiving in the U.S.”. Found at: http://www.caregiving.org/research/caregiving-research/general-caregiving

3 AARP. 2011. “Valuing the Invaluable: 2011 Update. The Growing Contributions and Costs of Family Caregiving.” Retrieved from: http://assets.aarp.org/rgcenter/ppi/ltc/i51-caregiving.pdf.

4 ACL, The Older Americans Act National Family Caregiver Support Program (Title III-E and Title VI-C): Compassion in Action, 2004. http://www.aoa.gov/aoaroot/Program_Results/docs/Program_Eval/FINAL%20NFCSP%20Report%20July22,%202004.pdf,, Foreword by Josefina G. Carbonell, p. i

5 Ibid, p. i.

6 ACL, Supporting Family Caregivers Through Title III of the OAA, AoA Research Brief Number 5, September 2011. Found at: http://www.aoa.gov/Program_Results/docs/2011/AoA5_SupportFamilyCaregvrs.pdf

7 ACL, The Older Americans Act National Family Caregiver Support Program (Title III-E and Title VI-C): Compassion in Action, 2004. http://www.ACL.gov/ACLroot/Program_Results/docs/Program_Eval/FINAL%20NFCSP%20Report%20July22,%202004.pdf, Foreword by Josefina G. Carbonell, p. i

8 GAO, The Older Americans Act: More Should be Done to Measure the Extent of Unmet Need for Services., 2011. http://www.gao.gov/products/GAO-11-237,

9 Sources: Bureau of Labor Statistics, National Compensation Survey, 2010, May 2011, Bulletin 2753. (http://www.bls.gov/ncs/ncswage2010.htm#Wage_Tables): SUA, AAA, LSP staff: Average hourly wage of state and local government social and community service managers; participants and non-participants: national minimum wage

10 HHS, “Guidelines for Ensuring the Quality of Information Disseminated to the Public – B. Administration for Community Living.”

11 HHSA, “Guidelines for Ensuring the Quality of Information Disseminated to the Public, B. Administration for Community Living,” http://thecre.com/pdf/20021026_hhs-ACL-dqfinal.pdf

12 AAAs plan, coordinate, and advocate for the development of a comprehensive service delivery system to meet the needs of older adults in a specific geographic area. They administer state and federal funds for community-based services. http://www.tjaaa.org/glossary-of-terms.aspx.

13 Area agencies normally contract with local for-profit or nonprofit or public providers (LSPs) to deliver benefits. The contract service providers nationwide, providing care under the act, are the largest single network of long-term care providers in the country. An agency may be allowed to directly provide supportive services, nutrition services, or in-home services if it can prove that it can provide these services more effectively. http://www.longtermcarelink.net/eldercare/area_agencies_on_aging.htm.

14 National and state Title III-E clients and expenditures by NFCSP service type are collected through ACL’s State Program Report (SPR) and reported through ACL’s online Aging Integrated Database (AGID) at http://www.agid.acl.gov/ .

15 A 100% response rate is based on the 100% response rate to the SUA survey administered in early 2014 as part of the Title III-C Elderly Nutrition Services Program.

7