Senior Companion Longitudinal Evaluation OMB Supporting Statement A 3_25_15

Supporting Statement

Senior Corps Longitudinal Evaluation

Introduction

The Corporation for National and Community Service (CNCS) requests approval to conduct a longitudinal evaluation of the Foster Grandparent Program (FGP) and the Senior Companion Program (SCP). The Senior Corps longitudinal evaluation will consist of two studies. One study will focus on the Foster Grandparent and Senior Companion volunteers. The second study will focus on caregivers receiving respite services.

The Volunteer Study will enroll new volunteers and will have three rounds of data collection: a baseline and two follow-ups. This study will also use a quasi-experimental design to measure the effects of national service on volunteers experience, satisfaction, self-reported health, symptoms of depression and loneliness. The study will also examine volunteer retention and satisfaction with the volunteer experience. The planned analyses include descriptive analysis, multiple regression analysis and propensity score matching to compare the Senior Corps volunteers to participants in the University of Michigan’s Health and Retirement Study (HRS).

The Caregiver Study will enroll new caregivers receiving respite services. This study uses a pre/post design with a baseline and one follow-up. The planned analyses include descriptive analysis and regression analysis to measure change in caregivers’ satisfaction, self-reported health and self-assessment of symptoms of depression.

In the research literature, the social benefits of volunteering are well documented. Preliminary findings indicate that compared to populations of demographically-similar non-volunteers, there is an association between volunteerism and lower mortality rates, greater functional ability, and lower rates of depressive symptoms in later life. However, the psychological outcomes such as satisfaction with volunteering, life satisfaction, self-assessment of symptoms of depression and loneliness are not well understood. The proposed data collection will contribute to this emergent research on how volunteering is associated with satisfaction, symptoms of depression and loneliness. Importantly, this research will also provide new information on volunteer satisfaction and retention which will be useful in strengthening CNCS-funded National Service program.

1. Justification

1. Explain the circumstances that make the collection of information necessary. Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.

The goal of both studies is to assess how national service participation is associated with volunteers’ and caregivers’ psychosocial outcomes including satisfaction, self-rated health, and symptoms of depression. Both studies are designed to contribute to CNCS’s ability to report on its progress in increasing the impact of national service on participants as well as strengthening national service programs.

The longitudinal evaluation is the second national evaluation of the Senior Corps Program. The first evaluation was a cross sectional data collection which collected information on characteristics of volunteers, life satisfaction, self-efficacy and loneliness. The earlier study did not gather information on caregivers’ characteristics. The proposed longitudinal evaluation will continue to build the evidence base of the effectiveness of Senior Corps programs and is designed to provide information on how to strengthen the National Service program by directly examining retention and satisfaction with volunteering. Each study focuses on increasing the levels of evidence for Senior Corps program and progress in meeting the goals of CNCS’s 2011-2015 Strategic Plan. The Caregiver Study addresses Goal 1 of the Strategic Plan: Increasing the impact of national service on community needs. This is measured in the output of the number of caregivers receiving respite services; and the outcome as measured by the change in caregivers’ satisfaction with respite services. The Volunteer Study addresses Goal 2 of the Strategic Plan: Increasing the number of adult national service volunteers, with the outcome measured by volunteer satisfaction and retention, self-rated health, symptoms of depression and loneliness.

2. Indicate how, by whom, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection.

The data collection will be useful to CNCS program staff and the Senior Corps grantees by providing evidence-based findings of the effectiveness of National Service. First, the data will be useful to assess the benefits of National Service participation on the volunteers’ and caregivers psychosocial outcomes. Second, the data will be useful to CNCS in identifying: 1) best practices for recruitment and retention of volunteers and caregivers, 2) obstacles to high response rates, and 3) assessments of the psychometric characteristics of the surveys to inform future research with this population.

CNCS has a track record of successfully building on evaluation findings to improve programs and inform the development of further research. In particular, the surveys for the longitudinal evaluation build upon the surveys previously tested and used in the first evaluation of Senior Corps program. The previous evaluation included two studies with new recipients of Senior Companion services. One of the studies was a pre/post pilot data collection to look at the feasibility of conducting multiple rounds of data collection among this population. The findings from this pilot informed the design of the longitudinal data collection. The first evaluation showed that respondents were more likely to participate when the sponsor and/or station contacted them. The earlier study also showed that that telephone surveys were more effective when combined with a paper survey that the respondents received from their project/station. The response rates increased when an additional mode was added and when there was grantee involvement in the process. The pilot showed that seniors are cautious about answering phone calls from strangers, so volunteers and clients are reluctant to participate in phone surveys unless a trusted member of the program vouches for the research. As further described below, based on these earlier findings, the likelihood of success increases by embedding the enrollment in the grantees’ intake and sign-up procedures. This approach also minimizes burden to grantees and participants.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also describe any consideration of using information technology to reduce burden.

Previous data collection experience with the first Senior Corps evaluation and consultation with Senior Corps grantees indicate that a mixed mode data collection is the best approach to achieve high response rates while minimizing burden and costs to the government. A self-report paper survey is the primary mode for administering the surveys; a telephone survey will be used to contact nonrespondents. Informed consent will be obtained during the caregiver intake and during volunteer orientation, both of which occur prior to the start of service. Other paper forms are completed at intake and orientation which makes the proposed method for the data collection consistent with the grantees’ current process.

It is important to administer the baseline survey prior to the start of caregiver respite services and before volunteers begin volunteering in order to isolate the effects of program participation. Any gap between entry into the program and administration of the baseline survey could mean that the respondent has begun to experience participation in national service. The timing of the baseline data collection period is important to documenting baseline stress, health, symptoms of depression and loneliness prior to the start of any services. Administering the survey at intake would meet both the grantees’ goal and desire to start service as soon as possible, and the research objective to collect baseline data before the participants’ exposure to participation in national service.

On January 13 and 14, 2015, CNCS convened a working group panel consisting of FGP and SCP project directors to explore how to recruit new caregivers and new volunteers prior to the start of service, the feasibility of various data collection approaches (e.g. paper, phone, online) as well as which mode should be used for the data collection in order to minimize burden on FGP and SCP projects, stations and respondents. The working group recommended, and prior experience fielding the data collection for the first evaluation confirmed, that a paper survey embedded during the projects’ and/stations’ intake and training processes will minimize burden for participants and is the best method to achieve the goal of collecting baseline data collection prior to the start of service. To achieve high response rates, CNCS proposes a telephone survey for non-respondents – these are the individuals that consented to be in the study but did not return a completed survey following the completion of the intake process or volunteer orientation. CNCS’s approach to administer the survey is further described below. For the telephone surveys, interviewers will enter the data directly into computer-assisted telephone interviewing (CATI) system. . CNCS proposes to use a scannable paper survey form which will be coded electronically.

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above.

The surveys build upon both the performance measures CNCS developed to gauge grantees’ performance and the survey items tested in the first evaluation. The surveys also include items taken from the HRS questionnaire which will be used in forming a matched comparison group. The data collection is not a duplication of the first evaluation or the HRS data. The proposed longitudinal evaluation will measure self-reported volunteer satisfaction, symptoms of depression and loneliness. The first evaluation of volunteers measured basic demographic information, limited measures of self-efficacy, loneliness and life satisfaction, while no information on caregivers’ characteristics was collected. The proposed data collection also includes questions intended to inform best practices for recruitment and retention of National Service participants. The data collection is not a duplicate of the HRS because it consists of a sample of caregivers receiving respite care and of adults as they begin National Service as Foster Grandparent or Senior Companion volunteers. The HRS does not collect information that can identify respite caregivers. The HRS is a national representative sample of non-institutionalized adults age 50 and older, and collects a diverse set of information on health care, housing, assets, pensions, employment, disability, and psychosocial well-being, all which are intended to inform a broad discussion about retirement. One of the objectives of the proposed data collection is to understand the effects of CNCS-funded services on the caregivers served by the SCP projects. The HRS does collect information about volunteering among adults age 55 and older which will be used in forming a comparison groups of volunteers and non-volunteers using propensity score matching methods to compare to the FGP and SCP volunteers.

5. If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden.

The proposed data collection will require input from FGP and SCP grantees that are non-profit entities; it will not impact small businesses. As mentioned above, to ensure that the baseline data collection occurs before the start of service, CNCS proposes to embed the informed consent process, enrollment and data collection during the intake process when new caregivers apply for respite care and during orientation for new volunteers. The grantees will be asked to explain the study to potential participants, obtain informed consent, and provide the survey to the participants. To offset the modest burden associated with the data collection activities, CNCS proposes a $10 research reimbursement per respondent to FGP and SCP projects for costs related to printing, phone use, and transmitting surveys to the contractor, JBS International. Research shows that indirect incentives to organizations increase cooperation rates (Beckler & Ott, 2006), as is the case for incentives to individual respondents.

6. Describe the consequence to Federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.

The Domestic Volunteer Service Act of 1973, as amended (Public Law 93-113), directs CNCS to assess the impact and effectiveness of Senior Corps Programs at least once every three years. In addition, CNCS’s 2011-2015 Strategic Plan outlines the agency’s commitment to building an evidence base that will allow for informed decisions and the allocation of resources. The proposed data collection is one of the activities the agency is undertaking to build its capacity to contribute to this evidence base with data collected through uniform, outcome-based performance measures and studies with a comparison group. The proposed data collection will provide information to CNCS on how to strengthen its national service programs so that caregivers receiving CNCS-supported respite care and volunteers participating in National Service achieve stronger psychosocial well-being. The FGP and SCP’s dual purpose is to engage individuals age 55 and older, especially those with limited income, in volunteer service to meet community needs, and to provide a high quality experience for the volunteers. The FGP’s program funds are used to support Foster Grandparents in providing supportive services to children with exceptional or special needs. The SCP program’s funds are used to support Senior Companion volunteers in providing supportive, individualized services to help adults with special needs maintain independence. The data collection will form the basis for improving and strengthening these services for volunteers and caregivers in National Service programs.

7. Explain any special circumstances that would cause an information collection to be conducted in a manner inconsistent with the general information guidelines in 5 CFR 1320.5.

There are no special circumstances that would require the collection of information in any other ways specified. This data collection is consistent with 5 CFR 1320.5.

8. If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the agency's notice, required by 5 CFR 1320.8(d), soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the agency in response to these comments. Specifically address comments received on cost and hour burden.

An agency’s notice in the Federal Register was completed. The 60-day Federal Notice was published on October 1, 2014 in the Federal Register, volume 79, number 190, pages 59254-59255.

• Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and record-keeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or reported.

CNCS convened a Technical Working Group (TWG) consisting of seven researchers with expertise in longitudinal surveys and evaluation designs, recruitment and retention strategies, survey design, psychometrics, and gerontology. Two WebEx meetings were held on January 13 and 15, 2015. CNCS sought recommendations on effective recruitment and retention strategies, feedback and suggestions on the surveys previously submitted with the 60-day Federal Notice, and recommendations on survey modality that would fill the knowledge gap on the health benefits of volunteering. Based on the TWG’s discussion and recommendations, CNCS revised several questions, and added questions to be able to measure loneliness, an area with a significant knowledge gap in the literature. Additional sociocultural questions were included to permit CNCS to reliably and efficiently estimate the effect of volunteerism on the psychosocial outcomes.

CNCS also held a meeting with staff at the National Institute of Aging to discuss the research gap on the effects of volunteering on psychosocial well-being including measures of satisfaction and loneliness and self-efficacy.

9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

Respondents will receive $20 for completing the baseline survey and each follow-up survey. Respondents can choose to receive the payment as a check or a gift card. The payment will be mailed directly to the respondents upon returning the completed survey. Research indicates that incentives increase response rates and retention; incentives can assist in locating respondents for follow-ups (Singer & Ye, 2013) which has been shown to offset the costs of follow-up and recruitment of non-respondents (Jäckle & Lynn 2008; Zhang, 2010).

10. Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statute, regulation, or agency policy.

The contractor, JBS International, will provide training and technical assistance (T/TA) to participating grantees/projects on the enrollment protocol; and JBS will provide T/TA to project staff on how to obtain informed consent. FGP and SCP project staff will explain the study to new caregivers and new volunteers when they sign up for services, provide information on who is conducting the study and why the study is being conducted. Project staff will describe the background, research activities, risks and benefits, and provide assurance of confidentiality. Respondents will also receive information on whom to contact if they have questions. Respondents will be informed of the voluntary nature of the research and their right to end participation at any time at each round of data collection. Respondents will be informed that their responses will be anonymous. Upon enrolling in the study, project staff will inform participants of the safeguards to protect their privacy.

The data collection contains limited, but important, PII. Per contractual agreement with JBS, CNCS has in place several safeguards to protect participants’ privacy. JBS will comply with CNCS privacy policies and procedures, adhere to CNCS standards for the protection of personally identifiable information, and staff working on the project will undergo CNCS privacy and network training. JBS will maintain information that identifies persons or institutions in files that are: physically separate from other research data; accessible only to authorized CNCS and JBS personnel; are maintained in an information system that has security controls in place as specified by the National Institute of Standards and Technology (NIST) Special Publication 800-53; have been certified and accredited in accordance with NIST SP800-37; and is compliant with FISMA. Furthermore, JBS personnel involved with the data collection, storage cleaning, coding, preparation and analysis has undergone criminal background clearance checks including fingerprinting.

11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private. This justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

Of the proposed questions, those on self-assessment of symptoms of depression, and loneliness are likely to be the most sensitive. The literature suggests that adults respond well to those questions and the data obtained are reliable, valid and of high quality. CNCS proposes these questions because evidence-based research findings are needed to better understand how volunteering is associated with these psychosocial outcomes. A rigorous analysis examining this association is important for CNCS to measure and report on progress in meeting its goal to strengthen National Service. Self-report of depressive symptoms are the 10-item scale developed by the Center for Epidemiological Studies Depression Scale (CES-D). This widely used scale in research has shown this scale to be reliable and valid. As part of the intake and sign up process, grantees already collect self-report information on health and medical conditions from caregivers and volunteers. The privacy assurances, as well as the rapport that exists between the grantees and the participants make these questions less intrusive. Appendix A includes a copy of the consent form to be used at each round of data collection for both studies.

Information will be analyzed in aggregate form and survey responses will be de-identified for analysis. Unless required by law, CNCS will hold all personal identifiers such as name and address in total confidence and will not release them. Upon conclusion of the data collection all personal identifiers will be destroyed.

Surveys: The proposed surveys are attached as Appendices B through G. There are two surveys for the Caregiver Study (baseline and one follow-up). There are four surveys for the Volunteer Study (baseline and two follow-ups). There are two versions of the second follow-up survey. Participants who left the volunteer program at the first follow-up survey will be asked to complete a shorter survey at the second follow-up survey.

The Caregiver Surveys collect information on how caregivers learned about the Senior Companion respite services, health characteristics, medical conditions, depressive symptoms, type and hours of respite services, satisfaction with respite services, life satisfaction, self-report health, and loneliness. The baseline Caregiver Survey also collects information about the relative or friend (e.g., the care recipient). The questions about the care recipient include their relationship to the caregiver, the care recipient’s health and functional status conditions, other support services the care recipient receives. The Caregiver Survey includes demographics and background measures such as age, gender, race/ethnicity, education, marriage and veteran status.

The Volunteer Surveys collect information on participants’ decisions to volunteer and decision to continue volunteering, type of volunteering activities, number of hours volunteering with FGP or SCP, volunteer experience with other organizations and hours performing volunteer activities with other organization, satisfaction with FGP or SCP volunteer experience, life satisfaction, self-report health and medical conditions, symptoms of depression, and loneliness. The surveys also include questions on whether the volunteers feel that their skills align with their role as volunteer, and the support they received from their project and station. Given that they are multiple data collection points, the data can be used to determine retention rate in the FGP or SCP program and the reasons that volunteer leave national service. The first follow-up Volunteer Survey primarily will include questions to measure the outcomes of interest: satisfaction with volunteer experience, life satisfaction, retention, self-rated health, symptoms of depression, and loneliness. All surveys include demographics and background measures such as age, gender, race/ethnicity, education, marriage and veteran status.

The baseline surveys and the first follow-up Volunteer Survey collect information for sending respondents the incentives and to aid in locating them for the follow-up data collection including name, address, telephone number and e-mail. The surveys also collect contact information for up to two relatives or friends to assist in locating respondents who might have changed address. As described previously, to safeguards the privacy of respondents, contact information will be kept physically separate from the research data and will be accessible only to authorized CNCS and JBS personnel who have undergone background clearance checks and CNCS security training.

12. Provide estimates of the hour burden of the collection of information. The statement should indicate the number of respondents, frequency of response, annual hour burden, and an explanation of how the burden was estimated. If this request for approval covers more than one form, provide separate hour burden estimates for each form and aggregate the hour burdens in Item 13 of OMB Form 83-I. Provide estimates of annualized cost to respondents for the hour burdens for collections of information, identifying and using appropriate wage rate categories.

The burden hours for each category of respondents subject to this clearance are described in Table 1.

Table 1. Burden Hours

Category of Respondents	No. of Respondents	Participation Time (Minutes) per Respondent	Burden Hours per Respondent	Burden Hours All Respondents
Caregiver Study
Baseline	926	30	0.50	463
Follow-up	740	20	0.33	244.2
Volunteer Study
Baseline	1,224	20	0.33	403.92
First follow-up	979	15	0.25	244.75
Second follow-up	783	20	0.33	258.39
SCP project staff
Caregiver Study1: Baseline and follow-up	142	90	1.50	213
Volunteer Study2: Baseline and two follow-ups	170	120	2.00	340
FGP project staff
Volunteer Study2: baseline and two follow-ups	309	120	2.00	618

¹ In the Caregiver Study, it is projected that 142 SCP grantees will be eligible to participate.

² In the Volunteer Study, it is projected that 170 SCP and 309 FGP grantees will participate.

The time to complete each survey will vary and depend on the characteristics of those completing the survey. Based on the pretest results, JBS estimates that respondents will take between 20 to 30 minutes depending on the survey, with the exception of the shorter first follow-up Volunteer Survey which will take an average of 15 minutes to complete.

• Caregivers: The estimated cost for caregivers is $17,085.95 which is based on an estimated hourly wage rate of $24.16 and total burden hours of 707.2 for caregivers. Caregivers can be of any age, though some proportion of them might employed or retired. The hourly wage estimates are based on the average hourly earnings of all employees in the private sector. The estimates were taken from the Bureau of Labor Statists, accessed at http://www.bls.gov/web/empsit/compaehes.txt.

• Volunteers: The estimated annualized cost for volunteers is $8,607.99 which is based on an estimated hourly wage rate of $9.49 and total burden hours of 907.06 for volunteers. Hourly wage estimates for elderly adults were based on average Social Security benefit for retired workers. The average monthly Social Security benefit for a retired worker was $1,328 in January 2015. Assuming a 35-hour work week, the average monthly benefit yields $9.49 per hour. The estimates were obtained from “Average monthly Social Security benefit for a retired worker”, accessed at https://faq.ssa.gov/ics/support/KBResult.asp?searchFor=average-monthly-social-security-benefit-for-a-retired-worker&button=Search&task=knowledge&basicSearch=1&page=1.

• FGP and SCP Project staff: The estimated annualized cost for grantees is $29,977.60 which is based on an estimated hourly wage rate of $25.60 and total burden hours of 1,171. Wage estimates are based on wage data for nonprofit private industry. These estimates were taken from the Bureau of Labor Statistics accessed at http://www.bls.gov/bdm/nonprofits/nonprofits.htm. For 2012, BLS estimated the average weekly wage in the nonprofit industry was $896. Assuming a 35-hour work week, the hourly wage in the nonprofit industry was estimated at $25.60.

13. Provide an estimate for the total annual cost burden to respondents or record keepers resulting from the collection of information.

The collection of this information does not have any capital or start-up costs; and it does not have any operation and maintenance costs.

14. Provide estimates of annualized cost to the Federal government; provide a description of the method used to estimate cost which should include quantification of hours, operational expenses, and any other expense that would not have been incurred without this collection of information.

For the survey design, sampling, implementation and analysis of the data collected the estimated annual cost to the Federal government is $460,895.19. This number is based on the portion of the contract with JBS International for Fiscal Year 2015 that is devoted to the design, sampling, implementation, and analysis of the data collection.

15. Explain the reasons for any program changes or adjustments reported on the burden worksheet.

This is an application for new collection. There are no program changes.

16. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

• Plans for Tabulation and Publication

JBS will analyze the data and provide CNCS a report for each study at the end of each data collection period (baseline, first and second follow-up). The reports will describe the evaluation findings for each round of data collection. The reports may be in the form of manuscripts intended for academic peer-review journals. JBS will provide supporting materials that will include research data files and a user’s guide for future research. CNCS will include the evaluation results in its Performance and Accountability Reporting to Congress and in Congressional Budget Justification.

• Analytic Techniques

Exploratory and descriptive analyses: JBS will conduct exploratory and descriptive analyses to assess data quality and reliability of the survey items. This is an important initial first step to detect errors, determine if there are any violations of statistical assumptions, determine the relationships among variables, assess the direction and size of relationships between explanatory and outcome variables, and generate hypotheses. This initial analysis will provide a solid knowledge base about the variables of interest and the relationships among them. In this initial phase of the analysis, JBS will calculate the response rates, screen the data for incomplete or missing data, detect outliers in responses, and combine items into scales.

Missing data and dropouts: There are two potential sources of missing data. One source of missing data is that participants may refuse to answer some questions. Another source is that participants exit the study and cannot be found to complete the follow-up data collection or decline to participate in the follow-up when contacted. During the exploratory analysis phase, JBS will identify the extent of missing data for the items in each survey. It is important to understand the source of missing values in order to select the most appropriate imputation method. This is important whenever missing values may also be related to observed characteristics of respondents. For example, if certain types of individuals refused to answer sensitive questions, or if participants who leave the study are different from those who remain. These are non-ignorable missing data (Allison, 2002). There are several classes of models for handing this type of missing data; a notable one is Heckman’s (1976) model for sample selection bias.

Imputation: Restricting the analysis to include only cases with complete data could bias sample statistics, such as regression coefficients, resulting in less powerful and reliable estimates. To address these potential limitations, JBS will explore the imputation of missing values for all variables with a large number of missing values using the imputation strategy described by Allison (2002) and Schaffer and Graham (2002). JBS will review, select, and apply the most efficient method, based on careful consideration of the dataset and type of missing data. It is important that the imputation model include all variables that are related to the variable being imputed and variables that are potentially related to the missing imputed variable. Moreover, the inclusion of a broad set of potential variables in the imputation model strengthens the validity of subsequent analyses of the imputed data set.

Statistical modeling: The Caregiver Study uses a pre/post design with a baseline and one follow-up data collection. The possible research questions include:

1. What are the demographic and health characteristics of caregivers receiving respite services?

2. What are the health status, medical conditions and functional limitations of the relative or friend in need of assistance (e.g., the care recipient)?

3. What service activities does the Senior Companion provide to care recipients? What is the average number of hours of respite services the Senior Companion provide in a typical week?

4. What other support services do care recipients have in addition to the Senior Companion? How long have the care recipients been using other support services? Are there differences in the use of other support services by care recipient’s health status, medical conditions and functional limitations? What is the association between the use of other support services, income and employment status of the caregiver?

5. What are caregivers’ expectations for seeking respite services? Do these expectations differ by the demographic and health characteristics of the caregiver? Do these expectations differ by health status, medical conditions and functional limitations of the care recipient?

6. How do caregivers’ health status, medical conditions, functional limitations, expectations and satisfaction with Senior Companion respite services contribute to the retention of caregivers with the Senior Companion respite services?

7. What is the variation in caregivers’ self-rated health, symptoms of depression, and loneliness at entry into the Senior Companion program? What are the observed changes following entry into the Senior Companion program?

8. Are there differences in change in caregivers’ life satisfaction, self-rated health, symptoms of depression and loneliness by types of program, demographic and health characteristics, and type of respite services received?

9. What is the association between type and hours of respite services, life satisfaction, self-rated health, symptoms of depression and loneliness controlling for caregivers’ demographic and health characteristics such as prior medical and psychiatric conditions? Are there differences by care recipient demographic and health characteristics?

The Volunteer Study consists of a baseline and two follow-ups. The possible research questions for the Volunteer Study include:

1. What are the reasons for national service participation?

2. What proportion of adults is new to volunteering as adults? What proportion adults previously volunteered with other organizations? What proportion of adults engage in volunteering opportunities with other organizations?

3. What types of sources do adults access to learn about volunteering? Are there differences by demographic characteristics, geographic location and type of program (FGP and SCP)?

4. Is there a difference in level of satisfaction with national service across type of program, demographic characteristics, and type of volunteering activities?

5. What are the primary reasons that adults stop volunteering? Is there an association between reasons participants stop volunteering, hours of service activities and level of satisfaction with the program? Are there differences by volunteers’ demographic characteristics?

6. What is the prevalence of engagement in other volunteering activities after leaving national service?

7. What are the differences in retention rates of national service volunteers? Do medical conditions, functional limitations and level of satisfaction contribute to exit out of national service participation? Are there differences by type of program?

8. What is the variation in volunteers’ self-rated health, symptoms of depression, and loneliness?

9. Is there a difference in volunteers’ retention, life satisfaction, self-rated health, symptoms of depression and loneliness across types of program, demographic and health characteristics, type of service activities and hours engaged in national service participation?

10. What is the pattern in volunteers’ life satisfaction, self-rated health, symptoms of depression and loneliness following entry into national service?

11. Are there differences in pattern of life satisfaction, self-rated health, symptoms of depression and loneliness by types of program, demographic and health characteristics, type of service activities and hours engaged in national service participation?

12. What is the association between national service participation, life satisfaction, self-rated health, symptoms of depression and loneliness accounting for demographic and health characteristics such as prior medical and psychiatric conditions? Are there differences by types of program?

13. Do national service volunteers differ in life satisfaction, self-rated health, symptoms of depression and loneliness compared to similar adults in the general population in the United States who do not volunteer and those who do volunteer?

The types of possible analysis to address these research questions include descriptive analysis, ANOVA, regression models, logistic regression, and multinomial logistic regression model. An example of a multinomial logistic model is where the dependent variable is measured as a categorical on a Likert scale such as life satisfaction or loneliness while controlling for demographic, health and functional characteristics, and hours in volunteering activities. If, for example, the dependent variable “loneliness” or “life satisfaction” is measured as an ordinal scale, the statistical model would be an ordered probit or logit model. An analysis that examines volunteer retention which is anticipated to be measured as dichotomous variable, the proposed model would be logistic or probit regression model to predict the odds that volunteers exit national service. Due to the clustered design robust standard errors that account for the non-independence of observations will be computed for all statistical techniques and models. Given that the Volunteer Study has three data collection points on the outcomes, it is possible to model change and event occurrence. These classes of statistical models are growth curve models and survival analysis or event history analyses (Blossfeld, Hamerle, & Mayer, 1989; Singer, & Willett, 2003) which estimate survivor functions of when, for example, volunteers exit national service, or how long volunteers remain in National Service.

Propensity score matching with HRS data: For the Volunteer Study, CNCS proposes to construct a comparison group of volunteers and non-volunteers from the HRS. In making such a comparison, the analysis must take into account that the FGP and SCP volunteers self-select to be part of the program. As such, a matched comparison analysis is needed to reduce selection bias. In this model, the participants in the Volunteer Study data collection will be matched to the respondents in the HRS data based on the estimated propensity to participate in Senior Corps program. The propensity score model would be estimated using a probit or logit model. The variables available for forming the comparison group are critical to justify the assumption that, after controlling for observable characteristics, the comparison group has, on average, the same outcomes as the Senior Corps volunteers group. It is important, therefore, for the comparison to be drawn from the same source (e.g., the same data collection survey), such that the variables used are similarly constructed. The proposed Volunteer Surveys (Appendices D through G) contain common variables with the HRS. Table 2 shows the list of common variables in the Volunteer Surveys and HRS datasets.

Table 2. Corresponding Common Variables in the Volunteer Survey and HRS

Senior Corps Volunteer Surveys	HRS Data	Items for propensity score model	Items that are outcomes to compare between the two groups
Baseline survey questions: Q5, Q7-Q12, Q15-Q28, Q29.3, Q29.5, Q30-Q38, Q33-Q35, Q37-Q44	Contains corresponding questions	Q5, Q7, Q10-Q11, Q15, Q16, Q20-Q28	Health (Q7, Q9); life satisfaction (Q8), symptoms of depression (Q12), loneliness (Q17)
Baseline survey questions: Q1-Q4, Q6, Q13-Q14, Q29.1, Q29.2, Q29.4	Not in the HRS data	Not applicable	Not applicable
First follow-up questions: Q9, Q10-Q20	Contains corresponding questions	Not applicable	Health (Q10, Q11), symptoms of depression (Q14), loneliness (Q17)
First follow-up questions: Q1-Q8	Not in the HRS data	Not applicable	Not applicable
Second follow-up questions: Q13-Q30 Second follow-up questions (for participants that left the program at first follow-up): Q3-Q12	Contains corresponding questions	Q17, Q18, Q20, Q21, Q24, Q25-Q30	Health (Q14, Q15); life satisfaction (Q16), symptoms of depression (Q19), loneliness (Q22) Second follow-up questions (for participants that left the program at first follow-up): Health (Q4); life satisfaction (Q5); symptoms of depression (Q7); loneliness (Q10)
Second follow-up questions: Q1-Q12, Q31-Q32 Second follow-up questions (for participants that left the program at first follow-up): Q1, Q2	Not in the HRS data	Not applicable	Not applicable

The common variables in the Volunteer Surveys and HRS datasets will be aligned in terms of their definition and measurements, and their distributions will be aligned as closely as possible to each other so as to minimize differences among common variables across datasets. For example, JBS will examine the ‘age’ variable in both data sets to assess whether the range of values for this variable is similar across both datasets. If they are not, then the ‘age’ variable will be harmonized across both datasets. JBS will handle missing data imputation similarly for both datasets. The HRS data will be appended to the data to create a combined data file. JBS will add a variable to indicate whether a case belongs to the Senior Corps volunteer group from the Volunteer Study data or a comparison case from the HRS data.

The propensity score model will identify and include variables that are either deterministic to time (e.g., age) or measured before participation so that these measures are not confounded with the outcomes; the measures will be constructed similarly in both datasets. Variables that are likely to affect both the participation and the outcome of interest will be considered. Moreover, missing data will be handled similarly for both groups. JBS will assess the degree of clustering within the data (it is anticipated that there will be clustering by geographic region, urban and rural placements) to determine whether to account for the clustering. JBS will compute robust standard errors that account for clustering at the grantee level.

As with the application of any statistical method, it is important to check the key assumptions that are made in the estimation as well as verify that the model specifications for creating the comparison group are appropriate and that the results do not create bias. These results need to be complemented with evidence of covariate balancing between groups and robustness checks. If these are not balanced, JBS will refine the model specification to improve the balance between the groups or potentially consider a different matching approach. JBS will use the nearest neighbor matching procedure to match the treatment cases to the comparison cases. JBS will use sampling without replacement, meaning each comparison group respondent will be included as a matched case only once. If there is one or more possible match, one case from the HRS group will be randomly selected. Each individual in the Senior Corps volunteer group will be matched to one individual in the HRS group and the properties of the matched have been verified and robustness checks are done, the impact of national service can be estimated as a weighted average of the difference in outcomes between Senior Corps volunteers and the HRS group.

Limitations: CNCS recognizes that the propensity score procedure for the Volunteer Study is subject to limitations and threats to internal validity that will be noted in all reports of the results. Specifically, the Volunteer Study seeks to match volunteers using propensity score matching procedures to the HRS using common measures from both collections that will be collected at different points in time by different organizations and under different conditions. Any observed differences may be due to these and other factors rather than effects of the volunteer program. However, CNCS believes that this research design will provide potentially useful insights about potential effects of the FGP and SCP program.

Statistical adjustments for mode effects: The primary survey mode is a self-report paper survey. A telephone survey will be used to contact non-respondents. Multimode survey designs are increasingly common as response rates fall and researchers attempt to find alternative ways to engage and solicit participation. The proposed paper and phone surveys will minimize cost by first soliciting participation through the most effective mode for this population and using more intensive non-response follow-up. The multimodal data collection is designed with high involvement and engagement with FGP and SCP grantees/projects to improve the overall response rate and promote retention for the follow-up surveys (Kolwnikov, & Kennedy2014). There are drawbacks to using a multimode approach in that it might introduce measurement error if participants answer particular questions mainly because of the mode in which the survey was administered. There are several approaches to statistically adjust for mode effects (Kolwnikov, & Kennedy 2014), including regression or multiple imputations. JBS will conduct an analysis to determine which survey items require mode effects adjustment, and apply the appropriate adjustments to these items.

Future research: In the final report, JBS will assist CNCS in identifying best practices for recruitment and retention, obstacles to response rates for future longitudinal studies, and assessments of the psychometric characteristics of the surveys used. These issues might include research objective, design, expected data analysis method, detectable power, sample size, type I error, variability, and effect size. JBS will use the results, including data from the first evaluation data collection, as well as prior results from other studies such as reports and papers that used the HRS data and peer-reviewed articles, to arrive at an informed set of values for the type of information needed to carry out a power analysis for future studies. JBS will conduct several analyses using reasonable ranges in each input component used in the power analysis to obtain a range of estimated sample size options and effect sizes. Using the results from the power analysis, JBS will provide cost estimates needed to implement future studies.

Data file for future research: JBS will produce a public access and agency specific data file for this data collection. The data files will be clean, well-organized, and adhere to privacy and security standards. JBS will create a de-identified and anonymous Public Use Data File. JBS will follow guidance on government-wide data sharing in all Federal statutes pertaining to data collection, privacy, and human subject research and take adequate steps to ensure that no individual is identifiable. In preparing public use data files, safeguards need to be established through the use of statistical disclosure limitation methodology for reporting findings. When collecting within small programs/projects, for example, it is important to ensure that project staff cannot identify their volunteers or caregivers. For the Public Use Data File, JBS will produce a sampling and methodology report that will include a copy of the survey, description of the methodology, and guidance on using post-sampling weights.

• Project Time Schedule (End dates)

	Caregiver Study		Volunteer Study
Activity/Task	Baseline	Follow-up	Baseline	1st Follow-up	2nd Follow-up
Data collection	10/31/15	10/31/16	10/31/15	10/31/16	10/31/17
Conduct data analysis	4/1/16	4/1/17	4/1/16	4/1/17	4/1/18
Final Report including feasibility of future studies	7/1/18	7/1/18	7/1/18	7/1/18	7/1/18
Submit data files to CNCS for future research	9/28/18	9/28/18	9/28/18	9/28/18	9/28/18

The first follow-up will occur 9-12 months after enrollment, and is the same for both studies. The follow-up window is tied to each participant’s initial enrollment date in the study. The first follow-up will begin nine months after the first sample member is enrolled, and that sample member will remain eligible for follow-up until the maximum time permitted has elapsed which is 12 months after enrollment. For example, if the first participant is enrolled on May 1, 2015. The follow-up for this participant hits nine months later, on February 1, 2016. This participant would remain eligible for the follow-up through June 1, 2016, when the participant hits the 12^th month since enrolling in the study. The second follow-up for the Volunteer Study will occur 21-24 months from the date of initial enrollment. Participants will remain eligible for the second follow-up through their 24 month following enrollment. The four months window maximizes the follow-up periods and the response rate.

17. If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

The expiration date for OMB approval of the information collection will be displayed on the first page of the surveys.

18. Explain each exception to the topics of the certification statement identified in “Certification for Paperwork Reduction Act Submissions” (5 CFR 1320.9).

There are no exceptions claimed in Item 19, "Certification for Paperwork Reduction Act Submissions" of OMB Form 83-I.

Page 8 of 8