20200716 life cycle evaluation ORC SSA

Corporation for National and Community Service (dba AmeriCorps) is a Federal agency that promotes volunteering and service through a wide range of programs and grants to “…improve lives, strengthen communities, and foster civic engagement.” In response to the 2017 public health emergency declaration, AmeriCorps increased its efforts to fund programs specifically targeting the opioid crisis. One promising strategy to address the rising rates of opioid use disorders (OUD) and drug overdose is peer recovery coaching (also known as recovery coaching or peer coaching) through AmeriCorps volunteers. Peer recovery coaching is the process in which a non-clinical professional with lived experience provides guidance to individuals with an OUD by helping them access care and supporting them in the removal of barriers to recovery.

A 2020 report by AmeriCorps’ Office of Research and Evaluation reviewed the use of the peer recovery coach model by previous AmeriCorps grantees. While the report showed that the use of the model is promising, it noted that additional study of grantees’ use of the model was necessary. Empirical evidence is limited on how the peer recovery coach model contributes to outcomes of volunteers and beneficiaries when implemented by grantees. Empirical evidence is also limited regarding a greater understanding of program designs and implementation (e.g., how grantees recruit and train peer recovery coaches, what kinds of support grantees provide to peer recovery coaches, how grantees and volunteer peer recovery coaches implement the recovery coach model, how the grantee organization and volunteer peer recovery coaches work with outside organizations to provide holistic care to beneficiaries, and how the model affects beneficiaries’ and peer recovery coaches’ recovery paths).

AmeriCorps is committed to using research and evaluation to systematically learn from the programs it supports, and especially about solutions (such as the more recent use of the peer recovery coach model) that work and the contexts in which these solutions work. This study is part of an effort to assess how well peer recovery coaching works for individuals with OUD served by AmeriCorps-funded grantees and the extent to which this model may be used successfully by other AmeriCorps grantees in the future.

AmeriCorps has contracted with ICF to conduct a bundled evaluation of grantees that are implementing peer recovery coach models. By bundling, AmeriCorps combines a group of small programs across different funding streams with similar program models and intended outcomes into a single evaluation. Spanning 27 months, the evaluation works with 14 grantees to examine program design, implementation, and outcomes using surveys, interviews, and focus groups. The evaluation covers a wide range of stakeholders including grantee staff, volunteers who support the peer recovery coach model, beneficiaries, and staff at organizations that partner with grantees.

The information obtained from the evaluation will allow AmeriCorps to understand the program design and implementation, determine the efficacy of the model, and provide guidance for how the model might best be used by grantees in the future.

A2. Indicate how, by whom, and for what purpose the information is to be used.

The purpose of this evaluation is to study questions regarding grantees’ use of the peer recovery coach model; better determine how effective the model is at increasing individuals’ recovery capital, increasing attendance of health services, and decreasing incidences of substance use; and measure the effectiveness of the model on improving outcomes for the peer recovery coaches and grantee organizations. The research questions for this evaluation are:

What do peer recovery coach models look like (activity, setting, modality, etc.)?
What are promising practices and challenges in implementing these models?
To what extent do peer recovery coach models contribute to improving outcomes for grantee organizations, recovery coaches, and beneficiaries?

Thematic analyses will be conducted from the data collected through focus groups. For data collected through surveys, analysis will focus on changes between responses within the first and second survey, differences between the outcomes of beneficiaries who received peer recovery coaching versus beneficiaries who received other services for OUD, and any relationships between implementation and outcomes. In addition, the project will provide evaluation capacity building activities to participating grantees using experiential learning, and will assess the effect of these activities using pre–post outcome surveys and post-presentation surveys.

Information gained from this evaluation will be used by AmeriCorps and the field to better understand the model’s efficacy and best practices. We will disseminate 1) a final technical report with detailed information on research, evaluation, and capacity building; 2) a logic model, theory of change, specific tools, and data files designed to guide action by informing specific aspects of program design; and 3) practitioner and research briefs to increase awareness.

A3. Minimize burden: use of improved technology to reduce burden.

Several efforts have been made to minimize the burden on respondents. In designing the surveys and focus groups, each question was carefully considered to avoid duplication of data available from other sources.

The surveys will be administered online using Qualtrics. Links to the online surveys will be emailed out to potential respondents. Focus groups may be carried out in person or virtually using Microsoft Teams.

A4. Non-duplication.

The instruments developed for this evaluation go beyond any reporting that AmeriCorps grantees currently submit. Development of data collection instruments was informed by extensive review of literature and grantee applications. However, information regarding grantees’ use of the peer recovery coach model, peer recovery coaches’ attitudes toward and roles in the model, grantee partners’ view of the model, and beneficiaries’ experiences with the model is not currently collected. It is needed to understand program design and implementation, and to determine the efficacy, challenges, and promises of the model for future AmeriCorps grantees.

A5. Minimizing economic burden for small businesses or other small entities.

Every effort was made to minimize the length of the data collection (e.g., focus groups are 60 minutes and the surveys take 10-15 minutes).

For focus groups, the researchers will schedule the data collection at times that are convenient to the participants in an effort to minimize disruption or interference with the normal activities of the respondents. Surveys will be administered online so respondents can respond when it is most convenient for them.

A6. Consequences of the collection if not conducted, conducted less frequently, as well as any technical or legal obstacles to reducing burden.

Currently, AmeriCorps has limited information on the design, implementation, and efficacy of the peer recovery coach model as used by its grantees to tackle the opioid crisis. To properly assess the model’s usefulness and compile best practices and challenges for the model, there is an urgent need from the agency and the field to learn about grantees’ use of the model and the model’s impact on peer recovery coach grantees, volunteers, and beneficiaries. As a key component of decision-making in the future about whether and under what conditions grantees might incorporate the peer recovery coach model, these empirical data are important to aid those considering the options and the modifications that may be needed. Without these data, uninformed and inadequate decisions might be made. This input also provides information and insights to help policymakers and researchers determine the research agenda in subsequent years.

A7. Special circumstances that would cause information collection to be collected in a manner requiring respondents to report more often than quarterly; report in fewer than 30 days after receipt of the request; submit more than an original and two copies; retain records for more than three years; and other ways specified in the instructions focused on statistical methods, confidentiality, and proprietary trade secrets.

There are no special circumstances that would require the collection of information in these ways.

A8. Provide copy and identify the date and page number of publication in the Federal Register of the agency’s notice. Summarize comments received and actions taken in response to comments. Specifically address comments received on cost and hour burden.

As required by 5 CFR 1320.8(d), a notice was published in the Federal Register on May 17, 2021, page number 86: 26703, for 60 days. A notice was also published in the Federal Register on July 27, 2021 86: 40202, for 30 days.

AmeriCorps contractor ICF:

Performed a review of grantee applications and previous studies of the peer recovery coach model to gain an understanding of how the programs operate, grantees’ use of the model, and scholarly research on the use of the peer recovery coach model with individuals who had substance use disorders including OUD.
Convened a Technical Working Group made up of six external experts in national service and evaluation methodologies to receive input regarding the evaluation.
Consulted a Field Working Group made up of six AmeriCorps program leaders for input on the evaluation.

A9. Payment to respondents.

In general, no payment will be offered to respondents. However, to increase response rates from the comparison group that is not receiving peer recovery coaching, ICF will offer a $10 gift card for their completion of each of two surveys.

A10. Assurance of confidentiality and its basis in statute, regulation, or agency policy.

ICF will comply with AmeriCorps privacy policies and procedures and adhere to AmeriCorps standards for the protection of personally identifiable information.

In addition, all the data collection instruments, consent forms, and recruitment materials are approved by ICF Institutional Review Board, where assurance of confidentiality is guaranteed. Consent will be collected as part of the online survey or verbally for focus groups prior to the data collection. Data will be securely stored in a project folder and accessible to the research team only. In the reporting, there will be no identifying information supplied with findings that would permit a comment or finding to be associated with an individual or a position.

A11. Sensitive questions.

Since the evaluation is on the use of the peer recovery coach model with individuals who have OUD, researchers will ask participants about substance use and opioid addiction. ICF does not ask beneficiaries to describe the circumstances of their addiction, in order to minimize the possibility of retraumatization, but it is possible that they may choose to share that information as part of the survey or focus groups. All survey and focus group participants will be informed of the study purpose and the nature of the researchers’ questions prior to their participation. All respondents will be asked to give consent prior to participating. Should a respondent not wish to respond to any question, they will not be required to do so.

A12. Hour burden of the collection.

Exhibit A-1. Estimated hour burden

Respondent Category	Number	Participants	Time (minutes)	Total Hours
Beneficiary focus groups	9 groups	Up to 90	60 mins/focus group	90 hours
Beneficiary surveys	2 times	Up to 500 (assuming 30% response rate)	15 mins/survey	250 hours
Peer recovery coach surveys	2 times	Up to 150	15 mins/survey	75 hours
Grantee staff surveys	2 times	Up to 150	15 mins/survey	75 hours
Evaluation technical assistance (TA) pre–post outcome surveys	2 times	Up to 30	10 mins/survey	10 hours
Evaluation TA post-presentation surveys	1 time	Up to 135 (i.e., 9 presentations)	10 mins/survey	45 hours
Total hour Annual total hour				545
Total hour Annual total hour				218

A13. Cost burden to the respondent.

Peer recovery coaches: The estimated total cost for volunteers is $648, which is based on an estimated hourly wage rate of $8.64 and a total burden hours of 75. Assuming a 35-hour work week, the average monthly stipend yields $8.64 per hour based on a monthly living allowance of $1,210 per month. This stipend amount was obtained from “Round Robin: AmeriCorps: Living Allowance Distributions,” accessed at https://www.nationalservice.gov/sites/default/files/resource/rr3-ac-living-allowances-4-13-12.pdf.

Beneficiaries: Assuming a $16.54 hourly wage, the estimated total cost for beneficiaries is $5,623.60, which is based on a total burden hours of 340. This estimate is based on the 2019 living wage from MIT’s Living Wage calculator, accessed at https://livingwage.mit.edu/articles/61-new-living-wage-data-for-now-available-on-the-tool#:~:text=The%20living%20wage%20in%20the,wage%20for %20most%20American%20families. For 2019, the estimate was $16.54 per hour for a family of four (two working adults and two children).

Project staff: The estimated total cost for grantees is $3,068, which is based on an estimated hourly wage rate of $23.60 and total burden hours of 130. Wage estimates are based on wage data for private nonprofit establishments in the health care and social assistance industry. These estimates were taken from the Bureau of Labor Statistics (BLS), accessed at http://www.bls.gov/bdm/nonprofits/nonprofits.htm. For 2017, BLS estimated the average weekly wage in the nonprofit industry was $944. Assuming a 40-hour work week, the hourly wage in the health care and social assistance, nonprofit industry was estimated at $23.60.

A14. Cost to government.

For designing and implementing the plan and implementation of the peer recovery coach bundled evaluation, the estimated annual cost to the federal government is $9,339.6 ($648+$5,623.6+$3,068=$9,339.6). This number is based on the portion of the contract with ICF for fiscal years 2020-2025 that are devoted to this evaluation.

A15. Reasons for program changes or adjustments in burden or cost.

No change in burden is requested. This submission to the Office of Management and Budget (OMB) is for an initial request for approval.

A16. Publication of results.

A report of findings, tools, and a brief will be prepared for AmeriCorps. Publication and dissemination will be decided by AmeriCorps after the completion of the evaluation in 2023.

A17. Explain the reason for seeking approval to not display the expiration date for OMB approval of the information collection.

AmeriCorps does not seek this exemption. The OMB expiration date will be displayed on the introductory page of the data collection instrument.

A18. Exceptions to the certification statement.

AmeriCorps is not requesting an exception to the certification requirements.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Holbrook, Brynn
File Modified	0000-00-00
File Created	2021-07-29