20210930 VGF Life Cycle Evaluation SSA

CNCS

Supporting Statement A for

Paperwork Reduction Act Submissions

Volunteer Generation Fund (VGF) Grant Program Bundled Evaluation

A1. Need for information collection.

This new information collection, for the Volunteer Generation Fund (VGF) Grant Program Bundled Evaluation, will evaluate the:

1. Design, implementation and outcomes of the AmeriCorps Volunteer Generation Fund

2. Effectiveness of evaluation capacity building workshops to be provided to the bundle participants.

AmeriCorps is a federal agency that promotes volunteering and service through a wide range of programs and grants to improve lives, strengthen communities, and foster civic engagement. AmeriCorps engages more than five million Americans in service each year through its AmeriCorps, AmeriCorps Seniors, AmeriCorps VISTA and Volunteer Generation Fund (VGF) grant programs. The VGF was launched in 2010 to provide funding to AmeriCorps state service commissions that administer initiatives to expand, strengthen and develop volunteer management capabilities.

The VGF Bundle is part of a multi-project contract titled “Life Cycle Evaluations” (LCE). This project is being conducted by ICF International, through a contract with AmeriCorps. The LCE contract will conduct up to 4 sequenced program evaluations (descriptive and/or outcome based) and provide tailored evaluation capacity building workshops to grantee organizations “bundled” together in programmatic or priority focus areas.

The bundling approach seeks to support grantee organizations that may not yet have the required capacity to conduct evidence building activities on their own and/or where no evidence base exists for their program area of work. By bundling, AmeriCorps can combine groups of programs across different funding streams with similar models and intended outcomes into a single evaluation while at the same time, providing a peer-learning and networking opportunity through evaluation capacity building workshops. Furthermore, this approach supports the establishment of communities of practice to identify common best practices and lessons learned that can be used by the broader public.

This study will be the first time AmeriCorps will synthetize information about VGF and the state service commissions approaches and models. More specifically, the effort will assess the approaches that VGF funded state service commission and their partner organizations use to sustainably increase volunteer recruitment and retention within their states, and the extent to which these approaches may be used successfully by other AmeriCorps VGF grantees in the future.

ICF conducted a systematic review of fiscal year (FY) 2020 grantee application narratives and intended outcomes and, an extensive scholar literature review that informed the drafting of the VGF evaluation plan. ICF also identified two evaluations of previous VGF grant programs, one in Iowa and one in Florida. Both evaluations focused on grantee support of community-based entities and the impact of this support on subgrantees and volunteers. Both grantees provided volunteer management training and technical assistance to these entities. While these studies showed that the use of these approaches to volunteer management training and technical assistance is associated with positive outcomes, they both noted that additional study of grantees’ use of these models is necessary.

The literature review found that empirical evidence is limited on how differences in approaches to volunteer management training and technical assistance impact volunteer recruitment and retention. Additionally, neither VGF evaluation study explored other components of VGF grantee approaches to volunteer recruitment and retention, including developing volunteer organization networks, using volunteer management systems and tools (e.g., online platforms and portals), creating opportunities for specific volunteer populations (e.g., youth, corporate employees), and expanding disaster and crisis response.

Spanning 27 months, this bundled evaluation will include all 14 grantees from FY 2020 to examine program design, implementation, and outcomes using surveys, interviews, focus groups, and site visits. FY20 grantees were selected to ensure that each program had at least some degree of maturity that could be evaluated.

The evaluation respondents will include a wide range of stakeholders including

• grantee program directors and staff

• volunteer management training and capacity-building personnel;

• VGF grant program beneficiaries;

• Volunteers volunteering with beneficiaries.

The instruments submitted for approval include questions about grantees’ approaches for using VGF funds to recruit, retain and support volunteers, promising practices, and key challenges, as well as preliminary outcomes on volunteer organizations.

A2. Indicate how, by whom, and for what purpose the information is to be used.

The purpose of this evaluation is to understand grantees’ use of VGF grant program funds to support volunteer organizations and better determine how effective grantees’ approaches are at enhancing the capacity of these organizations, increasing volunteer recruitment and retention, and increasing implementation of volunteer management best practices within their states. The research questions for this evaluation are:

1. What are the grantees’ approaches for utilizing VGF funds to improve volunteer recruitment, retention, and support of volunteers within their states and among volunteer organizations?

2. What are promising practices and challenges in implementing the VGF grants?

3. What are intended outcomes of VGF grants on volunteer organizations e.g., improved quality, effectiveness and reach of volunteer services delivered to target populations and communities?

The project has two goals: 1) gather data to improve the VGF program and its outcomes; 2) improve the evaluation capacity of its grantees by delivering targeted technical assistance based on the data collected, and measure the impact of that TA.

Thematic analyses will be conducted from the data collected through interviews and focus groups. For data collected through surveys, analysis will focus on volunteer organizations’ perceptions of volunteer management training and technical assistance, relative to their needs, and its effects. In addition to participating in the evaluation, the grantees will participate in evaluation capacity building workshops with 12 sessions facilitated by ICF in one year using experiential learning. To that end, ICF will assess the effect of evaluation capacity building using pre–post outcome surveys and post-presentation surveys.

Information gained from this evaluation will be used by AmeriCorps and the field to better understand the program’s efficacy and best practices. We will disseminate
1) a final technical report with detailed information on research, evaluation, and capacity building, 2) a logic model, theory of change, specific tools, and data files designed to guide action by informing specific aspects of program design, and 3) practitioner and research briefs to increase awareness.

A3. Minimize burden: use of improved technology to reduce burden.

Several efforts have been made to minimize the burden on respondents. In designing the surveys and focus groups, each question was carefully considered to avoid duplication of data available from other sources.

The surveys will be administered online using the FedRAMP-approved Qualtrics survey platform for secure data collection. Links to the online surveys will be emailed to potential respondents, allowing respondents to complete the survey at their convenience without retrieving and/or returning paper forms. Interviews and focus groups may be carried out in person or virtually. In-person interviews and focus groups will be conducted at a convenient time and location for the participants. Virtual interview and focus groups will be conducted using ICF-secured and managed Microsoft Teams.

A4. Non-duplication.

The instruments developed for this evaluation go beyond any reporting that AmeriCorps VGF grantees currently submit. Development of data collection instruments was informed by extensive review of literature and grantee applications. However, information regarding grantees’ approaches to supporting volunteer management, program implementers’ approaches to training organizations in volunteer management practices, beneficiaries’ perceptions of training and technical assistance, and volunteers’ experiences with organizations that have received training and capacity building services is information that is not currently collected. It is needed to understand program design and implementation, and to determine the efficacy, challenges, and promises of the model for future AmeriCorps VGF grantees.

A5. Minimizing economic burden for small businesses or other small entities.

Every effort was made to minimize the length of time for each data collection effort (e.g., surveys take 10-15 minutes).

For interviews and focus groups, the researchers will schedule the data collection at times that are convenient to the participants in an effort to minimize disruption or interference with the normal activities of the respondents. Surveys will be administered online so respondents can respond when it is most convenient for them.

A6. Consequences of the collection if not conducted, conducted less frequently, as well as any technical or legal obstacles to reducing burden.

Currently, AmeriCorps has limited information on the design, implementation, and efficacy of grantees’ VGF grant program approaches to increase volunteer recruitment and retention in their states. To properly assess the program’s usefulness and compile promising practices and challenges, there is an urgent need from the agency and the field to learn about grantees’ approaches and the outcomes of these approaches on program implementers, beneficiaries, and volunteers.

As a key component of decision-making about what approaches are effective in sustainably increasing the impact of volunteers on critical community needs, these empirical data are important to aid those considering the options and the modifications that may be needed. Without these data, uninformed and inadequate decisions might be made. This input also provides information and insights to help policymakers and researchers determine the research agenda in subsequent years.

A7. Special circumstances that would cause information collection to be collected in a manner requiring respondents to report more often than quarterly; report in fewer than 30 days after receipt of the request; submit more than an original and two copies; retain records for more than three years; and other ways specified in the instructions focused on statistical methods, confidentiality, and proprietary trade secrets.

There are no special circumstances that would require the collection of information in these ways.

A8. Provide copy and identify the date and page number of publication in the Federal Register of the agency’s notice. Summarize comments received and actions taken in response to comments. Specifically address comments received on cost and hour burden.

As required by 5 CFR 1320.8(d), a notice was published in the Federal Register on May 17, 2021, page number 86: 26703, for 60 days. A notice was also published in the Federal Register on July 27, 2021 86: 40202, for 30 days.

The AmeriCorps contractor ICF conducted the following efforts to consult with persons and information relevant to this information collection effort.

• A review of grantee applications and literature (scholarly and grey) of volunteer management practices was conducted to gain an understanding of how grantees approach developing and/or supporting community-based entities to recruit, manage, and support volunteers; what evidence base exists for volunteer recruitment, engagement, and management practices; and whether grantee approaches align with this evidence base.

• A Technical Working Group made up of six external experts in national service and evaluation methodologies was convened. It offered input regarding the evaluation.

• A Field Working Group made up of six AmeriCorps program leaders offered input to the evaluation.

A9. Payment to respondents.

No payment will be offered to respondents.

A10. Assurance of confidentiality and its basis in statute, regulation, or agency policy.

ICF will comply with AmeriCorps privacy policies and procedures and adhere to AmeriCorps standards for the protection of personally identifiable information.

In addition, all of the data collection instruments, consent forms, and recruitment materials are approved by ICF Institutional Review Board, where assurance of confidentiality is guaranteed. In the reporting, there will be no identifying information supplied with findings that would permit a comment or finding to be associated with an individual or a position.

A11. Sensitive questions.

Since the evaluation is of grantees’ approaches (e.g., volunteer management training and support) to increase volunteer recruitment and retention within their states, there are no questions of a sensitive nature or of matters that are commonly considered private.

A12. Hour burden of the collection.

Exhibit A-1. Estimated hour burden

Respondent Category	Number	Participants	Time (minutes)	Total Hours
State Service Commission (Grantee) Interviews, Round 1	1 interview	14 participants	90 mins /interview	21 hours
State Service Commission (Grantee) Interviews, Round 2	1 interview	14 participants	60 mins /interview	14 hours
Program Implementer Interviews	1 interview	14 participants	60 mins /interview	14 hours
Beneficiary (Volunteer Organization) Focus Groups (FGs)	1 FG	140 participants (10 participants /FG, 14 FGs)	90 mins /focus group	210 hours
Beneficiary (Volunteer Organization) Surveys	1 survey	1,920 participants (assuming 20% response rate)	15 mins /survey	480 hours
Volunteer Focus Groups (FGs)	1 FG	140 participants (10 participants /FG, 14 FGs)	90 mins /focus group	210 hours
Evaluation Technical Assistance Pre–Post Surveys	2 surveys	30 participants	10 mins /survey	10 hours
Evaluation Technical Assistance Post-Presentation Surveys	9 surveys	14 participants	10 mins /survey	21 hours
Total hour				980
Annual total hour				392

A13. Cost burden to the respondent.

Project staff (state commission, program implementers, and volunteer organization staff): The estimated total cost for grantees is $18,172, which is based on an estimated hourly wage rate of $23.60 and total burden hours of 770. Wage estimates are based on wage data for private nonprofit establishments in the health care and social assistance industry. These estimates were taken from the Bureau of Labor Statistics (BLS), accessed at http://www.bls.gov/bdm/nonprofits/nonprofits.htm. For 2017, BLS estimated the average weekly wage in the nonprofit industry was $944. Assuming a 40-hour work week, the hourly wage in the health care and social assistance, nonprofit industry was estimated at $23.60.

Volunteers: The estimated total cost for volunteers is $1,814.4 for 210 hours. Assuming a 35-hour work week, the average hourly stipend yields $8.64 based on a monthly living allowance of $1,210 per month. https://www.nationalservice.gov/sites/default/files/resource/rr3-ac-living-allowances-4-13-12.pdf.

A14. Cost to government.

For designing and implementing the evaluation, capacity building, and dissemination of the VGF bundled evaluation, the estimated annual cost to the federal government is $19,986.4 ($18,172=$1,814.4=$19,986.4). This number is based on the portion of the contract with ICF for fiscal years 2020-2025 that are devoted to this evaluation.

A15. Reasons for program changes or adjustments in burden or cost.

No change in burden is requested. This submission to the Office of Management and Budget (OMB) is for an initial request for approval. 

A16. Publication of results.

A report of findings, tools, and a brief will be prepared for AmeriCorps. Publication and dissemination will be decided by AmeriCorps after the completion of the evaluation in 2023.

A17. Explain the reason for seeking approval to not display the expiration date for OMB approval of the information collection.

AmeriCorps does not seek this exemption. The OMB expiration date will be displayed on the introductory page of the data collection instrument.

A18. Exceptions to the certification statement.

AmeriCorps is not requesting an exception to the certification requirements.

4