Justification_v5_11-13-12

SUPPORTING STATEMENT

For

Paperwork Reduction Act Submission

Understanding the Value of Service in Participants’ Experience and Outcomes

Submitted by:

Corporation for National & Community Service

1201 New York Avenue, NW
Washington, DC 20525

Prepared by:

ICF International, Inc.

November 15, 2012

Supporting Statement

Part A. Justification:

1. Necessity of Information Collection

The Corporation for National and Community Service (CNCS) engages more than five million Americans as community volunteers (i.e., service participants) in service through its core programs (e.g., Senior Corps, AmeriCorps). CNCS recently developed an ambitious five-year plan in which the agency commits to strengthening the national service experience for individuals who participate in its programs. One goal associated with this plan is that participants in CNCS-supported programs consistently have satisfying experiences that provide meaning and opportunity.

CNCS believes that outcomes for both the people who receive services and the service participants are improved when the participant has a positive service experience filled with opportunities for professional, educational, and personal growth. As such, the following goals underlie this research for CNCS:

• Strengthening national service by ensuring that participants consistently find intellectual and personal growth;

• Determining what contributes to a defining or transformative service experience;

• Assessing whether and how CNCS programs are helping to provide that defining experience;

• Understanding whether, how, and when participants experience connections to their service experience; and

• Developing knowledge and refining hypotheses about whether, how, when, and why participation in national service delivers benefits to the participants themselves.

To investigate the experiences of service participants and to help inform these goals, CNCS awarded ICF International a project that involves surveying current service participants and recent alumni of CNCS-supported programs to answer the following research questions:

• What relates to individuals’ decision to serve?

• What most positively impacts service-related attitudes?

• What influences service outcomes, such as improved educational or job opportunities and civic participation?

Thus, this survey is designed to help gather necessary information to understand the value of national service from the perspective of the national service participant. Information collected based on these research questions will help to inform the larger CNCS goals.

2. Needs and Uses

This data collection is specifically designed to solicit feedback from participants who are currently serving in CNCS service programs as well as participants who completed service with CNCS within the past one to two years about their decision making regarding entering service, participant experiences during service, and the impact of the service experience on service participants. The information gathered during this data collection will be used to better understand participants’ satisfaction with programs and how various aspects of the service experience relate to participant satisfaction and post-service outcomes (e.g., community involvement following service). Specifically, CNCS programs will use this information to help determine how to become more effective at offering a positive experience to service participants, increasing interest in prospective participants, and making national service accessible and attractive. The survey tool is designed to gather information from service participants on a variety of topics to help meet these needs.

This is a request for a new, one-time only, information collection. No prior data has been obtained.

3. Efforts to Minimize Burden

Efforts were made to minimize the burden on respondents. First, responding to the survey is voluntary and is not expected to have any residual effects on service participants. Further, in designing the survey, each question was carefully considered and efforts were made to avoid duplication. The burden will be further minimized by making sure that participants only receive questions that are relevant to them by employing skip patterns in the online survey. Finally, the survey will be completed primarily using an online administration. To complete the survey, participants will be electronically sent a link to an online survey tool. This means was selected to lessen the burden on participants by allowing them to complete the survey at a time and in a place that is most convenient to them. This also reduces burden by not requiring the participant to take the time or effort required to mail back a paper survey. If participants are not able to complete the survey online, they will be able to request a paper version of the survey, but it is expected that the majority of survey responses will be in an online format.

4. Efforts to Identify Duplication

There is no duplicative effort. The survey does not duplicate a current data collection instrument and there is not information currently available to sufficiently answer the research questions of the study.

5. Methods to Minimize Burden on Small Business

There is no significant impact on small businesses or other small entities as a result of this data collection.

6. Consequences of Less Frequent Collection

Without this data collection or with fewer respondents, CNCS will not be able to fully understand the service participant experience and would therefore be unable to work towards improving this experience for future service participants.

7. Special Circumstances Influencing Collection

There are no special circumstances that would influence the collection of information for this project. This is a one-time data collection.

8. Reasons for Inconsistencies with 5 CFR 1320.6

A 60-day public comment Notice was published in the Federal Register on November 25, 2011. This comment period ended January 24, 2012. One public comment was received from this Notice, submitted by the Washington Commission for National & Community Service. With one exception, all of the Commission’s recommendations were incorporated into the survey.

9. Payment or Gift to Respondents

No payments or gifts will be given to respondents for participation in the survey.

10. Assurance of Confidentiality

The data gathered in this survey will be confidential, but not anonymous. In order to avoid revealing respondents’ identities, the researchers will remove any individual identifiers from the data file, and survey responses will only be reported in aggregate form. This survey will be administered in an online format using Vovici EFM Community Web-based survey software. Vovici’s survey hosting environment has been designed with security as a foremost consideration, with features such as 128 bit SSL encryption and redundant firewalls. Participant email addresses will be uploaded into the Vovici Community secure Web site and each respondent will be sent a link to the survey via their email address. Although the survey administrators can identify a participant’s survey status (e.g., not started, started, completed), there are no individual identifiers in the data and there is no way to link an individual’s data to their email address. All data exported from the Vovici Community secure Web site will be kept in a secured folder. If the respondent does not have an email account or internet access, participants can request a paper-based survey mailed to them. A self-addressed stamped envelope for easy return of the survey to ICF International will be included with each survey mailed. Providing the self-addressed stamped envelope will help to maintain confidentiality of participants’ data as their project directors and CNCS will not have access to completed paper surveys.

Respondents will be informed of the confidentiality of their responses on the actual survey, whether it is the online version or the paper version. The following text will be provided before respondents complete any of the survey items:

Confidentiality Statement:

This survey guarantees respondent confidentiality. Confidentiality means that only the research team knows who responded to the survey. All data collected from you on this survey will be aggregated with data from other respondents. Therefore, data will only be used in a way that will make it impossible to determine the identity of individual respondents. The data will not be analyzed or reported in any way that compromises participant confidentiality. Raw data will only be accessible by ICF International employees directly involved in data analysis, will be stored in a secure folder, and any identifying information will be removed before data analysis begins.

11. Justification for Sensitive Questions

There may be some questions that some participants may perceive as sensitive or private. However, these questions are fundamental to understanding factors that impact reasons for serving and may help explain why service experiences differ. Participants will be briefed on the purpose of the data collection and they will be notified that they are not required to answer all questions and they can refuse to answer questions without consequence. They will also be assured that all responses will remain confidential and will not be attributed to them.

The study report will not include information that can be used to identify respondents. The report will include statistical data and other aggregated data for improving CNCS programs.

12. Estimate of Hour Burden

The total response burden for this information collection is provided in the exhibit below.

Estimated Respondent Burden for Data Collection Activities
Data Collection Activity	Number of Respondents	Average Burden Per Respondent	Burden Hours
Survey of CNCS service participants	4,000	20 minutes	1,333
TOTALS	4,000		1,333

This is a one-time data collection effort. The average time burden per participant was estimated by having ICF international employees pilot the survey instrument to determine the amount of time necessary to complete all questions.

13. Estimate of Cost Burden

The burden estimate for survey respondents is based on surveying 3,600 current service participants and recent alumni of CNCS-sponsored programs. The estimate of time to complete the survey is 20 minutes. The calculation of the total response burden and the cost to the respondent is provided in the exhibit below. It should be noted that while this is based on an estimated hourly rate for participants, participants will be asked to respond to the electronic survey on their personal time and will not subtract from their service or other employment time.

Estimated Response Burden for Data Collection Activities
Data Collection Activity	Number of Respondents	Average Burden Per Respondent	Total Burden Hours	Hourly Rate	Total Cost
Survey of CNCS service participants	4,000	20 minutes	1,333	21.79	$29,046
Totals	4,000		1,333		$29,046

For this data collection, the only cost is that for the time of the respondent (i.e., average of 20 minutes per respondent).

14. Estimated Annualized Cost to Federal Government

The total 1.5-year contract cost with ICF International is $376,119.97, which includes two data collections: focus group data collection and survey data collection. The survey data collection accounts for roughly half, or $188,060, of the full contract, in addition to the $26,148 calculated in A13 above.

15. Reason for Change in Burden

There are no changes. The survey discussed in this supporting statement comprises a new, one-time data collection.

16. Publication

At the conclusion of this project, frequency counts of responses will be calculated and a written final report will be provided to CNCS. Quantitative data will be analyzed using such statistical procedures as frequency distributions, correlations, and regression analyses. Survey administration will begin once OMB approval is received. This project and the associated final report are scheduled to be completed by March 22, 2013.

17. Request not to Display OMB Control Number

The OMB approval number and expiration date will be displayed on the survey that is completed by participants for this study.

18. Exceptions to Certification Statement

The CNCS office does not request an exception to the certification of this information collection.

Part B. Collections of Information Employing Statistical Methods:

1. Respondent Universe and Sampling Methods

The potential respondent universe for this data collection includes all current CNCS service participants as well as recent alumni who have been out of their program for 1 – 2 years. This population of both current and recent alumni members numbers approximately 200,000 individuals. Approximately 4,000 service participants and recent alumni will be identified using stratified sampling. Seventy percent of the survey respondents will be from AmeriCorps and the remainder from Senior Corps. Therefore, these same ratios will be used to determine which members and volunteers will be contacted for participation. For AmeriCorps, 30% of the contacted potential participants will be from each of the AmeriCorps programs – State & National, VISTA, and NCCC. Each of these groups will be stratified by current versus alumni members, with half of the contacted individuals currently serving and half of the individuals being recent alumni. In addition, 10% of the contacted potential participants, or 400 active members, will be from the NCCC program FEMA Corps. For Senior Corps, 30% of the contacted potential participants will be from each of the Senior Corps programs – RSVP, Senior Companions, and Foster Grandparents. All Senior Corps participants contacted will be current members.

Following this reasoning, the stratified sampling plan includes ten groups, or strata: current members in the four AmeriCorps programs, recent alumni from three AmeriCorps programs, and current volunteers in the three Senior Corps programs. Within each stratum, potential participants will be randomly selected to receive the survey instrument. We believe the results from the sample of approximately 4,000 will allow us to examine results at the group level as well as to generalize to the full population (i.e., current AmeriCorps/Senior Corps participants). This number of participants will allow for representation from different programs and members in the survey responses to promote generalizations of the findings to the whole of CNCS members and volunteers and each of the main CNCS programs.

Four hundred participants are desired in each group, or stratum, to allow for analysis of results at the group level. Four hundred respondents per program was identified using both power analysis and input from CNCS research and evaluation staff. Power analysis was conducted for the various statistical tests that will be conducted with the data (e.g., correlation, multiple regression), using an alpha of .05 and a power of .80. Power analysis for a small effect size resulted in a range of 485-620 suggested participants per group. Power analysis for a medium effect size resulted in a range of 64-102 participants per group. Based on the expectation of small to medium effect sizes and input from the CNCS Strategy Office, it was determined that approximately 400 respondents would be appropriate to detect these effects.

2. Procedures for the Collection of Information

Our recruitment effort for AmeriCorps and Senior Corps varies slightly. To recruit AmeriCorps members for potential participation in the survey, participants (both current members and alumni) will be recruited directly via email with the survey URL. For Senior Corps volunteers, participant contact information (including name and email address) is not known.  Therefore, Senior Corps project directors will be contacted via email to help in the recruitment of Senior Corps volunteers.

To help increase response rates, we will take a proactive approach to recruitment. Prior to contacting AmeriCorps members and Senior Corps project directors for potential participation in the survey, multiple stakeholders (e.g., CNCS Program staff, State Commissions, State Program Specialists, Project Directors, etc.) will be contacted to provide them with information regarding the survey and request their support in promoting the survey to potential survey participants. Contacting these individuals is important because they may be approached with questions regarding the survey from potential participants and it will be ideal for them to be aware of the study and support it. Additionally, these individuals, who are in closer contact with potential participants, can promote the survey on a more personal level than can be done by ICF or CNCS at the national level. Contacting these individuals will help to promote the survey by word-of-mouth as multiple levels will be informed of the study.

All individuals invited to take the survey will be presented the survey URL. The survey will be administered in an online format using Vovici EFM Community Web-based software. All data exported from the Vovici Community secure Web site will be kept in a secured folder.

If the respondent does not have access to the Internet, participants can request a paper-based survey mailed to them. A self-addressed stamped envelope for easy return of the survey directly to ICF International will be included with each survey mailed. This will help to maintain confidentiality of participants’ data as their Project Directors and CNCS will not have access to completed paper surveys.

Throughout this process, we will follow a detailed communication plan to ensure that all relevant individuals and groups are aware of the survey and its importance. Appropriate staff will be proactively engaged so that they are knowledgeable of the study and its value to CNCS and the programs, so that they can support participation in the survey.

ICF International will prepare a report that includes only aggregate-level results of the survey. Providing only aggregate-level data protects the confidentiality of individual data as responses will not be seen individually or in a way that individual respondents can be identified.

3. Methods to Maximize Response Rates and Deal with Issues of Non-Response

We expect the active member response rate to exceed 50% because this population typically feels a strong connection to their service sites and to the AmeriCorps program in which they serve.  The alumni rate may be a bit lower than the active member rate and is estimated to be less than 50%.  Senior Corps volunteers contact information (including name and email address) is not known at this time.  Consequently, Senior Corps project directors will be contacted via email to help in the recruitment of Senior Corps volunteers.  These directors suggest that a moderate to strong response rate will be achievable because project site staff are supportive of the evaluation and are eager to see and learn from the results of the study.

The following strategies will be employed to maximize response rate and minimize issues associated with non-response during the survey process:

• Communication with individuals who can support the survey to AmeriCorps members and Senior Corps volunteers;

• Utilizing word-of-mouth promotion of the survey within CNCS and its programs;

• Use of a well-designed, easy to use survey;

• Assuring respondents that their responses will be handled in a confidential manner;

• Providing respondents with a contact name and telephone number for inquiries;

• Providing respondents multiple reminders to complete the survey. The link to the web-based survey will be provided each time a reminder is needed;

• Providing respondents with an explanation of how their participation will help to inform positive changes to the service experience by CNCS and subsequently benefit current and future participants as well as the community they serve; and

• Offering the survey in paper-based form to be available for participants who do not have access or feel comfortable using the Web-based survey.

Employing these strategies will be helpful in improving the response rate across respondents.

4. Tests of Procedures of Methods to be Undertaken

The survey has been reviewed with five individuals. The purpose of this review was to determine the time required to complete the survey, the ease of responding to questions, and to ensure there was not duplicate information requested from participants. Suggested changes were incorporated into the survey instrument.

5. Individuals Consulted on Statistical Aspects of the Design and Organizations/Persons Collecting and Analyzing the Data

Beth Heinen, Ph.D.

Manager

Applied Organizational Research

ICF International

(314) 918-0373

8