Operation AmeriCorps National Evaluation
Spring Interviews- Protocol and Questions
Document 1: “Backbone” organization
Respondent Name:
OPAC grantee:
Interviewer Names:
Date:
Duration of call:
Follow up items:
Hi [respondent name], this is [interviewer name/s] calling from the Corporation for National and Community Service, how are you?
Is this still a good time for you to talk about [insert topic]? [If no, reschedule a time on the phone]. [If yes:] Great!
Let’s cover a few quick points before we get started with some questions. First, we want to thank you for taking the time to speak with us today about your Operation AmeriCorps project; our discussion today is going to help get a better sense of how things are unfolding for our grantees as they move into the latter halves of the program year. Second, we want to remind you that your responses to the questions we’ll ask you today are confidential, and are only going to be used by members of the evaluation team for the purpose of the national evaluation of Operation AmeriCorps. Your data is being protected and you won’t be personally identified or linked to your responses. Notes from the interview today will be stored electronically on a secure server, and at the end of the evaluation, we will destroy these records. Relatedly, we’ll be recording today’s interview to make sure we don’t miss anything important. The recordings will be transcribed; you will not be identified, nor will your responses be directly linked to your name. The audio files from your interview will be stored on a secure server, and only researchers from our evaluation team will have access to these. We will keep the files on the server until the evaluation is completed, and at that point they’ll be destroyed. Do we have your permission to record the interview? Finally, we want you to know that you can be as honest or blunt as you want; we really want to make sure that we’re capturing the realities of running an Operation AmeriCorps program, so there is no need to hold back on your answers and we encourage you to be candid. Do you agree to participate today?
Ok, the way this will work is that we have a short list of questions, that cover [insert topics: capacity your program is building; the different service streams your program is using, and how they’re interacting; early evidence of outcomes; and what you think about implementation of the Operation AmeriCorps grant]. [Interviewer may note that the question topics were pre-sent]. Also, please feel free at the end to add anything you think we missed that you think is important.
Q1: Interaction of Service Streams Questions:
We’d like to start out by hearing about what your program has been up to for the past few months.
Can you catch us up on what is currently going on in the project?
Have you made any mid-year changes to the project’s objectives?
Do you think the mission or goals of the project have changed? If so, how and why? Do you think they will need to change in the future?
What about the logic model, and theory of change? [Interviewers: a theory of change is a program’s statement of the problem it is addressing, the outcomes it hopes to achieve, and how it proposes to achieve them; a logic model is a visual representation of a program’s theory of change.]
Let’s talk a little more specifically about member activities for each service stream. [Interviewers: you may need to note which service streams they’re using in case you need to prompt them].
Can you describe what members from each service stream are doing as part of your Operation AmeriCorps project? [Interviewers: if respondent does not understand “service stream”, prompt with “type of AmeriCorps program”, or list the service streams you noted at the beginning of the interview].
How are members tracking and reporting their activities?
Can you tell us about how members are being managed and supervised?
Have you had to change any of the roles, responsibilities, or activities of members since you were planning the project or since it started in the fall?
If so, why?
Tell us about how national service members and volunteers interact in their work.
Q2: Community Capacity Questions:
As you’re aware, one of the main objectives of this grant is to build community capacity. You might be familiar with the term “community capacity”, which refers to the capabilities or competencies a group or network of organizations has to address a community problem. We want to talk a little about the community capacity you and your partners are building to successfully implement this project, achieve your goals, and sustain this work into the future.
Can you describe how your program’s partnership structure has evolved in the last several months?
Have these changes been made formally or informally?
Have the roles and responsibilities of each player changed?
What additional resources have you been able to recruit to the program? Were there any you were planning on that didn’t materialize?
Can you describe for me the conversations you and your partners have had about sustaining this program long-term?
[Interviewers: if approaching the 60 minute mark of the interview and you do not believe you will be able to finish, set up an additional interview to finish Q3 and Q4.]
Q3: Early Evidence of Outcomes:
We’re going to move on to early evidence for outcomes. We realize that many Operation AmeriCorps programs are new, and that one year is really not enough time to see a lot of progress on the outcomes you’re working towards. Regardless, we wanted to ask you about what kind of preliminary progress you’re seeing towards achieving your program’s outcomes.
What kind of early progress are you seeing towards achieving outcomes?
Is there anything surprising or interesting you are seeing as the program works towards preliminary outcomes?
What kinds of challenges or barriers have you had to overcome?
[If the respondent mentions an unexpected lack of progress] How have you dealt with unexpected lack of progress?
Q4: Grant Administration and Implementation:
Now we want to completely switch subjects and talk about the grant administration and implementation process. The Operation AmeriCorps grants were set up in part to reduce the burden on grantees who want to apply for members from multiple streams of national service, which normally would require coordinating multiple grants and their requirements.
Can you start by telling us about how you’ve been handling grant administration and implementation?
Can you tell us about how the combination of multiple programs into one grant has affected how you administer or implement the grant?
What has communication been like with CNCS?
[If respondent mentions particular challenges or successes, probe further]
Can you describe your approach to managing and reporting on funds?
What have you seen as challenges in managing this grant?
What do you see as benefits in managing this grant?
[After finishing question list or reaching end of scheduled time] Thank you so much for speaking with us, this has been really insightful. If you think of anything in the coming days that you want to tell us about, please feel free to email me or [note taker’s name] directly. Thanks so much for your time and your service!
Operation AmeriCorps National Evaluation
Spring Interviews- Protocol and Questions
Document 2: Legal applicant organization
Respondent Name:
OPAC grantee:
Interviewer Names:
Date:
Duration of call:
Follow up items:
Hi [respondent name], this is [interviewer name/s] calling from the Corporation for National and Community Service, how are you?
Is this still a good time for you to talk about [insert topic]? [If no, reschedule a time on the phone]. [If yes:] Great!
Let’s cover a few quick points before we get started with some questions. First, we want to thank you for taking the time to speak with us today about your Operation AmeriCorps project; our discussion today is going to help get a better sense of how things are unfolding for our grantees as they move into the latter halves of the program year. Second, we want to remind you that your responses to the questions we’ll ask you today are confidential, and are only going to be used by members of the evaluation team for the purpose of the national evaluation of Operation AmeriCorps. Your data is being protected and you won’t be personally identified or linked to your responses. Notes from the interview today will be stored electronically on a secure server, and at the end of the evaluation, we will destroy these records. Relatedly, we’ll be recording today’s interview to make sure we don’t miss anything important. The recordings will be transcribed; you will not be identified, nor will your responses be directly linked to your name. The audio files from your interview will be stored on a secure server, and only researchers from our evaluation team will have access to these. We will keep the files on the server until the evaluation is completed, and at that point they’ll be destroyed. Do we have your permission to record the interview? Finally, we want you to know that you can be as honest or blunt as you want; we really want to make sure that we’re capturing the realities of running an Operation AmeriCorps program, so there is no need to hold back on your answers and we encourage you to be candid. Do you agree to participate today?
Ok, the way this will work is that we have a short list of questions, that cover [insert topics: capacity your program is building; the different service streams your program is using, and how they’re interacting; and what you think about implementation of the Operation AmeriCorps grant]. Also, please feel free at the end to add anything you think we missed that you think is important.
[Interviewers: the following questions should only be asked of organizations that are the legal applicant/sub-applicant but that are NOT also the backbone organization.]
Q1: Interaction of Service Streams Questions:
We’d like to start out by hearing about what your organization has been up to for the past few months.
Can you catch us up on what is currently going on in the project?
Have you made any mid-year changes to the project’s objectives?
Do you think the mission or goals of the project have changed? If so, how and why? Do you think they will need to change in the future?
What about the logic model, and theory of change? [Interviewers: if the respondent is confused about theories of change and logic models, you can use the following definitions: a theory of change is a program’s statement of the problem it is addressing, the outcomes it hopes to achieve, and how it proposes to achieve them; a logic model is a visual representation of a program’s theory of change.]
Let’s talk a little more specifically about some of the activities taking place. [Interviewers: you may need to note which service streams they’re using in case you need to prompt them].
Are you working directly with members?
[If yes] Can you describe what members from each service stream are doing as part of your Operation AmeriCorps project? [Interviewers: if respondent does not understand “service stream”, prompt with “type of AmeriCorps program”, or list the service streams you noted at the beginning of the interview].
How are members tracking and reporting their activities?
Can you tell us about how members are being managed and supervised?
Have you had to change any of the roles, responsibilities, or activities of members since you were planning the project or since it started in the fall?
If so, why?
Tell us about how national service members and volunteers interact in their work.
[If no] What is your primary role or responsibility when it comes to the project’s activities?
Has this changed since the fall?
If so, why?
Q2: Community Capacity Questions:
As you’re aware, one of the main objectives of this grant is to build community capacity. You might be familiar with the term “community capacity”, which refers to the capabilities or competencies a group or network of organizations has to address a community problem. We want to talk a little about the community capacity you and your partners are building to successfully implement this project, achieve your goals, and sustain this work into the future.
Can you describe how your program’s partnership structure has evolved in the last several months?
Have these changes been made formally or informally?
Have the roles and responsibilities of each player changed?
What additional resources have you been able to recruit to the program? Were there any you were planning on that didn’t materialize?
Can you describe for me the conversations you and your partners have had about sustaining this program long-term?
Q3: Early Evidence of Outcomes:
We’re going to move on to early evidence for outcomes. We realize that many Operation AmeriCorps programs are new, and that one year is really not enough time to see a lot of progress on the outcomes you’re working towards. Regardless, we wanted to ask you about what kind of preliminary progress you’re seeing towards achieving your program’s outcomes.
What kind of early progress are you seeing towards achieving outcomes?
Is there anything surprising or interesting you are seeing as the program works towards preliminary outcomes?
What kinds of challenges or barriers have you had to overcome?
[If the respondent mentions an unexpected lack of progress] How have you dealt with unexpected lack of progress?
Q4: Grant Administration and Implementation:
Now we want to completely switch subjects and talk about the grant administration and implementation process.
Can you start by telling us about how you’ve been handling grant administration and implementation?
Have you had an AmeriCorps grant in the past?
[If yes] How similar is the administration and implementation process to what you’re used to? How is it different?
Have you had a grant similar to the Operation AmeriCorps grant before? [Interviewers: some examples might include place-based initiative grants, community change grants, etc.]
The Operation AmeriCorps grants were set up in part to reduce the burden on grantees who want to apply for members from multiple streams of national service, which normally would require coordinating multiple grants and their requirements.
Can you tell us about how the combination of multiple programs into one grant has affected how you administer or implement the grant?
What has communication been like with CNCS?
[If respondent mentions particular challenges or successes, probe further]
Can you describe your approach to managing and reporting on funds?
What have you seen as challenges in managing this grant?
What do you see as benefits in managing this grant?
[After finishing question list or reaching end of scheduled time] Thank you so much for speaking with us, this has been really insightful. If you think of anything in the coming days that you want to tell us about, please feel free to email me or [note taker’s name] directly. Thanks so much for your time and your service!]
Operation AmeriCorps National Evaluation
Spring Interviews- Protocol and Questions
Document 3: Select Operating Partners
Respondent Name:
OPAC grantee:
Interviewer Names:
Date:
Duration of call:
Follow up items:
Hi [respondent name], this is [interviewer name/s] calling from the Corporation for National and Community Service, how are you?
Is this still a good time for you to talk about [insert topic]? [If no, reschedule a time on the phone]. [If yes:] Great!
Let’s cover a few quick points before we get started with some questions. First, we want to thank you for taking the time to speak with us today about your Operation AmeriCorps project; our discussion today is going to help get a better sense of how things are unfolding for our grantees as they move into the latter halves of the program year. Second, we want to remind you that your responses to the questions we’ll ask you today are confidential, and are only going to be used by members of the evaluation team for the purpose of the national evaluation of Operation AmeriCorps. Your data is being protected and you won’t be personally identified or linked to your responses. Notes from the interview today will be stored electronically on a secure server, and at the end of the evaluation, we will destroy these records. Relatedly, we’ll be recording today’s interview to make sure we don’t miss anything important. The recordings will be transcribed; you will not be identified, nor will your responses be directly linked to your name. The audio files from your interview will be stored on a secure server, and only researchers from our evaluation team will have access to these. We will keep the files on the server until the evaluation is completed, and at that point they’ll be destroyed. Do we have your permission to record the interview? Finally, we want you to know that you can be as honest or blunt as you want; we really want to make sure that we’re capturing the realities of running an Operation AmeriCorps program, so there is no need to hold back on your answers and we encourage you to be candid. Do you agree to participate today?
Ok, the way this will work is that we have a short list of questions, that cover [insert topics: capacity your program is building; the different service streams your program is using, and how they’re interacting; and what you think about implementation of the Operation AmeriCorps grant]. Also, please feel free at the end to add anything you think we missed that you think is important.
Q1: Interaction of Service Streams Questions:
Can you start by telling us about your organization’s role in the Operation AmeriCorps project?
Do you work with members?
If yes: Can you describe what members are currently doing on the project?
How are you supervising and managing members?
How are the different streams of member activities complementing one another?
Have there been challenges to coordinating the members’ activities?
How are you working to overcome these challenges?
How has the implementation of member activities differed from what you planned?
How have you been tracking member activities?
If no: does your organization interact with members at all?
Are you in charge of providing financial resources?
If yes: can you describe these resources and their role in the project? [Interviewers: probe for details if they are comfortable sharing, such as type, amount, and source]
Are you in charge of providing non-financial resources?
If yes: can you describe these resources and their role in the project? [Interviewers: probe for details if they are comfortable sharing, such as type, amount, and source]
Q2: Community Capacity Questions:
As you’re aware, one of the main objectives of this grant is to build community capacity. You might be familiar with the term “community capacity”, which refers to the capabilities or competencies a group or network of organizations has to address a community problem. We want to talk a little about the community capacity you and your partners are building to successfully implement this project, achieve your goals, and sustain this work into the future.
Can you describe the program’s partnership structure?
Who are some of your key partners?
What types of partners do you believe add significant value to the project? Do you feel the project lacks any of these?
Do you believe the partnership structure has evolved in the last several months?
Have these changes been made formally or informally?
Has your organization’s role or responsibilities changed?
Do you think the mission or goals of the project have changed?
Have you made any mid-year changes to your organization’s objectives in the Operation AmeriCorps project? If so, please describe these changes.
Can you describe for me the conversations you and your partners have had about sustaining this program long-term?
Q3: Preliminary Evidence of Effectiveness:
We’d now like to ask you about your perception of preliminary progress towards achieving the project’s outcomes.
What kind of early progress are you seeing towards achieving outcomes?
Is there anything surprising or interesting you are seeing as the program works towards preliminary outcomes?
What kinds of challenges or barriers have you had to overcome?
[If the respondent mentions an unexpected lack of progress] How have you dealt with unexpected lack of progress?
Q4: Grant Administration and Implementation:
Now we want to switch subjects and talk about the grant administration and implementation process.
Are you responsible for any part of the grant administration process?
[If yes]: How has your organization been involved in the administrative aspects of implementing the grant? This might include involvement in tracking and reporting on member activities, coordinating funding, or working directly with CNCS or other funders to ensure the grant is in compliance.
Does your role require you to be in direct contact with CNCS? If so, how have you found the process of communicating with CNCS for this grant?
[If respondent mentions particular challenges or successes, probe further]
[If respondent has indicated that they manage CNCS funds]: Can you describe your approach to managing and reporting on funds?
What have you seen as challenges in participating in this grant?
What do you see as benefits in participating in this grant?
[After finishing question list or reaching end of scheduled time] Thank you so much for speaking with us, this has been really insightful. If you think of anything in the coming days that you want to tell us about, please feel free to email me or [note taker’s name] directly. Thanks so much for your time and your service!]
Operation AmeriCorps National Evaluation
Spring Interviews- Protocol and Questions
Document 4: Select peripheral supporters
Respondent Name:
OPAC grantee:
Interviewer Names:
Date:
Duration of call:
Follow up items:
Hi [respondent name], this is [interviewer name/s] calling from the Corporation for National and Community Service, how are you?
Is this still a good time for you to talk about [insert topic]? [If no, reschedule a time on the phone]. [If yes:] Great!
Let’s cover a few quick points before we get started with some questions. First, we want to thank you for taking the time to speak with us today about your Operation AmeriCorps project; our discussion today is going to help get a better sense of how things are unfolding for our grantees as they move into the latter halves of the program year. Second, we want to remind you that your responses to the questions we’ll ask you today are confidential, and are only going to be used by members of the evaluation team for the purpose of the national evaluation of Operation AmeriCorps. Your data is being protected and you won’t be personally identified or linked to your responses. Notes from the interview today will be stored electronically on a secure server, and at the end of the evaluation, we will destroy these records. Relatedly, we’ll be recording today’s interview to make sure we don’t miss anything important. The recordings will be transcribed; you will not be identified, nor will your responses be directly linked to your name. The audio files from your interview will be stored on a secure server, and only researchers from our evaluation team will have access to these. We will keep the files on the server until the evaluation is completed, and at that point they’ll be destroyed. Do we have your permission to record the interview? Finally, we want you to know that you can be as honest or blunt as you want; we really want to make sure that we’re capturing the realities of running an Operation AmeriCorps program, so there is no need to hold back on your answers and we encourage you to be candid. Do you agree to participate today?
Ok, the way this will work is that we have a short list of questions, that cover [insert topics: capacity your program is building; the different service streams your program is using, and how they’re interacting; and what you think about implementation of the Operation AmeriCorps grant]. Also, please feel free at the end to add anything you think we missed that you think is important.
Q1: Interaction of Service Streams Questions:
Can you start by telling us about your organization’s role in the Operation AmeriCorps project?
Do you work with members?
If yes: Can you describe what members are currently doing on the project?
How are you supervising and managing members?
How are the different streams of member activities complementing one another?
Have there been challenges to coordinating the members’ activities?
How are you working to overcome these challenges?
How has the implementation of member activities differed from what you planned?
How have you been tracking member activities?
If no: does your organization interact with members at all?
Are you in charge of providing financial resources?
If yes: can you describe these resources and their role in the project? [Interviewers: probe for details if they are comfortable sharing, such as type, amount, and source]
Are you in charge of providing non-financial resources?
If yes: can you describe these resources and their role in the project? [Interviewers: probe for details if they are comfortable sharing, such as type, amount, and source]
Q2: Community Capacity Questions:
As you’re aware, one of the main objectives of this grant is to build community capacity. You might be familiar with the term “community capacity”, which refers to the capabilities or competencies a group or network of organizations has to address a community problem. We want to talk a little about the community capacity you and your partners are building to successfully implement this project, achieve your goals, and sustain this work into the future.
Can you describe the project’s partnership structure?
Who are some of your key partners?
What types of partners do you believe add significant value to the project? Do you feel the project lacks any of these?
Do you believe the partnership structure has evolved in the last several months?
Have these changes been made formally or informally?
Has your organization’s role or responsibilities changed?
Do you think the mission or goals of the project have changed?
Have you made any mid-year changes to your organization’s objectives in the Operation AmeriCorps project? If so, please describe these changes.
Can you describe for me the conversations you have had with the program about long-term sustainability?
Q3: Preliminary Evidence of Effectiveness:
We’d now like to ask you about your perception of preliminary progress towards achieving the project’s outcomes.
What kind of early progress are you seeing towards achieving outcomes?
Is there anything surprising or interesting you are seeing as the program works towards preliminary outcomes?
What kinds of challenges or barriers have you had to overcome?
[If the respondent mentions an unexpected lack of progress] How have you dealt with unexpected lack of progress?
Q4: Grant Administration and Implementation:
Are you responsible for any part of the grant administration process?
[If yes] Can you describe your role and responsibilities?
[After finishing question list or reaching end of scheduled time] Thank you so much for speaking with us, this has been really insightful. If you think of anything in the coming days that you want to tell us about, please feel free to email me or [note taker’s name] directly. Thanks so much for your time and your service!]
Operation AmeriCorps National Evaluation
Spring Interviews- Protocol and Questions
Document 5: Select supervisors of national service members, volunteers
Respondent Name:
OPAC grantee:
Interviewer Names:
Date:
Duration of call:
Follow up items:
Hi [respondent name], this is [interviewer name/s] calling from the Corporation for National and Community Service, how are you?
Is this still a good time for you to talk about [insert topic]? [If no, reschedule a time on the phone]. [If yes:] Great!
Let’s cover a few quick points before we get started with some questions. First, we want to thank you for taking the time to speak with us today about your Operation AmeriCorps project; our discussion today is going to help get a better sense of how things are unfolding for our grantees as they move into the latter halves of the program year. Second, we want to remind you that your responses to the questions we’ll ask you today are confidential, and are only going to be used by members of the evaluation team for the purpose of the national evaluation of Operation AmeriCorps. Your data is being protected and you won’t be personally identified or linked to your responses. Notes from the interview today will be stored electronically on a secure server, and at the end of the evaluation, we will destroy these records. Relatedly, we’ll be recording today’s interview to make sure we don’t miss anything important. The recordings will be transcribed; you will not be identified, nor will your responses be directly linked to your name. The audio files from your interview will be stored on a secure server, and only researchers from our evaluation team will have access to these. We will keep the files on the server until the evaluation is completed, and at that point they’ll be destroyed. Do we have your permission to record the interview? Finally, we want you to know that you can be as honest or blunt as you want; we really want to make sure that we’re capturing the realities of running an Operation AmeriCorps program, so there is no need to hold back on your answers and we encourage you to be candid. Do you agree to participate today?
Ok, the way this will work is that we have a short list of questions, that cover [insert topics: capacity your program is building; the different service streams your program is using, and how they’re interacting; and what you think about implementation of the Operation AmeriCorps grant]. Also, please feel free at the end to add anything you think we missed that you think is important.
Q1: Interaction of Service Streams Questions: What partners do you interact with? Describe your role.
Can you start by telling us about your role in the Operation AmeriCorps project?
What are your day to day responsibilities?
Are you in charge of managing any financial resources?
If yes: can you describe your responsibilities and tasks?
Are you in charge of managing any non-financial resources?
If yes: can you describe your responsibilities and tasks?
Have there been challenges to coordinating the members’ activities?
How are you working to overcome these challenges?
How has the implementation of member activities differed from what you planned?
How have you been tracking member activities?
Do you believe the different streams of member activities are complementing one another? If so, how?
Q2: Community Capacity Questions:
As you’re aware, one of the main objectives of this grant is to build community capacity. You might be familiar with the term “community capacity”, which refers to the capabilities or competencies a group or network of organizations has to address a community problem. We want to talk a little about the community capacity you and your partners are building to successfully implement this project, achieve your goals, and sustain this work into the future.
What is your understanding of the goals of this project?
Who are some of your key partners in your day to day work?
Q3: Preliminary Evidence of Effectiveness:
We’d now like to ask you about your perception of preliminary progress towards achieving the project’s outcomes.
What kind of early progress are you seeing towards achieving outcomes?
Is there anything surprising or interesting you are seeing as the program works towards preliminary outcomes?
What kinds of challenges or barriers have you had to overcome?
[If the respondent mentions an unexpected lack of progress] How have you dealt with unexpected lack of progress?
Q4: Grant Administration and Implementation:
Are you responsible for any part of the grant administration process?
[After finishing question list or reaching end of scheduled time] Thank you so much for speaking with us, this has been really insightful. If you think of anything in the coming days that you want to tell us about, please feel free to email me or [note taker’s name] directly. Thanks so much for your time and your service!]
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | DiTommaso, Adrienne |
| File Modified | 0000-00-00 |
| File Created | 2021-01-24 |