NASA Institutional Research Capacity Evaluation Survey and Interview Protocol

EPSCoR Institutional Research Capacity (IRC) Outcome Assessment

Cognitive Interview Protocol

The purpose of this interview is to better understand how NASA awardees answer questions on a survey related to NASA EPSCoR-funded activities. We want to make sure that we are asking things in the right way. We will be reviewing survey questions, and I will be asking you about the questions and your responses. We would like to understand what people are considering when they answer each question and why they make particular choices. We are also looking for your input on how to make the survey better. There are no right or wrong answers. Please tell me at any time if what I am asking is unclear. The results of this survey will help to provide regular data to understand the NASA EPSCoR-funded activities better. 

We expect the interview to last approximately 40 minutes. Your participation is voluntary and you may choose to stop participating at any time. If it is okay with you, I will be recording the interview to ensure that I record your responses accurately and to minimize time taking notes. Only the staff assigned to this project will hear the recordings, which will be destroyed at the completion of the study.  

Do you agree to participate in this interview?  

Yes No (If No, thank them and excuse them from the interview.) 

Is it okay to record this interview?  

Yes (turn on recorder) No (If No, let them know you will just be taking notes.) 

Do you have any questions before we begin? Now let's get started. 

1. How did you complete the survey?

   1. Did you use a computer, mobile device, etc.?

   2. Did you have any accessibility issues?

2. What did you use to support you in responding to the survey questions?

   1. Did you use any references, such as the RPPR?

3. About how much time did the survey take for you to complete?

4. Were there any questions that stood out to you?

   1. Why? Can you elaborate on this?

5. In the introduction, we provide four components of institutional research capacity. What does institutional research capacity mean to you?

   1. What does institutional research capacity mean for your institution?

6. Do our four categories (show visual from survey) align with your perspectives on the institutional research capacity components?

   1. As you were completing the survey were you wondering about additional components that we did not ask about on the survey?

      1. Visual from survey:

7. At the end of each section, we asked you a question about how the metrics in that section related to the component (show an example question from survey).

   1. Survey Question: Given this list, have we missed any other metrics related to student engagement and support efforts to show your impact or progress in building institutional research capacity? If easily accessible, please provide a number for each.

   1. How did you understand this question as you completed the survey?

   2. Did you feel like you were able to express your ideas on each component and add additional metrics when you got to this question in each section? Why?

8. Tell us about your experience answering this question (show the question)? Can you elaborate on your reasoning and thinking behind your responses?

   1. Survey Question: Move the slider left or right to show your perception of the usefulness of each component for measuring your institutions’ institutional research capacity at this time.

9. Are there any questions we didn't ask that we should have? 

   1. If so, what should we have asked? 

10. Is there anything else you would suggest we change to make the survey easier to take? 

11. Can you reflect on the overall process of completing the institutional research capacity outcome assessment survey and interview?