ETAP PI User Satisfaction Survey

Principal investigator feedback interview protocol

94

Principal Investigator interview protocol on pilot experiences
Note: This semi-structure interview protocol is intended to gather feedback from Principal Investigators
(PIs) who used the NSF’s Education and Training Application (ETAP) and volunteered for a debriefing call
through the Principal Investigator feedback survey (question 11).
Before the call: Read respondent answers in the P I feedback survey and have survey responses available
during the call to follow up on issues raised or seek clarifications. Review the PI’s awards data available in
the Tableau Dashboard and think about whether there is anything in the award data that are unexpected.
For example, if their award does not have any participants reported as of the day of the interview, you
may ask whether there was something about the system that made it difficult to report participants. If the
award seems to have a larger-than-expected number of NSF-funded participants reported in the system,
you may confirm whether all participants reported were funded with the award participating in the ETAP
pilot (and not other awards, from NSF or a different source).
Introduction
Thank you so much for taking the time to talk to me today. We appreciate your participation on the ETAP
pilot and your willingness to share your experiences. We have read your thoughtful feedback from the
survey and we wanted to take this opportunity to talk through some of the points you raised in the
survey, learn how you interacted with the ETAP interface, or answer any question you might have.
Opening question
1.

Why don’t we start with your overall impression of the system – what you liked and didn’t like?
Suggested probes (use as needed)
•
•

You mentioned in the survey that… can you elaborate on that?
What type of system functionality you would have found helpful to have?

Additional questions
Let me take this opportunity to better understand how you managed applications to your award using
ETAP.
2.

Can you describe how you used ETAP to review applications?
Suggested probes (use as needed)
•
•
•
•
•

Did you download the applicant list as a csv file and assign applicants to various reviewers?
Did you pre-screen candidates before reviewing? How? (Interviewer: We want to understand
how the system was used, say data sorted, to screen applicants.)
Did you review applications on a rolling basis?
Did you save or print application PDFs?
Did you enter admissions at once or iteratively?

[If respondent identifies a system limitation or inefficient process:] Can you think of a system revision that
would make this process easier for you? [If you can think of a possible solution:] Would you have found it
helpful to have [explain a system functionality that might potentially help]? Why or why not?
95

3.
Have you received any questions or feedback from students about ETAP? Was there anything
students found confusing when applying to your project/Site?
Wrapping up
4.
5.

Is there anything else you would like to share with us today?
Do you have any questions for us or that you would like us to share with NSF?

Thank you!

96