NPRA Non-Respondent Debriefing Interviews

ii: Debriefing Protocol for Non-respondents

Debriefing Protocol for Non-respondents
Organization Name: ______________________________
Organization ID: __________________
Performer:
Funder:

 Yes
 Yes

 No
 No

 Don’t Know
 Don’t Know

Non-respondent (never logged in)

 Yes  No

Logged in, non-respondent

 Yes  No

Answered Yes to Q 7

 Yes  No

Answered Yes to Q 8

 Yes  No

Partial completer with response only to Q7 and/or Q8

 Yes  No

If yes, by:
Screener Card

 Yes  No

Phone

 Yes  No

E-mail

 Yes  No

Online

 Yes  No

Paper survey

 Yes  No

Approximately 15 minutes before the interview, send the interviewee a copy of the
Prenotification Letter and Screener Card we sent them.
Thank you for agreeing to participate in this short call about the National Science Foundation’s
FY 2016 Survey of Nonprofit [Hospital] Research Activities. The information you provide will
help us to better understand why some nonprofit organizations did not complete the survey and
how we can make the survey more appealing to these organizations.
Before we begin our interview, I want to review some information with you.







Your participation in this interview is voluntary.
The interview should last no more than 20 to 30 minutes.
There is another person on the phone with me who is taking notes. Her name is .
If appropriate: , from NSF is also listening to our discussion today. Is that
okay? If not or the answer is no, NSF staff member will hang up the phone and end
his/her join.me session.
I would like to audio record this interview so that I get all of your comments. Is that okay
with you?

Page |Cii-1



Before we continue with the interview, I need your consent to be interviewed. I’m going
to read a short consent form to you, then ask for your verbal consent.

The National Science Foundation designs, conducts, and sponsors surveys on science,
engineering, and technology. In order to produce the best information possible, the National
Science Foundation routinely evaluates its surveys.
You have volunteered to take part in a study to improve one of the National Science
Foundation’s surveys. In order to have a complete record of your comments, your interview
session will be audio recorded. We plan to use the recording to improve the survey. NSF and
ICF staff directly involved in this research project will have access to the recording.
This study is authorized by law (42 U.S.C. 1862 Section 3.a.6). The OMB control number for
this study is XXXX-XXXX.
I have volunteered to participate in this study, and I give permission for my recording to be
used for the purposes stated above.
If you consent to participate in this interview, please state your name and that you consent.
We noticed that although we invited your organization to participate, you [did not] OR [only told
us that you do not perform and/or fund research]. The following few questions ask about your
reasons for not responding and will help us be more precise in inviting nonprofit organizations to
participate in the full implementation of the survey later this year.
1. What was your initial reaction to NSF’s request for your organization to complete this
survey?
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
2. Skip Q2 if the respondent answered this in Q1.
Why did your organization decide not to complete the survey?
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________

Page |Cii-2

When I sent you the e-mail confirming our discussion today, I attached two documents: (1) the
letter that was sent to your organization and, (2) the screener card.
Depending on who is interviewed, ask:
3. Did you review the letter and screener card when it was initially sent to you? (This would
have been in January 2018 and it looks like the attachment you received.)
 Yes
 Don’t remember
 No, why not?
_____________________________________________________________________
_____________________________________________________________________
OR
Did forward the letter and handout to you?
 No
 Don’t remember
 Yes. If yes ask:
Did you remember reading the letter and handout?
 Yes
 Don’t remember
 No, why not?
_____________________________________________________________________
_____________________________________________________________________
4. Ask if the organization did not mail the screener to ICF. Otherwise, skip to Q4.
I noticed that you did not complete and return the screener card. Can you explain why? (Do
not read, but check as appropriate)
 Don’t remember seeing it
 Too busy/takes too much time to fill it out
 Easier to call/e-mail (Ask for more information about this) _____________________
_______________________________________________________________________
 Not a priority for our organization
 Was told not to respond (Ask that person’s position _______________________)
 Meant to, but forgot/got too busy
 Don’t know/don’t remember
 Other ________________________________________________________________
Page |Cii-3

5. Ask only if organization returned their screener to ICF.
How easy or difficult was it for you to complete and return the screener
card?
 Easy
 Neutral
 Difficult
6. Who made the decision not to complete the survey? Enter the name and title of the
person(s) who made the decision. Be sure to capture the person’s title. Only read the
position listed below if the interviewee has trouble remembering the position.
Name _____________________________ Title ______________________________
Possible options:
 Chief Executive Officer (CEO)/President/Executive Director/Chief Operating Officer
(COO)
 Chief Financial Officer (CFO)
 Board of Trustees
 Secretary/Admin Assistant/Executive Assistant
 Other, please specify ____________________
Instructions:
1. If organization is a non-responder (never logged in and did not answer any question)
ask Q7 and Q8.
2. If organization logged in, ask Q9-14. If organization answered Q6 and/or Q7 ask Q 13
and Q14, as appropriate.
3. If partial complete by phone, e-mail, or screener ask Q 15 and 16.
Ask non-responders Q 7 and Q8
7.

Does your organization perform any research? This includes research performed by your
organization’s employees or consultants. Research includes creative and systematic work to
increase the stock of knowledge, OR devise new applications of available knowledge
(including technology).
 Yes
 No

Page |Cii-4

8. Does your organization fund any research performed outside your organization? This
includes all grants, contracts, subcontracts, and sub-awards awarded by your organization to
external recipients to perform research. If necessary, read: Research includes creative and
systematic work to increase the stock of knowledge, OR devise new applications of available
knowledge (including technology).
 Yes
 No
Skip to Q17
Ask Questions 9-14 to participants who logged into the survey.
9. We see that someone in your organization logged into the survey. Were you the person who
logged in?
 Yes [continue]
 No [skip to Q13]
If you are not speaking to the person who logged onto the survey, skip to Q13.
10. Was there anything about the survey that influenced your decision not to respond?
 Yes (please explain)
__________________________________________________________________
__________________________________________________________________
 No
11. Did you have any problems accessing the web survey?
 Yes (please explain)
__________________________________________________________________
__________________________________________________________________
 No

Page |Cii-5

12. Do you have any problems navigating within the web survey?
 Yes (please explain)
__________________________________________________________________
__________________________________________________________________
 No
The following questions are ONLY for organization that logged-in and responded “Yes” to Q7
and/or Q8. Otherwise skip to Q17.
13. We noticed that you answered a question indicating that your organization performed
research activities in FY 2016. A definition of research was provided. Was that definition
helpful in responding to the question?
 Yes
 No
 Don’t remember
13a. Can you briefly describe some of the research projects you thought of when you
answered the question?
__________________________________________________________________
__________________________________________________________________

14. We noticed that you said your organization funded research performed outside your
organization in FY 2016. A definition of research was provided. Was that definition helpful
in responding to the question?
 Yes
 No
 Don’t remember
14a. Can you briefly describe some of the research projects you thought of when you
answered the question?
__________________________________________________________________
__________________________________________________________________

Page |Cii-6

Ask only if this is a partial complete by phone, e-mail, or screener card
15. In our telephone contact(s) with your organization, we asked if your organization performed
any research activities in FY 2016. Was the definition of research provided helpful or not
helpful in responding to the question?
 Helpful
 Not helpful
 (don’t read, but check if this response is provided) No definition was provided
Ask 15a. Ask only if organization answered “Yes” to Q7
15a. You told us that your organization performed some research activities in FY 2016. Can
you briefly describe some of the research projects you thought of when you answered the
question?
__________________________________________________________________
__________________________________________________________________
16. In our telephone contact(s) with your organization, we asked if your organization funded
any research performed outside your organization in FY 2016. Was the definition of research
provided helpful or not helpful in responding to the question?
 Helpful
 Not helpful
 (don’t read, but check if this response is provided) No definition was provided
Ask 16a only if organization answered “Yes” to Q8
16a. You told us that your organization funded some research performed outside your
organization in FY 2016. Can you briefly describe some of the research projects you
thought of when you answered the question?
__________________________________________________________________
__________________________________________________________________

ALL RESPONDENTS
17. What would increase your interest in completing this survey?
_____________________________________________________________________
_____________________________________________________________________

Page |Cii-7

18. Are there particular professional associations whose endorsements might have influenced
you to participate?
 No
 Yes. If yes, which ones?
_____________________________________________________________________
_____________________________________________________________________
19. Are there particular topics that you think should be covered by the survey?
 No
 Yes. If yes, which ones?
_____________________________________________________________________
_____________________________________________________________________
19a. If these topics were included, would this influence your decision to participate in the
survey?
 No
 Yes. If yes, how so?
_____________________________________________________________________
_____________________________________________________________________
As I mentioned when we first started talking today, your feedback will help us be more precise in
inviting nonprofit organizations to participate in the full implementation of the survey later this
year. I’m interested in your thoughts about who we should address our initial letter to – a letter
similar to the one we mailed to .
20. Do you think we should be mail the letter to the CEO/President/Executive Director of the
organization?
 No
 Yes
If NO, who would be a better person in the organization to mail the letter to? For example,
the CFO, Grants Manager, Research Director, or someone else?
________________________________________________

Page |Cii-8

20a. Do you recommend that we send the initial e-mail invitation to the same person?
If yes, Should we copy (cc) someone else in the organization?
 No
 Yes – if yes Who? _____________________
21. Before we conclude this interview, I’d like to ask you if there is anything else we haven’t
discussed that you would like to share?
_____________________________________________________________________
_____________________________________________________________________
_____________________________________________________________________
On behalf of the National Science Foundation, thank you for your time and assistance.

Page |Cii-9