Form CMS-10467 Attachment 2: Qualitative Data Collection Protocol

Evaluation of the Graduate Nurse Education Demonstration Program (CMS-10467)

Attach2QualData

Data Collection

OMB: 0938-1212

Document [pdf]

Download: pdf | pdf

EVALUATION OF THE GRADUATE NURSE
EDUCATION DEMONSTRATION PROGRAM
QUALITATIVE DATA COLLECTION PROTOCOL FOR:
TELEPHONIC SITE VISIT
TELEPHONE CHECK-IN CALLS
TELEPHONE FOLLOW-UP CALLS
Final – FOR INTERNAL REVIEW
Contract No. HHSM-500-2011-00013I
Task Order No. HHSM-500-T0009
07/15/2015

Table of Contents
I. Extension Year Telephone Interview Protocol ....................................................... 1
Graduate Nurse Education Evaluation Interview Guide: GNE Project strategic
planning team ................................................................................................. 2
Graduate Nurse Education Evaluation Interview Guide: SON administration .... 6
Graduate Nurse Education Evaluation ............................................................... 10
Focus Group Guide: Clinical faculty ................................................................... 10
Graduate Nurse Education Evaluation Focus Group Guide: APRN students ..... 14
Graduate Nurse Education Evaluation ............................................................... 17
Interview Guide: Clinical placement coordinators ............................................ 17
Graduate Nurse Education Evaluation ............................................................... 21
Interview Guide: Preceptors .............................................................................. 21
Graduate Nurse Education Evaluation Interview Guide: Director of Nursing or
clinical director ............................................................................................. 26
Graduate Nurse Education Evaluation Interview Guide: Chief financial officer 30
II. Extension Year Check-In Call Protocol ................................................................. 34
Graduate Nurse Education Evaluation Interview Guide: Interim Check- In ...... 35
III. Post-Demonstration - Telephone-Based Interviews with Limited Stakeholders ... 38
Graduate Nurse Education Evaluation Interview Guide: GNE Project strategic
planning team ............................................................................................... 39
Graduate Nurse Education Evaluation Interview Guide: SON administration .. 43

I. Extension Year Telephone Interview
Protocol

GRADUATE NURSE EDUCATION EVALUATION INTERVIEW GUIDE: GNE PROJECT
STRATEGIC PLANNING TEAM
Obtain Informed Consent
Have interviewee read and sign the informed consent form. Give interviewee an unsigned copy
form to keep for his or her records.
Outline with Timings
Time in each
section
(in minutes)

Topic

Elapsed time at end of
section
(in minutes)

Introduction

Changes to demonstration project

Outcomes, barriers, and facilitators

Perceived return on investment and sustainability
plans

Conclusion

Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized
to receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

ask whether you told me anything that you wouldn’t want associated with you. If so, we will
structure and present the information so it cannot be linked to you.
Do you have any questions for me before we
begin? Okay, let’s get started.

If New Interviewee: Interviewee Background
1. What is your title?
2. How long have you been with this organization?
3. What are your main responsibilities in your position?
4. What are your main responsibilities in relation to the GNE Demonstration Project (if
different or more specific)?
5. Are these responsibilities similar to what they were prior to the GNE Demonstration
Project? In what way?
6. What percentage of your FTE is dedicated to the GNE Demonstration Project?

Changes since the GNE Demonstration Project (8 minutes)
I’d like to begin by discussing any key changes since the last time we spoke.
1. Have there been any changes to the process for coordinating students’ clinical
placements?


If yes, tell me about those changes.



What prompted those changes?

2. Have there been any key staffing changes?



If yes, tell me about those changes.
[If same interviewee as prior year] Has your role changed since the last interview?


If yes, what is your role now?



How does it relate to the Demonstration?

3. Are there any organizations or health care agencies that are no longer participating in
since the demonstration project ended?


If yes, which organizations are no longer participating?



What made these organizations decide not to continue their participation?

GNE Evaluation Qualitative Data Collection Protocol
July 2015

4. Are there any new partners?


If yes, who are the new organizations?



How did these organizations become involved?

5. Are there any organizations that increased or decreased the number of preceptors?


If applicable, which organizations increased the number of preceptors?



If applicable, which organizations decreased the number of preceptors?

6. What, if any, changes to policy or procedure has your organization made to support
the demonstration project?


Does your SON accept students into an online training program?



If yes, do you permit online-enrolled students to participate in the GNE
demonstration? Describe the GNE precepting process for online-enrolled
students.

Outcomes, barriers, and facilitators (30 minutes)
7. What effects do you believe the demonstration has had on enrollment?
Graduation?
Do you think changes in enrollment and graduations are primarily because
of the demonstration, or are there other factors influencing these?
8. What, if any, outcomes do you feel you have achieved through the
demonstration project?




How do you think this program has benefited your organization?



What are the benefits of this project?



To what extent did the demonstration project result in the outcomes you desired?



If no outcomes or benefits were achieved, why do you think this project has not
benefited your organization?

9. What components of your demonstration project have allowed you to be successful in
achieving the goals you set out to accomplish?
10. What are the most difficult challenges you have faced in implementing the
demonstration project?


What made those challenges so difficult?



What components of the demonstration project have created challenges to achieving
the goals you set out to accomplish?

GNE Evaluation Qualitative Data Collection Protocol
July 2015

11. What are positive and/or negative effects of the GNE demonstration on nondemonstration SONs?
12. Have any aspects of the demonstration produced undesirable or negative outcomes?
Describe these aspects, the negative outcomes, and possible reasons/explanations for
the negative outcomes.
13. If you had to start over from the beginning, what would you do differently knowing what
you now know?

Perceived return on investment and sustainability plans (15 minutes)
14. To what extent do you think that the investments you have made to
support the demonstration project have paid off?


Has it been worth it?



What, if at all, do you see as the business case for expanding the number of APRN
clinical placement opportunities?

15. In what ways, if any, will you sustain the GNE Demonstration Project activities beyond
grant funding?


If sustaining activities, what resources or investments will be necessary to sustain
them?



Where will those resources or investments come from?



If not sustaining activities, what makes the project unsustainable?

16. If you could redesign the GNE Demonstration Project for the future, what changes would
you suggest to the Centers for Medicare and Medicaid Services?

Conclusion (2 minutes)
17. Describe the best outcome or aspect of the GNE Demonstration? The worst?
18. Is there anything that I did not ask about that you would like to share with me, or
do you have any additional thoughts about what we have discussed today?
19. Are there any areas covered by this interview that you are particularly concerned
about having your identity connected with? If so, which ones?

GNE Evaluation Qualitative Data Collection Protocol
July 2015

GRADUATE NURSE EDUCATION EVALUATION INTERVIEW GUIDE: SON
ADMINISTRATION
Obtain Informed Consent
Have interviewee read and sign the informed consent form. Give interviewee an unsigned copy of
the form to keep for his or her records.
Outline with Timings
Time in each
section
(in minutes)

Topic

Elapsed time at end of
section
(in minutes)

Introduction

Changes to the demonstration project

Outcomes, barriers, and facilitators

Perceived return on investment and sustainability
plans

Conclusion

Introduction (5 minutes)
The purpose of this interview is to discuss your ongoing activities related to the GNE
Demonstration Project, including challenges, successful strategies, perceived outcomes,
and sustainability plans.
This interview will last approximately 60 minutes.
As with our first site visit and with your approval, we will record audio of the discussion to assist
with note-taking. No one outside the evaluation team will have access to this recording.
This interview will work best if you do most of the talking. Feel free to speak openly
and candidly about your experiences and perspectives regarding this demonstration.
In the final report, the data gathered through these interviews will be presented in an
aggregate manner, highlighting informational points from specific sites and not from particular
individuals. You will not be identified by name. However, it is possible that someone could
attribute specific comments to you. Hence, at the end of the interview, I’ll ask whether you
told me anything that you wouldn’t want associated with you. If so, we will structure and
present the information so it cannot be linked to you.
Do you have any questions for me before we
begin? Okay, let’s get started.
GNE Evaluation Qualitative Data Collection Protocol
July 2015

Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

Changes to the Nursing program or GNE Demonstration Project (8 minutes)
I’d like to begin by discussing any key changes to the demonstration project:
1. [If same interviewee as prior year] Has your role changed since the last interview?


If yes, what is your role now?



How does it relate to the Demonstration?

2. Have there been any changes to the nursing program overall?


For example, have there been changes to student enrollment, course offerings, specific
curriculum, or staffing?

3. Have there been any changes to the process for coordinating students’ clinical
placements?


If yes, tell me about those changes.



What prompted those changes?

4. How does the number of clinical opportunities affect your acceptance decisions for qualified
applicants?
5. Have there been any key staffing changes made to support the demonstration project?


If yes, tell me about those changes.

GNE Evaluation Qualitative Data Collection Protocol
July 2015

Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

6. What, if any, changes to policy or procedure has your organization made to support the
demonstration project?


Does your SON accept students into an online training program?



If yes, do you permit online-enrolled students to participate in the GNE demonstration?
Describe the GNE precepting process for online-enrolled students.

Outcomes, Barriers, And Facilitators (30 minutes)
7. What effects do you believe the demonstration has had on enrollment?
Graduation?


Do you think changes in enrollment and graduations are primarily because of the
demonstration, or are there other factors influencing these?

8. What, if any, outcomes do you feel you have achieved through the demonstration
project?


How do you think this program has benefited your organization?



What are the benefits of this project?



To what extent did the demonstration project result in the outcomes you desired?



If no outcomes or benefits were achieved, why do you think this project has not
benefited your organization?

What made those challenges so difficult?



What components of your demonstration project have created challenges to achieving
the goals you set out to accomplish?

GNE Evaluation Qualitative Data Collection Protocol
July 2015

Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.



Are additional precepted hours at the expense of preceptors at other schools? If yes to
other schools, Are these other schools within the GNE demonstration network or outside
of the GNE demonstration network?

13. What are positive and/or negative effects of the GNE demonstration on non-demonstration
SONs?
14. If you had to start over from the beginning, what would you do differently knowing what
you now know?
15. If you had to give advice to other organizations considering similar programs, what
advice would you give?

Perceived Return On Investment And Sustainability Plans (15 Minutes)
16. To what extent do you think that the investments you have made to support the
demonstration project have paid off?


Has it been worth it?



What, if at all, do you see as the business case for expanding the number of APRN clinical
placement opportunities?

17. In what ways, if any, are you sustaining the GNE Demonstration Project activities beyond
grant funding?


If sustaining activities, what resources or investments will be necessary to sustain
these activities?



Where will those resources or investments come from?



If not sustaining activities, what makes the project unsustainable?

18. If you could redesign the GNE Demonstration Project for the future, what changes would
you suggest to the Centers for Medicare and Medicaid Services?

Conclusion (2 minutes)
19. Describe the best outcome or aspect of the GNE Demonstration? The worst?
20. Is there anything that I did not ask about that you would like to share with me, or do
you have any additional thoughts about what we have discussed today?
21. Are there any areas covered by this interview that you are particularly concerned
about having your identity connected with? If so, which ones?
GNE Evaluation Qualitative Data Collection Protocol
July 2015

Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

GRADUATE NURSE EDUCATION EVALUATION
FOCUS GROUP GUIDE: CLINICAL FACULTY
Obtain Informed Consent
Have interviewee read and sign the informed consent form. Give interviewee an unsigned copy of
the form to keep for his or her records.

[Note: 3 – 4 participants]
Outline with Timings
Time in each
section
(in minutes)

Topic

Elapsed time at end of
section
(in minutes)

Introduction

Changes to the demonstration project

Outcomes, barriers, and facilitators

Perceived return on investment and perspectives
on sustainability

Conclusion

Introduction (5 minutes)
The purpose of this interview is to discuss your ongoing activities related to the GNE
Demonstration Project, including the challenges you have faced and the strategies you have
used to overcome these challenges.
This focus group will last approximately 60 minutes.
As with our first site visit and with your approval, we will record audio of the discussion to assist
with note-taking. No one outside the evaluation team will have access to this recording.
This focus group will work best if you do most of the talking. Feel free to speak openly
and candidly about your experiences and perspectives regarding this demonstration.
In the final report, the data gathered through these interviews will be presented in an
aggregate manner, highlighting informational points from specific sites and not from particular
individuals. You will not be identified by name. However, it is possible that someone could
attribute specific comments to you. Hence, at the end of the interview, I’ll ask whether you
GNE Evaluation Qualitative Data Collection Protocol
July 2015

Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

told me anything that you wouldn’t want associated with you. If so, we will structure and
present the information so it cannot be linked to you.
Do you have any questions for me before we begin?

Participant Introductions
To get started, let’s go around and introduce ourselves. Please tell us your name, which nursing
program you teach in, how long you have been teaching clinical courses at this organization, how
long you have taught clinical courses overall, and what, if any didactic courses you teach.

Changes to the GNE Demonstration Project (5 Minutes)
I’d like to begin by discussing any key changes to the demonstration project
1. Have there been any changes to the process for coordinating students’ clinical
placements?


If yes, tell me about those changes.



What prompted those changes?

2. How does the number of clinical sites and preceptors affect your acceptance
decisions for qualified applicants?
3. Since we last spoke, what, if any, changes to policy or procedure has your organization
made?

Outcomes, Barriers, And Facilitators (30 minutes)
4. What milestones or successes have you or the SON achieved because of the GNE
demonstration?
5. What components of the demonstration project do you think facilitated the achievement
of these successes?


How did these components facilitate success?

6. Did the demonstration affect your faculty workload? If so, what was the change? Was it
positive or negative? Please describe how this demonstration affected your work.

GNE Evaluation Qualitative Data Collection Protocol
July 2015

Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

7. What are the key challenges you have faced related to your role and responsibility in the
demonstration project?


Were you able to overcome these challenges?
o

If yes, how so?

If no, what made it difficult to overcome these challenges?

8. How did you increase preceptor hours to meet the needs of your students?


Are additional precepted hours at the expense of preceptors at other schools? If yes to
other schools, are these other schools within the GNE demonstration network or outside
of the GNE demonstration network?



If yes to other schools, are the hours then reduced for other schools within the GNE
demonstration network or for schools outside of the GNE demonstration network?

9. If you had to start over from the beginning, what would you do differently knowing what
you now know?

Perceived return on investment and perspectives on sustainability (15 minutes)
10. To what extent do you think that the investments your organization has made to support
the demonstration project have paid off?


Has it been worth it?



In general, what do you see as the benefits of the project?



What do you see as the benefits to expanding the number of APRN clinical placement
opportunities?

11. Are you sustaining the GNE Demonstration Project activities beyond grant funding?


What is enabling/preventing you from sustaining the activities beyond grant funding?

12. If you could redesign the GNE Demonstration Project for the future, what changes would
you suggest to the Centers for Medicare and Medicaid Services?

Conclusion (5 minutes)
13. Describe the best outcome or aspect of the GNE Demonstration? The worst?
14. Is there anything that I did not ask about that you would like to share with me, or do
you have any additional thoughts about what we have discussed today?
GNE Evaluation Qualitative Data Collection Protocol
July 2015

Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

15. Are there any areas covered by this interview that you are particularly concerned
about having your identity connected with? If so, which ones.

GNE Evaluation Qualitative Data Collection Protocol
July 2015

Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

GRADUATE NURSE EDUCATION EVALUATION FOCUS GROUP GUIDE: APRN
STUDENTS
Obtain Informed Consent
Have interviewee read and sign the informed consent form. Give interviewee an unsigned copy of
the form to keep for his or her records.

[Note: 3 – 4 students per focus group]
Outline with Timings
Time in each section
(minutes)

Topic

Elapsed time at end of section
(minutes)

Introduction

Participant introductions

Placement in clinical rotations

Clinical training experiences

Conclusion

Introduction (5 minutes)
My name is {name of moderator}, and I am with IMPAQ International (IMPAQ). This is {name
of note-taker}, who is also with IMPAQ. As {name of GNE awardee liaison} probably told you,
IMPAQ is the contractor responsible for evaluating the Graduate Nurse Education
Demonstration Project.
The overall goals of the evaluation are to characterize the demonstration project, identify the
challenges and successes of implementation, assess the outcomes, and evaluate the potential
for sustainability, improvement, and replication of its outcomes.
As the students in schools participating in the GNE demonstration, your experiences
and perspectives are of great value to our evaluation.
This focus group will last 70 minutes. With your approval, we will record audio of the discussion
to assist with note-taking. No one outside the evaluation team will have access to this
recording.
In the final report, the data gathered through these interviews will be presented in an
aggregate manner, highlighting informational points from specific sites and not from particular
GNE Evaluation Qualitative Data Collection Protocol
July 2015

Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

individuals. You will not be identified by name. However, it is possible that someone could
attribute specific comments to you. Hence, at the end of the interview, I’ll ask whether you
told me anything that you wouldn’t want associated with you. If so, we will structure and
present the information so it cannot be linked to you.
This interview will work best if you do most of the talking. Feel free to speak openly and
candidly about your experiences and perspectives regarding this demonstration.
Do you have any questions for me before we begin?

Participant Introductions (5 minutes)
To get started, let’s go around and introduce ourselves. Please tell us your name, what program
you are in, whether you are choosing a specialty, and when you anticipate that you will be
graduating.
{Allow participants to introduce themselves}
Thank you. It is nice to meet all of you.

Placement In Clinical Rotations (20 minutes)
1. Tell me how your organization matches students with preceptors for your clinical
education.


What is the process?



Are you asked what types of preceptors or settings you prefer? If so, how are your
preferences obtained?

2. What do you like about the process?
3. What don’t you like about the process?


What challenges have you faced?



What changes would you make?

4. To what extent have you received clinical assignments that match your interests?

Clinical Training Experiences (35 minutes)
5. Tell me about your most positive clinical rotation experience.


What made it the most positive experience?

GNE Evaluation Qualitative Data Collection Protocol
July 2015

Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

6. Tell me about your worst clinical rotation experience.


What made it the worst experience?

7. Overall, how satisfied are you with your clinical placements?


For example:
o

the distance of your placement?

the amount of one-on-one attention from preceptors?

your involvement in the placement process?

Conclusion (5 minutes)
8. Is there anything that I did not ask about that you would like to share with me, or do you
have any additional thoughts about what we have discussed today?

GNE Evaluation Qualitative Data Collection Protocol
July 2015

Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

GRADUATE NURSE EDUCATION EVALUATION
INTERVIEW GUIDE: CLINICAL PLACEMENT COORDINATORS
Obtain Informed Consent
Have interviewee read and sign the informed consent form. Give interviewee an unsigned copy of the
form to keep for his or her records.
Outline With Timings
Time in each
section
(in minutes)

Topic

Elapsed time at end of
section
(in minutes)

Introduction

Changes to the demonstration project

Outcomes, barriers, and facilitators

Perspectives on sustainability

Conclusion

Introduction (3 minutes)
The purpose of this interview is to discuss your ongoing activities related to the GNE
Demonstration Project, including the challenges you have faced and the strategies you have used
to overcome these challenges.
This interview will last about 45 minutes.
As with our first site visit and with your approval, we will record audio of the discussion to assist
with note-taking. No one outside the evaluation team will have access to this recording.
This interview will work best if you do most of the talking. Feel free to speak openly and
candidly about your experiences and perspectives regarding this demonstration.
In the final report, the data gathered through these interviews will be presented in an aggregate
manner, highlighting informational points from specific sites and not from particular individuals.
You will not be identified by name. However, it is possible that someone could attribute specific
comments to you. Hence, at the end of the interview, I’ll ask whether you told me anything that
you wouldn’t want associated with you. If so, we will structure and present the information so it
cannot be linked to you.
Do you have any questions for me before we begin?
GNE Evaluation Qualitative Data Collection Protocol
July 2015

Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be privileged
and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to receive the
information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

If new interviewee: Interviewee Background
1. What is your title?
2. How long have you been with this organization?
3. What are your main responsibilities in your position?
4. What are your main responsibilities in relation to the GNE Demonstration Project? Are they
different or more specific than in your overall position?
5. What percentage of your FTE is dedicated to the GNE Demonstration Project?

Changes to the GNE Demonstration Project (5 minutes)
The purpose of this interview is to discuss your ongoing activities related to the demonstration
project, including the challenges you have faced and the strategies you have used to overcome
these challenges. I’d like to begin by discussing any key changes to the demonstration project since
the last time we spoke.

1. [If same interviewee as prior year] Has your role changed since the last interview?
a. If yes, what is your role now?
b. How does it relate to the Demonstration?
2. Have there been any changes to the process for coordinating students’ clinical
placements?


If yes, tell me about those changes.

What prompted those changes?
3. Since we last spoke, what, if any, changes to policy or procedure has your
organization made to support the demonstration project?


Outcomes, barriers, and facilitators (30 minutes)
4. What milestones or successes have you achieved because of the demonstration?
5. What components of the demonstration project do you think facilitated the
achievement of these successes?
a. How did these components facilitate success?

GNE Evaluation Qualitative Data Collection Protocol
July 2015

Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

6. What are the key challenges you have faced related to your role and responsibility in the
demonstration project?
7. Were you able to overcome these challenges?
b. If yes, how so?
c. If no, what made it difficult to overcome these challenges?
8. How did you increase preceptor hours to meet the needs of your students?
9. Are additional precepted hours at the expense of preceptors at other schools? If yes to
other schools, are these other schools within the GNE demonstration network or outside of
the GNE demonstration network?
10. If yes to other schools, are the hours then reduced for other schools within the GNE
demonstration network or for schools outside of the GNE demonstration network?
11. What are positive and/or negative effects of the GNE demonstration on non-demonstration
SONs?
12. Did the demonstration seem to affect your workload? If so, was it in a positive or
negative way? Please describe any changes to your workload.
13. If you had to start over from the beginning, what would you do differently knowing what
you now know?
14. If you had to give advice to other organizations considering similar programs, what advice
would you give?

Perceived sustainability (5 minutes)
15. How likely do you think it is that your organization will sustain the GNE Demonstration
Project activities beyond grant funding?
16. What makes you say that it will be {likely/unlikely} to sustain these activities?
17. If you could redesign the GNE Demonstration Project for the future, what changes would
you suggest to the Centers for Medicare and Medicaid Services?

Conclusion (2 minutes)
18. Describe the best outcome or aspect of the GNE Demonstration? The worst?
19. Is there anything that I did not ask about that you would like to share with me, or do
you have any additional thoughts about what we have discussed today?
GNE Evaluation Qualitative Data Collection Protocol
July 2015

Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

20. Are there any areas covered by this interview that you are particularly concerned
about having your identity connected with? If so, which ones?

GNE Evaluation Qualitative Data Collection Protocol
July 2015

Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

GRADUATE NURSE EDUCATION EVALUATION
INTERVIEW GUIDE: PRECEPTORS
Obtain Informed Consent
Have interviewee read and sign the informed consent form. Give interviewee an unsigned copy of
the form to keep for his or her records.
Outline with Timings
Time in each section
(in minutes)

Topic

Elapsed time at end of section
(in minutes)

Introduction

Interviewee background

History as a preceptor

Preceptor experiences

Conclusion

Introduction (5 minutes)
My name is {name of moderator}, and I am with the IMPAQ International (IMPAQ). This is
{name of note-taker}, who is also with IMPAQ. As {name of GNE awardee liaison} probably
told you, IMPAQ is the contractor responsible for evaluating the Graduate Nurse Education
Demonstration Project.
The overall goals of the evaluation are to characterize the demonstration project, identify the
challenges and successes of implementation, assess the outcomes, and evaluate the potential
for sustainability, improvement, and replication of its outcomes.
As a preceptor, your experiences and perspectives are of great value to our evaluation.
This interview will last 1 hour. With your approval, we will record audio of the discussion to
assist with note-taking. No one outside the evaluation team will have access to this recording.
This interview will work best if you do most of the talking. Feel free to speak openly
and candidly about your experiences and perspectives regarding this demonstration.
In the final report, the data gathered through these interviews will be presented in an
aggregate manner, highlighting informational points from specific sites and not from particular
individuals. You will not be identified by name. However, it is possible that someone could
GNE Evaluation Qualitative Data Collection Protocol
July 2015

Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

attribute specific comments to you. Hence, at the end of the interview, I’ll ask whether you
told me anything that you wouldn’t want associated with you. If so, we will structure and
present the information so it cannot be linked to you.
Do you have any questions for me before we begin?

GNE Evaluation Qualitative Data Collection Protocol
July 2015

Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

Interviewee background (5 minutes)
1. What is your title? (Physician? APRN? Primary care? Acute care?)
2. How long have you been with this organization?

History as a preceptor (10 minutes)
3. How long have you been a preceptor?


Do you precept more than one student per semester?



If so, how many?



Do you precept every semester?

4. Tell me about when you were first asked to become a preceptor.


Who asked you?



What were your initial thoughts? Were you eager, reluctant, interested?



What was the administrative process that you went through to become a preceptor?
Did you or your organization need to sign contracts? Has this process changed in the last
year or two?

5. What made you decide to become a preceptor?


What are the benefits of being a preceptor?



What are the challenges in being a preceptor?

6. Have you participated in any preceptor training?
If not, do you believe this would have been of value to you? In what ways?
If yes, tell me about that training.


Who delivered the training?



What was the length of the training?



What was the format of the training? Was it written documentation? Or was it
a presentation format of some other type?



What were the learning objectives of the training?



How well do you think that training prepared you to be an effective preceptor?

7. Do you precept for online-enrolled students?


If so, describe the pros and cons of this arrangement

GNE Evaluation Qualitative Data Collection Protocol
July 2015

Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

Preceptor experiences (35 minutes)
8. How would you rate the skills and preparedness of students you’ve been training?
9. Tell me about your experience in working with the faculty member(s) who teach the
corresponding clinical courses.


How involved are clinical faculty in the practicum experiences?



What do clinical faculty members do to coordinate their students’ precepted experience
with you?



Do clinical faculty make visits to your clinical site? If so, how often and for how long?
What is the purpose of the site visit?



Has this process changed over time? If so, in what way?

10. Tell me about your most positive experience as a preceptor.


What made it the most positive experience?

16. Tell me about your worst experience as a preceptor.


What made it the worst experience?

17. In general, what do you like about precepting?
18. What don’t you like about precepting?
14. Do you intend to continue precepting in the future?


If yes, what makes you want to continue to precept?



If no, what makes you not want to continue to precept?

15. How does precepting seem to affect your own clinical practice?


Does it affect the quality of the care you provide?

16. Does it affect your efficiency?


Does it enhance your practice in any way?

17. Do you get compensated in any way for precepting?


If yes, how does this affect your interest in continuing to precept?

GNE Evaluation Qualitative Data Collection Protocol
July 2015

Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

18. What schools do you typically take students from? Has this changed during the
demonstration?

Conclusion (5 minutes)
19. Describe the best outcome or aspect of the GNE Demonstration? The worst?
20. Is there anything that I did not ask about that you would like to share with me, or do
you have any additional thoughts about what we have discussed today?
21. Are there any areas covered by this interview that you are particularly concerned
about having your identity connected with? If so, which ones?

GNE Evaluation Qualitative Data Collection Protocol
July 2015

Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

GRADUATE NURSE EDUCATION EVALUATION INTERVIEW GUIDE: DIRECTOR OF
NURSING OR CLINICAL DIRECTOR
Obtain Informed Consent
Have interviewee read and sign the informed consent form. Give interviewee an unsigned copy of
the form to keep for his or her records.
Outline with Timings
Time in each
section
(in minutes)

Topic

Elapsed time at end of
section
(in minutes)

Introduction

Changes to the demonstration project

Outcomes, barriers, and facilitators

Perceived return on investment and sustainability
plans

Conclusion

Introduction (5 minutes)
The purpose of this interview is to discuss your ongoing activities related to the GNE Demonstration
Project, including the challenges you have faced, success strategies, perceived outcomes, and
perspectives on sustainability.
This interview will last 60 minutes.
As with our first site visit and with your approval, we will record audio of the discussion to assist
with note-taking. No one outside the evaluation team will have access to this recording.
This interview will work best if you do most of the talking. Feel free to speak openly and candidly
about your experiences and perspectives regarding this demonstration.
In the final report, all the data gathered through these interviews will be presented in an aggregate
manner, highlighting informational points from specific sites and not from particular individuals.
You will not be identified by name. However, it is possible that someone could attribute specific
comments to you. Hence, at the end of the interview, I’ll ask whether you told me anything that you
wouldn’t want associated with you. If so, we will structure and present the information so it cannot
be linked to you.
Do you have any questions for me before we begin?
GNE Evaluation Qualitative Data Collection Protocol
July 2015

Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

If New Interviewee, Interviewee Background
1. What is your title?
2. How long have you been with this organization?
3. What are your main responsibilities in your position?
4. What are your main responsibilities in relation to the GNE Demonstration Project (if
different or more specific)?
5. What percentage of your FTE is dedicated to the GNE Demonstration Project?

Changes to the Demonstration Project (5 Minutes)
I’d like to begin by discussing any key changes to the demonstration project since the last
time we spoke.


[If same interviewee as prior year] Has your role changed since the last interview?
o

If yes, what is your role now?

How does it relate to the Demonstration?

1. Have there been any changes to the process for coordinating students’ clinical
placements?


If yes, tell me about those changes.



What prompted those changes?

2. Since we last spoke, what, if any, changes to policy or procedure has your organization
made to support the demonstration project?

Outcomes, barriers, and facilitators (30 minutes)
3. What effects do you believe the demonstration has had on enrollment?
Graduation?


Do you think changes in enrollment and graduations are primarily because of the
demonstration, or are there other factors influencing these?

4. What, if any, outcomes do you feel you have achieved through the demonstration
project?


How do you think this program has benefited your organization?



What are the benefits of this project?

GNE Evaluation Qualitative Data Collection Protocol
July 2015

Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.



To what extent did the demonstration project result in the outcomes you desired?



If no outcomes or benefits were achieved, why do you think this project has not
benefited your organization?

5. What components of your demonstration project have allowed you to be successful in
achieving the goals you set out to accomplish?
6. What are the most difficult challenges you have faced in implementing your
demonstration project?


What made those challenges so difficult?

7. What components of your demonstration project have created challenges to achieving the
goals you set out to accomplish?


Have any aspects of the demonstration produced undesirable or negative outcomes?
Describe these aspects, the negative outcomes, and possible reasons/explanations for
the negative outcomes.

8. If you had to start over from the beginning, what would you do differently knowing what
you now know?
9. If you had to give advice to other organizations considering similar programs, what
advice would you give?

Perceived Return On Investment And Sustainability Plans (15 Minutes)
10. To what extent do you think that the investments you have made to support the
demonstration project have paid off?


Has it been worth it?



What, if at all, do you see as the business case for expanding the number of APRN clinical
placement opportunities?

11. In what ways, if any, will you sustain the GNE Demonstration Project activities beyond grant
funding?


If sustaining activities, what resources or investments will be necessary to sustain
these activities?



Where will those resources or investments come from?



If not sustaining activities, what makes the project unsustainable?

GNE Evaluation Qualitative Data Collection Protocol
July 2015

Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

12. If you could redesign the GNE Demonstration Project for the future, what changes
would you suggest to the Centers for Medicare and Medicaid Services?


Are there aspects of other APRN/specialty programs you would have liked to include in
your demonstration?

Conclusion (5 minutes)
13. Describe the best outcome or aspect of the GNE Demonstration? The worst?
14. Is there anything that I did not ask about that you would like to share with me, or do
you have any additional thoughts about what we have discussed today?
15. Are there any areas covered by this interview that you are particularly concerned
about having your identity connected with? If so, which ones?

GNE Evaluation Qualitative Data Collection Protocol
July 2015

Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

GRADUATE NURSE EDUCATION EVALUATION INTERVIEW GUIDE: CHIEF FINANCIAL
OFFICER
Obtain Informed Consent
Have interviewee read and sign the informed consent form. Give interviewee an unsigned copy of the
form to keep for his or her records.
Outline with Timings
Time in each section
(minutes)

Topic

Elapsed time at end of section
(minutes)

Introduction

Interviewee background

Investments related to the GNE
Demonstration Project
Perceived return on investment
and sustainability plans

Conclusion

10
2

Introduction (5 minutes)
My name is {name of moderator}, and I am with IMPAQ International (IMPAQ). This is {name of
note-taker}, who is also with IMPAQ. As {name of GNE awardee liaison} probably told you,
IMPAQ is the contractor responsible for evaluating the Graduate Nurse Education
Demonstration Project.
The overall goals of the evaluation are to characterize the demonstration project, identify the
challenges and successes of implementation, assess the outcomes, and evaluate the potential
for sustainability, improvement, and replication of its outcomes.
As the {title and/or role in demonstration project}, your experiences and perspectives are of
great value to our evaluation.
This interview will last 30 minutes. With your approval, we will record audio of the discussion to
assist with note-taking. No one outside the evaluation team will have access to this recording.
This interview will work best if you do most of the talking. Feel free to speak openly and
candidly about your experiences and perspectives regarding this demonstration.
GNE Evaluation Qualitative Data Collection Protocol
July 2015

Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be privileged
and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to receive the
information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

In the final report, the data gathered through these interviews will be presented in an aggregate
manner, highlighting informational points from specific sites and not from particular individuals.
You will not be identified by name. However, it is possible that someone could attribute specific
comments to you. Hence, at the end of the interview, I’ll ask whether you told me anything that
you wouldn’t want associated with you. If so, we will structure and present the information such
that it cannot be linked to you.
Do you have any questions for me before we begin?

GNE Evaluation Qualitative Data Collection Protocol
July 2015

Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be privileged
and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to receive the
information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

Interviewee Background (3 minutes)
1. What is your title?
2. How long have you been with this organization?
3. What are your main responsibilities in your position?
4. What are your main responsibilities in the GNE Demonstration Project, if different or more
specific than your overall position?
5. What percentage of your FTE is dedicated to the GNE Demonstration Project?

Investments related to the GNE Demonstration Project (10 minutes)
6. What are the key investments your organization has made to support the demonstration
project?


What are the main costs associated with your participation in the training program?



What costs will not be recouped via the demonstration funding?



Where did the funding for these investments come from?

Perceived return on investment and sustainability plans (10 minutes)
7. To what extent do you think that the investments you have made to support the
demonstration project have paid off?


Has it been worth it?



What, if at all, do you see as the business case for expanding the number of APRN



Clinical placement opportunities?



Do you believe you’ll see effects of the demonstration even after it’s ended? Are most
outcomes of the evaluation able to be observed now?

8. Did this demonstration create additional work for you or your staff that seemed excessive?


If yes, do you have any suggestions for streamlining the required reporting or related
work?

9. How financially viable would it be for your organization to sustain GNE Demonstration Project
activities beyond grant funding?


If financially viable, what resources or investments will be necessary to sustain these
activities?

GNE Evaluation Qualitative Data Collection Protocol
July 2015

Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be privileged
and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to receive the
information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.



Where will those resources or investments come from?



If not financially viable, what makes the project unsustainable?



What would it take to make the project financially viable?

10. If you could redesign the GNE Demonstration Project for the future, what changes would
you suggest to the Centers for Medicare and Medicaid Services?

Conclusion (2 minutes)
11. Describe the best outcome or aspect of the GNE Demonstration? The worst?
12. Is there anything that I did not ask about that you would like to share with me, or do you
have any additional thoughts about what we have discussed today?
13. Are there any areas covered by this interview that you are particularly concerned about
having your identity connected with? If so, which ones?

GNE Evaluation Qualitative Data Collection Protocol
July 2015

Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be privileged
and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to receive the
information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

II. Extension Year Check-In Call Protocol

GRADUATE NURSE EDUCATION EVALUATION INTERVIEW GUIDE: INTERIM CHECKIN
Outline with Timings
Time in each
section
(in minutes)

Topic

Elapsed time at end of
section
(in minutes)

Changes to demonstration project

Challenges and successes

Conclusion

Introduction (5 minutes)
The overall goals of the evaluation are to characterize the demonstration project, identify the
challenges and successes of implementation, assess the outcomes, and evaluate the potential
for sustainability, improvement, and replication of its outcomes.
This interview will last about 30 minutes. With your approval, we will record audio of the
discussion to assist with note-taking. No one outside the evaluation team will have access to
this recording.
This interview will work best if you do most of the talking. Feel free to speak openly and
candidly about your experiences and perspectives regarding this demonstration.
In the final report, the data gathered through these interviews will be presented in an aggregate
manner, highlighting informational points from specific sites and not from particular individuals.
You will not be identified by name. However, it is possible that someone could attribute specific
comments to you. Hence, at the end of the interview, I’ll ask whether you told me anything that
you wouldn’t want associated with you. If so, we will structure and present the information so it
cannot be linked to you.
Do you have any questions for me before we begin?
Okay, let’s get started.

Evaluation Qualitative Data Collection Protocol –OMB Renewal July 2015
35
Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be privileged
and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to receive the
information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

Changes to the GNE Demonstration Project (8 minutes)
The purpose of this interview is to discuss your ongoing activities related to the GNE
Demonstration Project, including the challenges you have faced and the strategies you have
used to overcome these challenges. I’d like to begin by discussing any key changes to the
demonstration project since the last time we spoke.
1. Have there been any changes to the nursing program as whole?


For example, have there been changes to student enrollment, course offerings, specific
curriculum, or staffing?

2. Have there been any changes to the process for coordinating students’ clinical placements?


If yes, tell me about those changes.



What prompted those changes?

3. Are there any organizations that are no longer participating in the demonstration project?


If yes, which organizations are no longer participating?



What made these organizations decide not to continue their participation?

4. Are there any new partners to the demonstration project?


If yes, who are the new organizations?



How did these organizations become involved in the demonstration project?

5. Since we last spoke, what, if any, changes to policy, procedure, or staffing has your
organization made to support the demonstration project?

Challenges and successes (20 minutes)
6. What milestones or successes have you achieved so far?
7. What do you think facilitated the achievement of these milestones/successes?
8. What are the key challenges you have faced related to your role and responsibility in
the demonstration project?
9. Were you able to overcome these challenges?


If yes, how so?



If no, what made it difficult to overcome these challenges?

Evaluation Qualitative Data Collection Protocol –OMB Renewal July 2015
36
Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

10. Do you have any suggestions to improve upon the demonstration?

Conclusion (2 minutes)
11. Is there anything that I did not ask about that you would like to share with me, or do
you have any additional thoughts about what we have discussed today?
12. Were there any areas covered during this call that you are particularly concerned
about having your identity connected with? If so, which ones?

Evaluation Qualitative Data Collection Protocol –OMB Renewal July 2015
37
Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

III. Post-Demonstration - Telephone-Based
Interviews with Limited Stakeholders

GRADUATE NURSE EDUCATION EVALUATION INTERVIEW GUIDE: GNE PROJECT
STRATEGIC PLANNING TEAM
Obtain Informed Consent
Have interviewee read and sign the informed consent form. Give interviewee an unsigned copy of
the form to keep for his or her records.
Outline with Timings
Time in each
section
(in minutes)

Topic

Elapsed time at end of
section
(in minutes)

Introduction

Changes since demonstration project end

Outcomes, barriers, and facilitators

Perceived return on investment and sustainability
plans

Conclusion

Introduction (5 minutes)
The purpose of this interview is to discuss your ongoing activities related to the GNE Demonstration
Project, including challenges, successful strategies, perceived outcomes, and sustainability plans.
This interview will last approximately 60 minutes.
As with our first site visit and with your approval, we will record audio of the discussion to assist
with note-taking. No one outside the evaluation team will have access to this recording.
This interview will work best if you do most of the talking. Feel free to speak openly and candidly
about your experiences and perspectives regarding this demonstration.
In the final report, the data gathered through these interviews will be presented in an aggregate
manner, highlighting informational points from specific sites and not from particular individuals.
You will not be identified by name. However, it is possible that someone could attribute specific
comments to you. Hence, at the end of the interview, I’ll ask whether you told me anything that
you wouldn’t want associated with you. If so, we will structure and present the information so it
cannot be linked to you.
Evaluation Qualitative Data Collection Protocol –OMB Renewal July 2015
39
Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

Do you have any questions for me before we begin? Okay, let’s get started.

If new interviewee: Interviewee Background
1. What is your title?
2. How long have you been with this organization?
3. What are your main responsibilities in your position?
4. What are your main responsibilities in relation to the GNE Demonstration Project (if
different or more specific)?
5. Are these responsibilities similar to what they were prior to the GNE Demonstration
Project? In what way?
6. What percentage of your FTE is dedicated to the GNE Demonstration Project?

Changes since the GNE Demonstration Project (8 minutes)
I’d like to begin by discussing any key changes since the end of the demonstration.
Since the demonstration ended:
1. Have there been any changes to the process for coordinating students’ clinical
placements?


If yes, tell me about those changes.



What prompted those changes?

2. Have there been any key staffing changes?


If yes, tell me about those changes.



[If same interviewee as prior year] Has your role changed since the last interview?
o

If yes, what is your role now?

How does it relate to the Demonstration?

3. Are there any organizations that are no longer participating since the demonstration project
ended?


If yes, which organizations are no longer participating?



What made these organizations decide not to continue their participation?

Evaluation Qualitative Data Collection Protocol –OMB Renewal July 2015
40
Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

4. Are there any new partners?


If yes, who are the new organizations?



How did these organizations become involved?

5. Are there any organizations or clinical agencies that increased or decreased the number of
preceptors?


If applicable, which organizations or clinical agencies increased the number of
preceptors?



If applicable, which organizations or clinical agencies decreased the number of
preceptors?

6. Since the demonstration ended, what, if any, changes to policy or procedure has your
organization made?

Outcomes, barriers, and facilitators (30 minutes)
7. What effects do you believe the demonstration has had on enrollment?
Graduation?


Do you think changes in enrollment and graduations are primarily because of
the demonstration, or are there other factors influencing these?

8. What, if any, outcomes do you feel you have achieved through the demonstration
project?


How do you think this demonstration project has benefited your organization?



What are the benefits of this demonstration project?



To what extent did the demonstration project result in the outcomes you desired?



If no outcomes or benefits were achieved, why do you think this project has not
benefited your organization?

9. What components of the demonstration project have allowed you to be successful in
achieving the goals you set out to accomplish?
10. What are the most difficult challenges you faced in implementing the demonstration
project?


What made those challenges so difficult?



What components of the demonstration project created challenges to achieving the
goals you set out to accomplish?

Evaluation Qualitative Data Collection Protocol –OMB Renewal July 2015
41
Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

11. Have any aspects of the demonstration produced undesirable or negative outcomes?
Describe these aspects, the negative outcomes, and possible reasons/explanations for
the negative outcomes.
12. If you had to start over from the beginning, what would you do differently knowing what
you now know?
13. If you had to give advice to other organizations considering similar programs, what
advice would you give?

Perceived return on investment and sustainability plans (15 minutes)
14. To what extent do you think the investments you made to support the
demonstration project have paid off?


Has it been worth it?



What, if at all, do you see as the business case for having expanded the number of APRN
clinical placement opportunities?

15. In what ways, if any, are you sustaining the GNE Demonstration Project activities now that
the funding has ended?


If sustaining activities, what resources or investments are necessary to sustain them?



Where are those resources or investments coming from?



If not sustaining activities, what makes the project unsustainable?

16. If you could redesign the GNE Demonstration Project for the future, what changes
would you suggest to the Centers for Medicare and Medicaid Services?

Conclusion (2 minutes)
17. Describe the best outcome or aspect of the GNE Demonstration? The worst?
18. Is there anything that I did not ask about that you would like to share with me, or do
you have any additional thoughts about what we have discussed today?
19. Are there any areas covered by this interview that you are particularly concerned
about having your identity connected with? If so, which ones?

Evaluation Qualitative Data Collection Protocol –OMB Renewal July 2015
42
Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

Topic

Elapsed time at end of
section
(in minutes)

Introduction

Changes since demonstration project end

Outcomes, barriers, and facilitators

Perceived return on investment and sustainability
plans

Conclusion

Introduction (5 minutes)
The purpose of this interview is to discuss your ongoing activities related to the GNE Demonstration
Project, including challenges, successful strategies, perceived outcomes, and sustainability plans.
This interview will last approximately 60 minutes.
As with our first site visit and with your approval, we will record audio of the discussion to assist
with note-taking. No one outside the evaluation team will have access to this recording.
This interview will work best if you do most of the talking. Feel free to speak openly and candidly
about your experiences and perspectives regarding this demonstration.
In the final report, the data gathered through these interviews will be presented in an aggregate
manner, highlighting informational points from specific sites and not from particular individuals.
You will not be identified by name. However, it is possible that someone could attribute specific
comments to you. Hence, at the end of the interview, I’ll ask whether you told me anything that
you wouldn’t want associated with you. If so, we will structure and present the information so it
cannot be linked to you.
Do you have any questions for me before we begin? Okay, let’s get started.
Evaluation Qualitative Data Collection Protocol –OMB Renewal July 2015
43
Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

If new interviewee: Interviewee Background
1. What is your title?
2. How long have you been with this organization?
3. What are your main responsibilities in your position?
4. What are your main responsibilities in relation to the GNE Demonstration Project (if
different or more specific)?
5. Are these responsibilities similar to what they were prior to the GNE Demonstration
Project? In what way?
6. What percentage of your FTE is dedicated to the GNE Demonstration Project?

Changes to the Nursing program since the GNE Demonstration Project ended (8
minutes)
I’d like to begin by discussing any key changes since the end of the demonstration project.
Since the demonstration ended:
1. [If same interviewee as prior year] Has your role changed since the last interview?


If yes, what is your role now?



How does it relate to the Demonstration?

2. Have there been any changes to the nursing program overall?


For example, have there been changes to student enrollment, course offerings,
specific curriculum, or staffing?

3. Have there been any changes to the process for coordinating students’ clinical
placements?


If yes, tell me about those changes.



What prompted those changes?

4. How does the number of clinical opportunities affect your acceptance decisions for qualified
applicants?
5. Have there been any key staffing changes made since the end of the demonstration project?


If yes, tell me about those changes.

Evaluation Qualitative Data Collection Protocol –OMB Renewal July 2015
44
Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

6. Since the demonstration project ended, what, if any, changes to policy or procedure has
your organization made?

Outcomes, barriers, and facilitators (30 minutes)
7. What effects do you believe the demonstration has had on enrollment?
Graduation?


Do you think changes in enrollment and graduations are primarily because of
the demonstration, or are there other factors influencing these?

8. What, if any, outcomes do you feel you have achieved through the demonstration
project?


How do you think this program has benefited your organization?



What are the benefits of this project?



To what extent did the demonstration project result in the outcomes you desired?



If no outcomes or benefits were achieved, why do you think this project has not
benefited your organization?

9. What components of your demonstration project allowed you to be successful in
achieving the goals you set out to accomplish?
10. What were the most difficult challenges you faced in implementing your demonstration
project?


What made those challenges so difficult?



What components of your demonstration project created challenges to achieving the
goals you set out to accomplish?



If yes to other schools, are the hours then reduced for other schools within the GNE
demonstration network or for schools outside of the GNE demonstration network?

Evaluation Qualitative Data Collection Protocol –OMB Renewal July 2015
45
Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

13. If you had to start over from the beginning, what would you do differently knowing what
you now know?
14. If you had to give advice to other organizations considering similar programs, what
advice would you give?

Perceived return on investment and sustainability plans (15 minutes)
15. To what extent do you think the investments you made to support the
demonstration project have paid off?


Has it been worth it?



What, if at all, do you see as the business case for expanding the number of APRN clinical
placement opportunities?

16. In what ways, if any, are you sustaining the GNE Demonstration Project activities beyond
grant funding?


If sustaining activities, what resources or investments are necessary to sustain
these activities?



Where are those resources or investments coming from?



If not sustaining activities, what made the project unsustainable?

17. If you could redesign the GNE Demonstration Project for the future, what changes would
you suggest to the Centers for Medicare and Medicaid Services?

Conclusion (2 minutes)
18. Describe the best outcome or aspect of the GNE Demonstration? The worst?
19. Is there anything that I did not ask about that you would like to share with me, or do
you have any additional thoughts about what we have discussed today?
20. Are there any areas covered by this interview that you are particularly concerned
about having your identity connected with? If so, which ones?

Evaluation Qualitative Data Collection Protocol –OMB Renewal July 2015
46
Close Hold – Do Not Share
INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not been publicly disclosed and may be
privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to
receive the information. Unauthorized disclosure may result in prosecution to the fullest extent of the law.

File Type	application/pdf
File Title	Evaluation of the Graduate Nurse Education Demonstration Program
Subject	Evaluation of the Graduate Nurse Education Demonstration Program, Qualitative Data, Collection Protocol, Telephone site Visit, T
Author	Centers for Medicare and Medicaid Services (CMS)
File Modified	2015-09-30
File Created	2015-09-22