2nd Passback CMS-10467 Evaluation of GNE Demo Supporting Statement B_CLEAN_TO CMS

The following is a description of data collection procedures for the evaluation of the Graduate Nurse Education (GNE) Demonstration. Under the GNE Demonstration, CMS awarded contracts to five networks, each consisting of a hospital and affiliated schools of nursing (SON) and community-based primary care sites (CCS). The Demonstration began in 2012, when networks began enrolling new students and offering payments to clinical preceptors to educate these new students; networks are no longer allowed to enroll new students into the Demonstration beyond the spring 2016 semester. The Demonstration was extended through 2018 to allow students enrolled earlier to complete training and graduate. The evaluation will end in 2019.

During the first three years of the evaluation, primary qualitative and quantitative data were collected to obtain an in-depth understanding of how the Demonstration was implemented within each network, and to track implementation over time. Qualitative data collection included annual in-person site visits to each of the five networks and annual check-in calls with GNE oversight and SON administrators. During each site visit, in-depth interviews were conducted with the GNE Oversight Team, administrators of affiliated SONs, clinical placement coordinators, and preceptors; focus groups were conducted with clinical faculty members, and with Advanced Practice Registered Nurse (APRN) students. Annual quantitative data collection consisted of each network reporting yearly descriptive information using six instruments on 1) preceptors, 2) APRN students, 3) alumni, 4) GNE hospital, 5) GNE SONs, and 6) affiliated community-based care settings (CCSs).

However, data collection needs have changed given the shift from active Demonstration enrollment to extension-year project closing. Proposed revised data collection activities include annual qualitative telephone check-in calls/post-Demonstration interviews with GNE network leadership to track experiences and perceptions about the success and sustainability of the Demonstration. Quantitative primary data collection, and qualitative in-person interviews and focus groups with students, faculty, and clinical preceptors covered by the original ICR are no longer necessary.

1. Respondent Universe and Sampling Methods

The respondent universe for the proposed revised data collection includes all members of the Graduate Nurse Education Demonstration Oversight Team and GNE SON administration teams at each of the five networks. The Oversight Team for each network was determined by the network as part of their application to the Demonstration program; although Oversight Team composition is different across networks, each Oversight Team includes the GNE Project Director (who could be Director of Operations for the awardee hospital/hospital system or a faculty member of an affiliated SON), the Chief Financial Officer or someone familiar with financial management of the GNE award, and the GNE Project Coordinator hired specifically by the network to run the project. SON administration teams consist of the Dean of each GNE SON, and for some SONs, the Assistant Deans. Like the Oversight Team, SON Administrative teams were determined by each SON at the time of application to the Demonstration project. Sampling strategies will not be employed, as all members of the Oversight and SON administration teams are invited to participate in annual telephone interviews.

2. Procedures for the Collection of Information

Members of the GNE Oversight Team and SON administration teams at each network will be contacted by CMS’s contractor, IMPAQ International, via email, to schedule annual telephone interviews. Each network has one GNE Oversight Team, and at least one SON administration team; some networks have more than one associated SON, and telephone interviews will be conducted with the SON administration from each. Exhibit B-1 shows the breakdown of each telephone interview, annually.

Exhibit B-1: Qualitative Data Collection Activities per Network, per Year

Network	Qualitative Telephone Interviews per Year	Total Calls per Network per Year
Network #1	GNE Oversight Team Call, 1 SON Administration Team Call	2 calls
Network #2	GNE Oversight Team Call, 1 SON Administration Team Call	2 calls
Network #3	GNE Oversight Team Call, 4 SON Administration Team Calls	5 calls
Network #4	GNE Oversight Team Call, 4 SON Administration Team Calls	5 calls
Network #5	GNE Oversight Team Call, 9 SON Administration Team Calls	10 calls

The purpose of the annual Oversight Team and SON administration team telephone check-in calls is to learn about any changes that have been made to the Demonstration implementation at each network, outcomes achieved over the past year, perceived barriers and facilitators to the Demonstration’s success at each network, and perceived return on investment and sustainability plans. After the Demonstration has ended, post-demonstration telephone interviews will inquire about changes made since its end, and stakeholders’ summary thoughts on outcomes, barriers, and facilitators of success, perceived return on investment, and plans for the future. Check-in and post-demonstration calls will last an hour. Whereas prior year qualitative data collection consisted of in-person and telephone interviews and focus groups to gain detailed information to monitor Demonstration implementation, the proposed activities going forward are intended to capture perceptions of network leadership as the Demonstration comes to a close.

After each call is scheduled, the contractor will confirm the date and time with participants within the week prior to the call. OpenVoice, a telephone conferencing system, will be used to conduct the annual telephone check-in calls and post-demonstration telephone interviews. After participants dial into each call, the contractor will facilitate introductions, read a verbal informed consent statement (Attachment 2), and ask respondents if they agree to be audio-recorded. With all participants’ consent, the contractor will record the call and begin the 60-minute interviews. Audio-recordings will be stored on the contractor’s FISMA-compliant enclave, along with notes taken during the calls. Audi-recordings will then be reviewed to clean and add detail to notes.

3. Methods to Maximize Response Rates and Deal with No Response

In-person interviews and telephone check-in calls during the first four years of the project were very successful; stakeholders from all five networks consistently participated on a bi-annual basis. Stakeholders understand the importance of sharing their perspectives on the Demonstration’s successes and challenges to the overall evaluation. Participating in regular data collection activities is also an expectation of participating in the Demonstration. CMS and its contractor, IMPAQ International, anticipate that the GNE Oversight and SON administration team members will continue to participate in annual check-in calls and post-demonstration telephone interviews.

4. Test of Procedures or Methods to Be Undertaken

Questions contained in the telephone check-in call and post-demonstration telephone interview protocols about changes to the Demonstration, outcomes, barriers and facilitators of success, return on investment, and sustainability have been regularly asked to the GNE Oversight and SON administration team members since the beginning of the demonstration. Going forward, some additions and changes to wording are proposed to collect data specific to the ending of the Demonstration, and some additional probes have been added to existing questions (see Exhibit A-5 in Supporting Statement A).

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

This information collection request does not employ statistical methods.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Modified	0000-00-00
File Created	2021-01-24

2nd Passback CMS-10467 Evaluation of GNE Demo Supporting Statement B_CLEAN_TO CMS