Attachment 3 - Crosswalk for Changes to Survey Instrument

CMS-10675 Attachment 3

Crosswalk for Changes to Survey Instrument for Adverse Drug Events

Section	Type of Change	Rationale for Change
Introduction and Informed Consent	Added definition of adverse drug events (ADE)	Recommendation from comments received during the 60-day public notice
Introduction and Informed Consent	Started a new paragraph	To improve readability of the introduction and informed consent.
PRA Disclosure Statement	Changed “[Insert Time (hours or minutes)]” to “10-20 minutes”	To improve clarity of instructions.
Screener	S1 – Changed question from identifying industry to identifying state	Given the sample source, all participants will pass the industry question. It was changed to identify state for verification of the sample.
Screener	S2 – Separated “Physician Assistant” and “Nurse Practitioner” response options	To more accurately collect response options.
Screener	S2 – Deleted “Behavioral Health Clinician (e.g. therapist, clinical psychologist, counselor, social worker)” response option	To more accurately recruit the intended survey participants based on the inclusion/exclusion criteria.
Screener	S2 – Deleted “Nursing Home Administrator” and “Director of Nursing” options.	To more accurately recruit the intended survey participants based on the inclusion/exclusion criteria.
Screener	S3 – Changed response option “pharmacy” to “hospital pharmacy” (end survey) and “community or retail pharmacy” (continue with survey).	To more accurately recruit the intended survey participants based on the inclusion/exclusion criteria.
Screener	S3 –– Removed response options related to nursing homes.	Based on feedback from QIN-QIO stakeholders, the survey will no longer include nursing home administrators as participants.
Screener	S3 – Added “Community retail pharmacy” to response options.	To more accurately recruit the intended survey participants based on the inclusion/exclusion criteria.
Screener	S4 – Changed phrase “how many staff” to “how many providers.”	To ensure that we are obtaining the same information about all the practices we are assessing.
Screener	S4 – Removed references to nursing homes.	Based on feedback from QIN-QIO stakeholders, the survey will no longer include nursing home administrators as participants. As a result, we removed references to nursing home and long-term care facilities throughout the survey.
Screener	S4 – Indicated changes to “staff” through following criteria: [INSERT “providers” IF S3 = “Physician’s office or group practice” OR “pharmacists” IF S3 = “Community or Retail Pharmacy”]	To accurately collect information based on respondent’s status.
Screener	S4 – Changed “Pharmacy” to read as “Community or retail pharmacy”.	To more accurately recruit the intended survey participants based on the inclusion/exclusion criteria.
Screener	S5 – Changed question to read “Is the [INSERT “practice” or “pharmacy]” depending on participant’s setting] where you primarily work part of any of the following?”	To obtain more thorough responses based on survey pretests of participants’ settings.
Screener	S5 – Provided the following response options: [SINGLE PUNCH PER ITEM] [RANDOMIZE ITEMS]] [ITEMS] [IF S3 = “Physician’s office or group practice”] Network of practices [IF S3 = “Community or retail pharmacy”] Corporate chain [IF S3 = “Community or retail pharmacy”] Pharmacy co-op [ALL] Health system	To more accurately collect data.
Screener	S6, S7, S8 – Removed “facility” as part of work environment option.	This change reflects removing nursing homes from the sample.
I. Quality Improvement Initiatives	Q1 – Changed the question wording from “has your facility worked on any” to “has your practice or pharmacy begun or continued working on any.”	To obtain whether facilities were working on new efforts or continuing to work on existing QI efforts related to ADEs.
I. Quality Improvement Initiatives	Q1 – Changed the response options from “Yes/No” to “Yes, began new QI activities; Yes, Continued existing QI activities; No, haven’t worked on QI activities.” Inserted the parenthetical (QI) after the phrase “quality improvement” in the question text.	To obtain whether facilities were working on new efforts or continuing to work on existing QI efforts related to ADEs.
I. Quality Improvement Initiatives	Q3 – Used bold to highlight “your practice/pharmacy”.	To clarify that the question is referring to systematic changes at the institutional level, not based on the individual participant.
I. Quality Improvement Initiatives	Q3 – Added response options: Pharmacist case management for patients with several medications Medication management review Instituting electronic health record (EHR) or electronic medical record (EMR)-populated notifications of drug interactions and/or allergies Educating patients/customers on signs of opioid misuse, abuse, dependence, and addiction Teaching patients/customers and family members how to identify and treat an overdose Advising patients/customers on the availability of mental health and substance use disorder treatment methods and facilities Documenting and monitoring adverse drug event rates within your patients/customers	Based on pretest results, multiple participants recommended including these additional response options.
I. Quality Improvement Initiatives	Q3 – Option A: Changed “screening data” to “Screening or review of data, reports or graphs.”	Preferred labelling.
I. Quality Improvement Initiatives	Q3 – Option I: Changed “substance” to “opioid.”	To clarify intent of question.
I. Quality Improvement Initiatives	Q3 – Option N: Changed “Educate… on signs of opioid misuse, abuse, dependence, and addiction” to “Educating…on opioid guidelines”	Original specification wasn’t in this contract work
I. Quality Improvement Initiatives	Q3 – Option O: Changed “overdose” to “adverse drug events.”	Not all ADEs are overdoses.
I. Quality Improvement Initiatives	Q3 – Option P: “Advise [patients/ customers] on the availability of mental health and substance use disorder treatment methods and facilities” to “Coordinating with hospitals, skilled nursing facilities or other healthcare services around transfers of [patients/ customers] from these facilities.”	Original specification was not part of the C.3.6 contract or frame/or scope of ADE reduction. We substituted an activity more relevant to the task.
I. Quality Improvement Initiatives	Q3 and Q4 – Rephrased both Q3S and Q4 to indicate that “involving patients and family members” means “involving patients and family members in your efforts to improve medication safety.”	Based on multiple potential meanings of involving patients and family members mentioned during the survey pretests by participants, these items were revised to clarify the intended meaning.
I. Quality Improvement Initiatives	Q5 – Removed mentions of “facility” and “nursing homes.”	Based on feedback from stakeholders, we have removed nursing homes from our sample.
I. Quality Improvement Initiatives	Q5 – Added new responses and wording as detailed in Q3	Based on recommendations received in comments and pretesting
II. Outcome Attribution to QIO	Q6 – Added an introductory screen stating: “The next several questions ask about different programs, organizations, and other resources your practice/ pharmacy may have used when developing and implementing quality improvement activities related to preventing and reducing adverse drug events. These questions refer to your efforts since [for practices display ‘January 2015’. For pharmacies display ‘September 2016’.”	To clarify intent of question.
II. Outcome Attribution to QIO	Q6 – Added timeframe to the question.	Based on pretest results, clarified the timeframe for using the resources.
II. Outcome Attribution to QIO	Q6 – Updated response options.	Based on pretest results, included additional resources that were reported useful by participants.
II. Outcome Attribution to QIO	Q6 – For the introduction to the third group of items (information and resources), rephrased to say: “Did your [practice/pharmacy] use information and resources from any of these organizations in your efforts to promote medication safety or prevent adverse drug events?”	Modified text to be consistent with the introduction of other items.
II. Outcome Attribution to QIO	Q6 – For group 4 of response options, rephrased “No outside help – staff’s own initiative” as “Ideas and initiatives developed by your [practice’s/pharmacy’s] own staff.”	To clarify existing response option based on the pretest results.
II. Outcome Attribution to QIO	Q6 – Removed nursing home response options.	Based on feedback from stakeholders, we have removed nursing homes from our sample.
II. Outcome Attribution to QIO	Q6 – Option A: Changed “QIO Collaborative” to “QIO Collaborative or Campaign for Medicine Management.”	To clarify existing response option based on comments from stakeholders.
II. Outcome Attribution to QIO	Q7 – Highlighted the word “helpful” with bold font.	To help clarify what question is assessing based on participant confusion during pretests.
II. Outcome Attribution to QIO	Q7 – Switched response from allocating percentages to qualitative scales with all points labeled.	To increase comprehension and reduce burden of response, the response options were modified to a qualitative scale based on multiple participants’ difficulty in answering this question during pretests and suggestions to develop a scale.
II. Outcome Attribution to QIO	Q7 – Changed instructions to reflect qualitative scale: “in your quality improvement efforts to improve medication safety and reduce adverse drug events;” and to promote medication safety or prevent adverse drug events “since [for practices display ‘January 2015’. For pharmacies display ‘September 2016’]”	Reflects decision to adopt qualitative scale (see above).
II. Outcome Attribution to QIO	Q7 – Constructed new set of response options, including both updated versions of old response options and new response options.	To improve clarity of response criteria/more accurately collect data.
II. Outcome Attribution to QIO	Q7 and Q7AA – Option A: Changed “QIO Collaborative” to “QIO Collaborative or Campaign for Medicine Management.”	To clarify existing response option based on comments from stakeholders.
II. Outcome Attribution to QIO	Q7AA – Added tie-breaker question.	To provide information on what resource participants found most helpful.
III. Non-QIO Practices/ Pharmacies	Q10 – Bolded the phrase “participated in any activities.”	To help clarify the focus of the question based on feedback during pretests.
III. Non-QIO Practices/ Pharmacies	Q11 – Removed mentions of nursing homes.	Reflects decision to remove nursing homes from sample.
IV. Interaction with the QIN-QIO	Q14 – Used bold to highlight the type of interaction “one-on-one or small group meetings.”	To help clarify the type of interaction based on participant input during pretests.
IV. Interaction with the QIN-QIO	Q15 – Used bold to highlight the type of interaction “Apart from one-on-one exchanges.”	To clarify the type of interaction based on participant confusion during pretests.
IV. Interaction with the QIN-QIO	Q16 – Used bold to highlight the type of interaction “meetings with these other healthcare providers or pharmacists.”	To clarify the type of interaction based on participant confusion during pretests.
V. Activities and Resources Provided By QIN-QIO	Q19 – Changed response option D to read: Technical assistance on using data to monitor potential ADEs or occurrence of ADEs with patients/customers	Based on participant input during pretests.
V. Activities and Resources Provided By QIN-QIO	Q19 – Option A: Changed “Data” to “Data, reports or graphs.”	To clarify existing response option based on comments from stakeholders.
V. Activities and Resources Provided By QIN-QIO	Q20 – Minor wording change from: “level of engagement in” to “level of engagement with.”	To improve the clarity of the sentence.
V. Activities and Resources Provided By QIN-QIO	Q21 – Added an instruction, “Please provide enough detail so that CMS can understand what type of assistance you would need for this quality improvement area.”	To clarify the questions instructions and obtain more informative responses.
V. Activities and Resources Provided By QIN-QIO	Q21 – Added instruction: IF QIO EXPERIENCE = 0, ASK Q22. ELSE, SKIP TO FINAL SCREEN.	Programming instruction to set up respondents for Q22
V. Activities and Resources Provided By QIN-QIO	Added additional question (Q22) for participants not involved with the QIO program to gauge interest in TA provided by QIN-QIO program.	To assess the potential usefulness of instituting the activities identified in Q3 for participants not involved in the QIO program.
V. Activities and Resources Provided By QIN-QIO	Q22 – Made changes consistent with Q3

12