Form 15 Attachment R: Web Survey of CDS Connect Users

Attachment R

Form Approved
OMB No. 0935-XXXX
Exp. Date XX/XX/20XX

Attachment R: Web Survey of CDS Connect Users

Section 0 – Introduction

In partnership with the Agency for Healthcare Research and Quality (AHRQ), NORC at the University of Chicago, an independent non-profit research organization, is administering a survey to understand the population accessing Clinical Decision Support (CDS) Connect resources, their reasons for using the resources, how they use these resources, and their perceptions about their value.

0.1 Instructions for Completing Survey (Consent)

This survey should take an average of ten minutes to complete.

In collaboration with AHRQ, we have identified you as someone who is likely to have used CDS Connect resources, and to know others that have used it as well. We will be using the names and e-mails of people to track the completion of surveys and reach out to people you may refer us to. However, we will keep your name confidential to the extent permitted by law, including AHRQ’s confidentiality statute, 42 USC 299c-3(c), and we will not refer to you by name in any public facing report. We will submit de-identified survey data to AHRQ at the end of the Evaluation period.

If at any time you need to close the survey before you are finished, you can do so. You will be able to come back to the survey where you left off by following the link you used to first access the survey.

If you have any questions, please contact [project email] or call us at [project phone].

To signal your consent to participate in this survey, please let us know if you agree with the following statement:

I have received and read the above information about voluntary participation and agree to take this survey.

• Yes [Continue to 1.1]

• No [Skip to Section 10]

This survey is authorized under 42 U.S.C. 299a. The confidentiality of your responses to this survey is protected by Sections 944(c) and 308(d) of the Public Health Service Act [42 U.S.C. 299c-3(c) and 42 U.S.C. 242m(d)].  Information that could identify you will not be disclosed unless you have consented to that disclosure. Public reporting burden for this collection of information is estimated to average XX minutes per response, the estimated time required to complete the survey. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number.  Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: AHRQ Reports Clearance Officer Attention: PRA, Paperwork Reduction Project (0935-XXXX) AHRQ, 5600 Fishers Lane, Room #07W42, Rockville, MD 20857.

Section 1 – Respondent Background

1.1 Respondent Background (RB-Role)

[All respondents]

What role(s) do you play in the clinical decision support (CDS) field?

Please select all that apply.

[Response options will be randomized in web survey]

Clinician	Medical professionals who care for patients (physicians, nurses, etc.).
Health IT Developer	Professionals that provide health-related technology solutions (electronic health record [EHR] vendors, CDS vendors, etc.).
Informaticist	Institutional health IT professional (e.g., clinical informaticist, CMIO, CCIO)
Knowledge Author	Professionals such as domain experts and professional societies who write guidelines or other materials that provide clinical evidence to users in unstructured formats (narrative text, image files, etc.).
Knowledge Curator	Professionals who maintain knowledge artifact libraries to ensure evidence is trustworthy (accurate, reliable, timely, etc.).
Knowledge Distributor	Professional at organizations that package, market, or sell knowledge artifacts as private organizations or in public-private partnerships.
Knowledge Engineer	Professionals who translate clinical guidelines into artifacts in semi-structured human readable form (L2), a computer interpretable form (L3), and machine executable formats (L4).
Organizational Governance Bodies Representative	Individual that is part of a governance body that reviews and approves CDS to be used in an organization or across networks.
Patient or Consumer Advocate	Professionals or volunteers that advocate on behalf of particular patient populations or consumers either as an individual or as part of an organization.
Payer	Organizations that pay clinicians or patients for health-related activities (Medicaid Managed Care Organizations, Medicare Advantage Plans, Aetna, Cigna, etc.).
Policymaker	Professionals who develop legal or policy guidance that guide care or payment.
Population Health End User	Professionals who support clinicians and clinical teams by monitoring population health trends and recommending actions.
Standards Developer	An individual that is part of a clinical standards development organization that develops and maintains specifications related to standards-based content
Quality Improvement Analyst	Professionals who measure the impact of implemented CDS within health IT.
Other	Please describe, _____________

[Continue to 1.2]

1.2 Respondent Background (RB-Yrs)

[All respondents]

Approximately how many years have you been involved in developing, implementing, or using CDS?

__________ years

• less than 1 year

[Continue to 2.1]

Section 2 – Branch

2.1 Use of CDS Connect Resources (Use-YN)

The following questions ask about your experience with CDS Connect resources. AHRQ began sponsoring CDS Connect in 2016. CDS Connect provides several resources, including:

• Artifact Repository:

• Houses evidence-based, shareable CDS artifacts contributed by interested stakeholders

• Currently includes a searchable inventory of over 50 CDS artifacts that address a variety of domains (e.g., cholesterol management, pain management, and mental health)

• Authoring Tool:

• Provides an interface for creating CDS logic using simple forms and exporting CDS logic as HL7 Clinical Quality Language (CQL) artifacts

• Leverages Health Level Seven (HL7) Fast Healthcare Interoperability Resources (FHIR)

• GitHub Resources:

• Open source resources available to download and use in your own software; including the CDS Connect Application Program Interface (API), CDS Clinical Quality Language (CQL) Services Tool and the CQL Testing Framework

[All Respondents]

Have you ever used any resources from CDS Connect?

• Yes [Continue to 2.2]

• No [Skip to 6.1]

2.2 Use of CDS Connect Resources (Use-Branch)

[All respondents]

Have you ever used any of the following resources from CDS Connect?

Please select all that apply.

• Artifact Repository [If selected complete Section 3/If not selected complete 6.1-3]

• Authoring Tool [If selected complete Section 4 / If not selected complete 6.4-6]

• GitHub Resources [If selected complete Section 5]

• Don’t know [If selected skip to 6.1]
  

Section 3 – Artifact Repository

3.1 Artifact Repository (AR-Visits)

You indicated that you have used the Artifact Repository resource on CDS Connect. The Artifact Repository:

• Houses evidence-based, shareable CDS artifacts contributed by interested stakeholders

• Includes a searchable inventory of over 50 CDS artifacts that address a variety of domains (e.g., cholesterol management, pain management, and mental health)

[Complete if 2.2 = Artifact Repository]

Have you visited this Repository more than once?

• Yes [Continue to 3.2]

• No [Skip to 3.3]
  
  

3.2 Artifact Repository (AR-Freq)

[Complete if 3.1 = Yes]

In the past year, how often did you visit the Repository?

• Daily

• Weekly

• Monthly

• Every few months

• Twice a year

• Annually

• Less than once a year

[Continue to 3.3]

3.3 Artifact Repository (AR-How)

[Complete if 2.2 = Artifact Repository]

How have you used the Repository?

Please select all that apply

To browse the artifacts	Visited the Repository and looked at what artifacts were available.
To inspect CDS artifacts	Reviewed particular CDS artifacts in the Repository and made assessments (e.g., fitness for use) based on available metadata.
To adapt CDS artifacts for use	Downloaded and adapted the CDS artifacts for use in a local environment (whether implemented or not).
To contribute CDS artifacts	Offered original CDS artifacts to the repository by contacting ARHQ and MITRE through the “Grow the Repository” feature on the website.
To provide feedback	Offered input about the veracity or value of a knowledge artifact(s), or to offer implementation experiences, to knowledge artifact authors/ stewards/ publishers/ contributors.
Other	Please describe:_____________________

[Continue to 3.4]

3.4 Artifact Repository (AR-Barriers1)

[Complete if 2.2 = Artifact Repository]

Have you ever experienced any issues when using the Repository?

• Yes [Continue to 3.5]

• No [Skip to 3.6 ]

3.5 Artifact Repository (AR-Barriers1Describe)

[Complete if 3.4 = Yes]

Please indicate whether you have ever experienced any of the issues below when using the Repository:

Please answer yes or no to each response option.

	Yes	No
I had difficulty finding the types of artifacts that were relevant to me or my organization’s needs	o	o
I had difficulty navigating the Repository	o	o
I couldn’t compare artifacts easily	o	o
I didn’t understand the metadata	o	o

• Other, please specify: ____________________________

[Continue to 3.6]

3.6 Artifact Repository (AR-TA)

[Complete if 2.2 = Artifact Repository]

Have you ever received any technical support from the CDS Connect team when using the Repository?

• Yes [Continue to 3.7]

• No [Skip to 3.8]

3.7 Artifact Repository (AR-TAQuality)

[Complete if 3.6 = Yes]

How satisfied were you with the quality of the technical support?

• Very satisfied

• Satisfied

• Neither satisfied nor dissatisfied

• Dissatisfied

• Very dissatisfied

[Continue to 3.8 and follow skip patterns.]

3.8 Artifact Repository (AR-Topics)

[Complete if 3.3 = “To inspect” and/or “To adapt CDS artifacts for use”.]

Please identify any artifacts from the Repository that you inspected and/or adapted. The artifacts are grouped by general topic for ease of search and recall.

Please select all that apply.

3.8A - Primary Care

• Aspirin Therapy for Primary Prevention of CVD and Colorectal Cancer

• Clostridoides Difficile (C. diff) Infection (CDI) Treatment Pathway

• Eye Care Documentation Template

• Immunization Calculation Engine (ICE)

• Management of Community-Acquired Pneumonia in Adults

• Primary Care: Anaphylaxis Treatment Protocol Order Set

• Primary Care: Family Health History Documentation Template

• Primary Care: General Clinical Note - History and Physical Exam Documentation Template

• Primary Care: Tobacco Assessment and Cessation Counseling Documentation Template

• On this topic but I do not recall exactly which one.

3.8B - Cardiovascular Disease

• CMS’s Million Hearts® Model Longitudinal ASCVD Risk Assessment Tool for Baseline 10-Year ASCVD Risk

• CMS’s Million Hearts® Model Longitudinal ASCVD Risk Assessment Tool for Shared Decision Making

• CMS’s Million Hearts® Model Longitudinal ASCVD Risk Assessment Tool for Updated 10-Year ASCVD Risk

• Healthful Diet and Physical Activity for CVD Prevention in Adults With Cardiovascular Risk Factors Healthy Weight Care Assistant

• Statin Therapy for the Prevention and Treatment of Cardiovascular Disease (CVD) eCQM

• Statin Use for the Primary Prevention of CVD in Adults: Clinician-Facing CDS Intervention

• Statin Use for the Primary Prevention of CVD in Adults: Patient-Facing CDS Intervention

• On this topic but I do not recall exactly which one

3.8C - Diabetes Mellitus

• Abnormal Blood Glucose and Type 2 Diabetes Mellitus: Part One, Screening

• Abnormal Blood Glucose and Type 2 Diabetes Mellitus: Part Two, Counseling

• Endocrinology: Hypoglycemia

• Endocrinology: Hypoglycemia Documentation Template

• Endocrinology: Hypoglycemia Order Set

• Endocrinology: Hypoglycemia Rule

• Occupational Factors Impacting Diabetes

• On this topic but I do not recall exactly which one

3.8D - Mental Health

• Mental Health: Consult for Depression

• Mental Health: Consult for Depression Documentation Template

• Mental Health: Consult for Depression Order Set

• Mental Health: Homelessness Documentation Template

• Mental Health: Iraq & Afghan Post-Deployment Screen Order Set

• Mental Health: Iraq & Afghanistan Post-Deployment Screen - Reminder to Perform Screening Rule

• Mental Health: Iraq & Afghanistan Post-Deployment Screen - Rule to Create Appropriate Order Set Rule

• Mental Health: Iraq & Afghanistan Post-Deployment Screen Documentation Template

• Mental Health: Positive Suicide Risk Screening Order Set

• Mental Health: Suicide Risk Assessment Documentation Template

• On this topic but I do not recall exactly which one

3.8E - Neurology

• Neurology: Traumatic Brain Injury (TBI)

• Neurology: Traumatic Brain Injury (TBI) - Polytrauma Interdisciplinary Rehab Clinic Order Set

• Neurology: Traumatic Brain Injury (TBI) Documentation Template

• Neurology: Traumatic Brain Injury (TBI) Screening Documentation Template

• Neurology: Traumatic Brain Injury (TBI) Screening Rule

• On this topic but I do not recall exactly which one

3.8F - Neurosurgery

• Neurosurgery Radiculopathy (Cervical)

• Neurosurgery Radiculopathy (Lumbar Thoracic)

• Neurosurgery: Radiculopathy (Cervical) Documentation Template

• Neurosurgery: Radiculopathy (Cervical) Order Set

• Neurosurgery: Radiculopathy (Lumbar Thoracic) Documentation Template

• Neurosurgery: Radiculopathy (Lumbar Thoracic) Order Set

• On this topic but I do not recall exactly which one

3.8G - Substance Misuse Disorders (Opioids and Alcohol)

• Factors to Consider in Managing Chronic Pain: A Pain Management Summary

• Interventions for Unhealthy Alcohol Use: SmartSet for Note Documentation, Referrals, and Medications

• Recommendation #10 - Urine Drug Testing

• Recommendation #11: Concurrent Use of Opioids and Benzodiazepines

• Recommendation #4 - Opioid Release Rate When Starting Opioid Therapy

• Recommendation #5 - Lowest Effective Dose

• Recommendation #7 - Opioid Therapy Risk Assessment

• Recommendation #8: Naloxone Consideration

• Screening and Interventions for Unhealthy Alcohol Use: Logic for Best Practice Alerts

• On this topic but I do not recall exactly which one

3.8H- Other

• Anthrax Post-Exposure Prophylaxis

• Refugee Health Decision Support

• Rheumatology: Psoriatic Arthritis Order Set

[If any artifact is selected continue to 3.9]

3.9 Artifact Repository (AR-Names)

[Complete if 3.3 = “To inspect” or “To adapt” AND selected artifacts in 3.8 (AR-topics).]

Based upon the artifacts you just selected, please indicate which you inspected and/or adapted for use in a local environment:

Please select all that apply.

	Inspected for use	Adapted for local use
[Insert 3.8 selection(s)]	⃝	⃝
[Insert 3.8 selection(s)]	⃝	⃝
[Insert 3.8 selection(s)]	⃝	⃝
[Insert 3.8 selection(s)]	⃝	⃝
[Insert 3.8 selection(s)]	⃝	⃝
[Insert 3.8 selection(s)]	⃝	⃝
[Insert 3.8 selection(s)]	⃝	⃝

[If “Adapted for local use” continue to 3.10]

3.10 Artifact Repository (AR-AppType)

[Complete if 3.9 = “To adapt CDS artifacts for use”]

For the artifacts that you adapted for local use please indicate the outcome:

Please select all that apply.

	Adapted for local use as a…			Was unsuccessful at adapting for use in a local environment
	clinician-facing app	patient-facing app	population health tool
Insert 3.9 selection(s)]	⃝	⃝	⃝	⃝
Insert 3.9 selection(s)]	⃝	⃝	⃝	⃝
Insert 3.9 selection(s)]	⃝	⃝	⃝	⃝
Insert 3.9 selection(s)]	⃝	⃝	⃝	⃝
Insert 3.9 selection(s)]	⃝	⃝	⃝	⃝
Insert 3.9 selection(s)]	⃝	⃝	⃝	⃝
Insert 3.9 selection(s)]	⃝	⃝	⃝	⃝

[Continue to 3.11]

3.11 Artifact Repository (AR-Barriers2)

[Complete if 3.10 = “To adapt a CDS artifact for local use”]

Have you ever experienced any issues when adapting the artifact(s) for use in a local environment?

• Yes [Continue to 3.12 ]

• No [Skip to 3.13 if 3.3 = “To Contribute”, otherwise skip to 3.15 AR-Value and follow skip

pattern instructions]

3.12 Artifact Repository (AR-Barriers2Describe)

[Complete if 3.11 = Yes]

Please indicate whether you have ever experienced any of the issues below when adapting the artifact(s) for use in a local environment:

Please answer yes or no to each response option.

	Yes	No
My organization didn’t have the technical capacity or knowledge to adapt it for a local environment	o	o
I am unfamiliar with how to introduce new CDS artifacts to my organization	o	o
There wasn’t enough support within my organization for implementation	o	o
Our EHR vendor only supports CDS that is coded in their proprietary format	o	o

• Other, please specify: _________________

[Continue to 3.13 if 3.3 = “To contribute”. Otherwise, skip to 3.15 and follow the instructions.]

3.13 Artifact Repository (AR-Contribute)

[Complete if 3.3 = “To contribute CDS artifacts”]

Were you able to contribute an artifact(s) to the Repository?

• Yes [Skip to 3.15 ]

• No [Continue to 3.14 ]

3.14 Artifact Repository (AR-ConrtibuteNo)

[Complete if 3.13 = No]

Please select the statement that reflects why you were not able to contribute an artifact(s) to the Repository:

• I couldn’t figure out how to submit an artifact

• I submitted an artifact but it wasn’t accepted

• Other, please specify: _________________

[Continue to 3.15 ]

3.15 Artifact Repository (AR-Value)

[Complete if 2.2 = Artifact Repository]

Please indicate the extent to which you agree or disagree with this statement:

I feel that the Repository is a valuable resource for making evidence-based, shareable CDS publically available.

• Strongly agree

• Agree

• Neither agree nor disagree

• Disagree

• Strongly disagree

[Continue to 3.16 ]

3.16 Artifact Repository (AR-Satisfaction)

[Complete if 2.2 = Artifact Repository]

How would you rate your overall satisfaction with the Repository?

Please consider the overall functionality, ease of use, and quality of information provided.

• Very satisfied

• Satisfied

• Neither satisfied nor dissatisfied

• Dissatisfied

• Very dissatisfied

[Continue to 3.17]

3.17 Artifact Repository (AR-NetPrmt)

[Complete if 2.2 = Artifact Repository]

How likely are you to recommend the Repository to others?

Not likely at all				Extremely likely
1	2	3	4	5

[If 2.2 = Authoring Tool, Continue to 4.1 and follow skip patterns]

[If 2.2 is NOT Authoring too, Skip to 5.1 if GitHub Selected, or 6.4 if not.]

Section 4 – Authoring Tool

4.1 Authoring Tool (AT-How)

You indicated that you have used the Authoring Tool on CDS Connect. The Authoring Tool:

• Leverages Health Level Seven (HL7) Fast Healthcare Interoperability Resources (FHIR)

• Provides an interface for creating CDS logic using simple forms and exporting CDS logic as HL7 Clinical Quality Language (CQL) artifacts

[Complete if 2.2 = Authoring Tool]

How have you used the Authoring Tool?

Please answer yes or no to each response option.

	Yes	No
To learn about the tool	o	o
To produce CQL logic for a CDS artifact	o	o
To test CQL code using synthetic data.	o	o
To teach others about CDS development	o	o

• Other, please specify: _________________

[Continue to 4.2]

4.2 Authoring Tool (AT-Reasons)

[Complete if 2.2 = Authoring Tool]

Have you experienced any issues with using the Authoring Tool?

• Yes [Continue to 4.3 ]

• No [Skip to 4.4 ]
  
  

4.3 Authoring Tool (AT-ReasonsType)

[Complete if 4.2 = Yes]

Please indicate whether you have ever had any of the issues below when using the Authoring Tool:

Please answer yes or no to each response option.

	Yes	No
I need more advanced capabilities than what the Authoring Tool provides	o	o
I need help organizing clinical knowledge into decision logic statements	o	o
I need help understanding FHIR clinical data types and attributes	o	o
I need help identifying, creating, or using existing value sets	o	o
I had problems testing my artifact	o	o

• Other, please specify: _________________

[Continue to 4.4]

4.4 Authoring Tool (AT-TechSupport)

[Complete if 2.2 = Authoring Tool]

Have you ever received any technical support from the CDS Connect team for using the Authoring Tool?

• Yes [Continue to 4.5]

• No [Skip to 4.6]
  
  

4.5 Authoring Tool (AT-TechSatisfaction)

[Complete if 4.4 = Yes]

How satisfied were you with the quality of the technical support?

• Very satisfied

• Satisfied

• Neither satisfied nor dissatisfied

• Dissatisfied

• Very dissatisfied

[If 4.1 “To produce CQL logic for a CDS artifact” = Yes continue to 4.6. Otherwise, skip to 4.9.]

4.6 Authoring Tool (AT-Freq)

[Complete if 4.1 “To produce CQL logic for a CDS artifact” = Yes]

For approximately how many unique CDS artifacts did you use the Authoring Tool to develop either partial or complete CQL logic?
Please consider multiple versions as one artifact.

_______ Artifact(s)

4.7 Authoring Tool (AT-Impl)

[Complete if 4.1 “To produce CQL logic for a CDS artifact” = Yes]

Do you have any plans to use the CDS artifact(s) that includes the CQL logic you produced using the Authoring Tool?

• Yes [Continue to 4.8 ]

• No [Skip to 4.9]

4.8 Authoring Tool (AT-ImplUse)

[Complete if 4.7 = Yes]

What was your goal for using the artifact(s) containing CQL logic created with the Authoring Tool? Was it to…

Please answer yes or no to each response option.

	Yes	No
Implement the artifact(s) as a patient-facing CDS tool	o	o
Implement the artifact(s) as a clinician-facing CDS tool	o	o
Further modify the artifact(s) using different programming software	o	o
Share the artifact(s) on the CDS Connect Repository	o	o
Share the artifact(s) on another repository, such as one hosted by an EHR vendor or professional organization	o	o

• Other, please specify: _________________

[Continue to 4.9]

4.9 Authoring Tool (AT-Value)

[Complete if 2.2 = “Authoring Tool”]

Please indicate the extent to which you agree or disagree with this statement:

I feel that the Authoring Tool is a valuable resource for implementing evidence-based guidelines as computable CDS artifacts.

• Strongly agree

• Agree

• Neither agree nor disagree

• Disagree

• Strongly Disagree

[Continue to 4.10]

4.10 Authoring Tool (AT-Satisfaction)

[Complete if 2.2 = “Authoring Tool”]

How would you rate your overall satisfaction with the Authoring Tool?

Please consider the overall functionality, ease of use, and quality of information provided.

• Very satisfied

• Satisfied

• Neither satisfied nor dissatisfied

• Dissatisfied

• Very dissatisfied

[Continue to 4.11 ]

4.11 Authoring Tool (AT-NetPrmt)

[Complete if 2.2 = “Authoring Tool”]

How likely are you to recommend the Authoring Tool to others involved in CDS development?

Not likely at all				Extremely likely
1	2	3	4	5

[If 2.2 = GitHub, Continue to 5.1 and follow skip patterns]

[If 2.2 is NOT Git Hub, skip to 7.1 CDS Connect Workgroup and follow skip patterns]

Section 5 – GitHub Resources

1. GitHub Resources (GH-Follow)

You indicated that you have used the GitHub Resources on CDS Connect. The GitHub resources are open source and are available to download and use in your own software. Includes;

• CDS Connect Application Program Interface (API) that provides a Representational State Transfer (REST) endpoint for accessing, creating, and modifying CDS artifacts on CDS Connect

• CDS Clinical Quality Language (CQL) Services Tool exposing CQL-based CDS logic through custom RESTful APIs and CDS Hooks APIs

• CQL Testing Framework that allows CQL authors to more easily develop and run test cases for validating CQL-based CDS logic

[Complete if 2.2 = GitHub Resources]

Have you starred or followed any of the following CDS Connect GitHub Resources?

Please answer yes or no to each response option.

	Yes	No
CDS Connect API	o	o
CDS CQL Services Tool	o	o
AHRQ-CDS CQL Testing Framework	o	o
CDS Connect Pain Management Summary SMART on FHIR App	o	o

[Continue to 5.2]

5.2 GitHub Resources (GH-Use)

[Complete if 2.2 = GitHub Resources]

Are you using any of the following CDS Connect GitHub Resources in your own development efforts?

Please answer yes or no to each response option.

	Yes	No
CDS Connect API	o	o
CDS Clinical CQL Services Tool	o	o
AHRQ-CDS CQL Testing Framework	o	o
CDS Connect Pain Management Summary SMART on FHIR App	o	o

[Continue to 5.3]

5.3 GitHub Resources (GH-Value)

[Complete if 2.2 = GitHub Resources]

Please indicate the extent to which you agree or disagree with this statement: 

I feel that it is valuable to have these CDS Connect resources as open source on Github. 

• Strongly agree

• Agree

• Neither agree nor disagree

• Disagree

• Strongly disagree

[Continue to 6.1 and follow instructions regarding additional skip patterns.]

Section 6 – Perspectives on Resources from Non-Users

6.1 Perspectives on Resources from Non-Users (NU-ARHeard)

You indicated that you have NOT used the Artifact Repository resource on CDS Connect. The Artifact Repository:

• Houses evidence-based, shareable CDS artifacts contributed by interested stakeholders

• Includes a searchable inventory of over 50 CDS artifacts that address a variety of domains (e.g., cholesterol management, pain management, and mental health)

In this section we would like to learn more about your reasons for not using the Artifact Repository and thoughts on its potential value.

[Complete if 2.2 = Artifact other or 2.2 or is not = Artifact Repository]

Had you heard of the CDS Connect Artifact Repository prior to taking this survey?

• Yes [Continue to 6.2 ]

• No [Skip to 6.3 and follow skip pattern instructions]
  
  

6.2 Perspectives on Resources from Non-Users (NU-ARUse)

[Complete if 6.1 = Yes]

Please indicate whether any of these statements are reasons for why you have not used the CDS Connect Artifact Repository.

Please answer yes or no to each response option.

I did not use the CDS Connect Artifact Repository because…

	Yes	No
My role doesn’t require me to access it	o	o
Artifacts in the repository aren’t relevant to my organization’s needs	o	o
None of the artifacts in the repository are executable in my EHR	o	o
I didn’t have the technical capacity or knowledge to integrate the artifacts in the Repository into my clinical system	o	o
I’m unfamiliar with how to introduce new CDS to my organization	o	o

• I prefer to use another repository, please name: _________________

[Continue to 6.3 regardless of response.]

6.3 Perspectives on Resources from Non-Users (NU-ARValue)

[Complete if 6.1 = Yes or No]

Please indicate the extent to which you agree or disagree with the following statement:

I believe that the CDS Connect Artifact Repository could be valuable to me/my organization.

• Strongly agree

• Agree

• Neither agree not disagree

• Disagree

• Strongly Disagree

[Continue to 6.4 and follow skip pattern instructions]

6.4 Perspectives on Resources from Non-Users (NU-ATHeard)

[Complete if 2.2 = Artifact other or 2.2 (is not = Authoring Tool]

Had you heard of the CDS Connect Authoring Tool prior to taking this survey?

• Yes [Continue to 6.5]

• No [Skip to 6.6 ]

6.5 Perspectives on Resources from Non-Users (NU-ATUse)

[Complete if 6.4 = Yes]

Please indicate whether any of these statements are reasons for why you have not used the CDS Connect Authoring Tool:

Please answer yes or no to each response option.

I did not use the CDS Connect Authoring Tool because…

	Yes	No
My role doesn’t require me to develop CDS CQL logic	o	o
I need more advanced programming capability than what the Authoring Tool provides	o	o
I need additional training on how to use the Authoring Tool	o	o
I use a proprietary CQL tool to develop CQL logic for CDS	o	o
I use another publicly available CQL tool to develop CQL logic for CDS: please name_________	o	o
Most commercial electronic health record vendors do not use the CQL standard	o	o

• Other, please specify: _________________

[Continue to 6.6 regardless of response.]

6.6 Perspectives on Resources from Non-Users (NU-ATValue)

[Complete if 6.4 = Yes or No]

Please indicate your level of agreement with the following statement:

I believe the CDS Connect Authoring Tool could be valuable to me/my organization.

• Strongly agree

• Agree

• Neither agree nor disagree

• Disagree

• Strongly Disagree

[Continue to 7.1 ]

Section 7 – CDS Connect Workgroups

7.1 CDS Connect Workgroups (WG-Attd)

The monthly Work Group advises the CDS Connect team on the ongoing identification and prioritization of key features and capabilities for CDS Connect, leveraging the expertise of work group members. In this section we will ask you about your experiences with the Workgroups, if applicable.

[All respondents]

Did you ever participate in the monthly CDS Connect Workgroup organized by MITRE?

• Yes [Continue to 7.2]

• No [Skip to 8.1]

7.2 CDS Connect Workgroups (WG-Reason)

[Complete if 7.1 = Yes]

What did you do in the workgroup?

Please answer yes or no to each response option.

	Yes	No
Learned about the development and functionality of the CDS Connect resources	o	o
Provided feedback on the development and functionality of the CDS Connect resources	o	o
Met other CDS developers	o	o

• Other, please specify: _________________

[Continue to 7.3 regardless of response]

7.3 CDS Connect Workgroups (WG-Value)

[Complete if 7.1 = Yes]

Please indicate the extent to which you agree or disagree with this statement:

I valued the CDS Connect Workgroup as a way to learn about developing, using and sharing evidence-based CDS with others.

• Strongly agree

• Agree

• Neither agree nor disagree

• Disagree

• Strongly Disagree

[Continue to 8.1 ]

Section 8 – Other AHRQ CDS Initiatives

8.1 Other AHRQ CDS Initiatives (OACI-Heard)

• In 2016, AHRQ launched the Patient Centered Outcomes Research (PCOR) Clinical Decision Support (CDS) Initiative aimed at helping health care providers move PCOR evidence into practice through CDS.

• In addition to CDS Connect, the Initiative included the Patient Centered CDS Learning Network and other CDS development and pilot testing grant opportunities.

[All respondents]

Have you heard about the AHRQ PCOR CDS Initiative prior to taking this survey?

• Yes [Continue to 8.2]

• No [Skip to 8.4]
  
  

8.2 Other AHRQ CDS Initiatives (OACI-Find)

[Complete if 8.1 = Yes]

How did you find out about the AHRQ PCOR CDS Initiative? Did you find out about it...

Please answer yes or no to each response option.

	Yes	No
Through AHRQ’s on-line and social media	o	o
Through professional societies or affiliated meetings	o	o
Through other government agencies websites or programs	o	o
Through personal/professional contacts	o	o
Through a Connectathon	o	o

• Other, please specify: _________________

[Continue to 8.3]

8.3 Other AHRQ CDS Initiatives (OACI-Components)

[All respondents]

Have you been involved in any of the following components of the AHRQ PCOR CDS initiative?

Please answer yes or no to each response option.

	Yes	No
Patient Centered (PC) CDS Learning Network (LN)	o	o
Quantifying Efficiencies Gained through Shareable Clinical Decision Support Resources	o	o
Advancing Evidence into Practice through Shared, Interoperable Clinical Decision Support Resources (U18)	o	o
Opioid-Related CDS Development Pilots	o	o

• Other, please specify: _________________

[Continue to 8.4]

8.4 Other AHRQ CDS Initiatives (OACI-Initiatives)

[All respondents]

Have you been involved in any other collaborative initiatives working to advance the field of evidence-based CDS development besides the AHRQ PCOR CDS Initiative (e.g. CDC, vendor specific, health system specific)?

• Yes [Continue to 8.5 ]

• No [Skip to 9.1]

8.5 Other AHRQ CDS Initiatives (ACI-Open Response)

[Complete if 8.4 = Yes]

Please list the names of the other collaborative CDS initiatives and the agencies/companies that are leading them.

[Continue to 9.1]

Section 9 – Referral: User Contact Information

9.1 Referral: User Contact Information (CI-Name)

In order to better understand different perspectives of CDS Connect users we ask that you identify 3 users to also receive this survey.

The individuals you select should be from your professional network who you believe may have used the CDS Connect resources. These individuals may include clinicians, informaticists, developers, etc.

With your permission, the individuals you identify will receive an email notifying them that you have identified them as another CDS Connect user along with a link to the survey.

For this portion of the survey, it may be easier for you to input the contact information on a computer rather than a tablet or mobile device.

Please provide the name and email address for each recommended respondent.

First Name: ____________________ [Continue to 9.2]

Last Name: ____________________ [Continue to 9.2]

Email Address: ____________________ [Continue to 9.2]

First Name: ____________________

Last Name: ____________________

Email Address: ____________________

First Name: ____________________

Last Name: ____________________

Email Address: ____________________

• I don’t know anyone else who may have used CDS Connect resources [Skip to 10 /terminate]

• I’m unable to provide names or emails at this time. [Skip to 10 /terminate]

• Please send me a follow up e-mail so I can provide contact information. My e-mail address is: ___________________________ [Skip to 10 /terminate]

9.2 Referral: User Contact Information (CI-Share)

[Complete if contact information is provided in 9.1]

May we inform the individuals that you identified that you referred them to us?

• Yes [Continue to 9.3]

• No [Skip to 10 /terminate]
  
  

9.3 Referral: User Contact Information (CI-Org)

[Complete if 9.2 = yes]

Please let us know the name of your organization if you believe it would be helpful to include it when contacting these individuals.

[Continue to 10 /termination]

Section 10 – End/Termination

Thank you for your time. For more information about AHRQ’s CDS initiative, please visit https://cds.ahrq.gov/.