CMS word template

Status: Original and derived artifacts are available for this document.

Download: docx | pdf | html

Focus Group Session Moderator Guide Script
See below for the main moderator language and follow-on moderator language.
Instructions
     Introductory text written in black should be read to participants verbatim.
The focus group session wording is intended as a guide for the focus group moderator. Blue sub-bullets are included as supplemental probes for the moderator or moderator support to use, if needed. A Mural Board will be leveraged to guide the focus group sessions. 
The Mural board will obtain questions for participants to answer. When participants enter the Mural board, they will be assigned a visiting animal and not enter their name. The questions will be located on the board with preset stickies for participants to select, document response, etc. The stickies will be color coded based on researcher classification. A ledger will be posted on the board for participants to know which color sticky they should click and drag. The moderator will lead the questions and the moderator or moderator support will ask the probing blue questions as participants document responses. 

Introduction [15 Minutes]
Welcome everyone! Thank you for participating in our focus group session. We recognize you all have very busy schedules, so we appreciate you making time. This session will be recorded and may take up 2-hours depending on the conversation and engagement. 

My name is ______ and I am joined by my colleague(s) ________ from the Centers for Medicare & Medicaid Services (CMS) to better understand your experience with the CCW VRDC. My colleague(s) will be taking notes and may add any questions, or comments along the way. In this session, we would like to understand your research experience, desired capabilities, and any recommendations for improvements. This session is not to discuss the Data Use Agreement (DUA) process issues or overall data request process bottlenecks. 

Your responses in this focus group session will help CMS learn about:
    • Researcher experiences with, and perspectives on, what happens once you obtain access.
    • Researcher experiences with navigating your access.
    • Researcher experiences with managing storage.
    • Researcher experiences with extracting data.
    • Researcher pain points and so much more.

Before we begin, I need to read a brief disclosure statement that is also presented on the screen: 
According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number.  The valid OMB control number for this information collection is 0938-1185 (Expires XX/XX/XXXX).  This is a voluntary information collection.  The time required to complete this information collection is estimated to average one-hour per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection.  If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: CMS, 7500 Security Boulevard, Attn: PRA Reports Clearance Officer, Mail Stop C4-26-05, Baltimore, Maryland 21244-1850. Please do not send applications, claims, payments, medical records or any documents containing sensitive information to the PRA Reports Clearance Office.  Please note that any correspondence not pertaining to the information collection burden approved under the associated OMB control number listed on this form will not be reviewed, forwarded, or retained. If you have questions or concerns regarding where to submit your documents, please contact Kristy Johnson.

Focus Group Session
Section 1: Mural Board Instructions/Exercise [20 Minutes] 
Using the link posted in the chat, please access the Mural board for this session. You may leave your name blank as you will be designated a visiting animal. There are roughly 18 questions we plan to cover. For each question there are pre-populated stickies for you to select depending on your CCW VRDC user type. Please reference the key in the introduction section or refer to the type identified above the sticky note. When you would like to record feedback, click and drag the sticky associated with your user type to any location within the designated section. You do not need to keep your feedback aligned with the original location of the sticky note. Some of you may need to use more than one sticky color, as some of you may have multiple roles and participate on multiple organizations. If you need help determining which classification you should select, please do not hesitate to ask.

Now, we want to practice using the Mural Board with a couple of questions about your experiences using the CCW VRDC. For purposes of our discussion, we’re focusing on your user experience pertaining to accessing the CCW VRDC, not the DUA or data request experiences. 
    1. How long have you been using the VRDC? 
    2. How frequently do you access the VRDC (daily, weekly, etc.)? 
    3. How did you find out about the VRDC?
        3.1. What was your experience like learning about the VRDC?
Now let’s review all our comments on the board and identify how we feel about them. Using the agree, disagree, requires more information, or most important icons take a moment to review everyone’s feedback and place your icon rating on the cards for each question. You may use as many icons as you would like, and you may use more than one icon on any specific note. Do you feel this is accurate? Are there any details we should add? Click the icon you feel most identifies with your thoughts and place it on the edge of the sticky note.
Section 2: Onboarding [10 Minutes] 
    1. Tell us about your VRDC Onboarding experience?
        1.1. What was it like getting access to the VRDC?
        1.2. How long did it take you to get access to the VRDC?
        1.3. Were there challenges getting access?
        1.4. Did you have any issues with the Onboarding requirements (software installation, security training)?
        1.5. Is the VRDC what you expected?
    2. Can you tell us about your experience navigating and using the CCW website?
        2.1. Were you easily able to find the resources and information you are looking for?
        2.2. How could the websites be more user-friendly?
    3. What issues or pain points did you encounter during your VRDC onboarding?
    4. Are there any changes you would recommend the VRDC onboarding process? 
Now let’s review all our comments on the board and identify how we feel about them. Using the agree, disagree, requires more information, or most important icons take a moment to review everyone’s feedback and place your icon rating on the cards for each question. You may use as many icons as you would like, and you may use more than one icon on any specific note. Do you feel this is accurate? Are there any details we should add? Click the icon you feel most identifies with your thoughts and place it on the edge of the sticky note.
Section 3: Navigating [30 Minutes] 
    5. Now tell us about your experience navigating the VRDC?
        5.1. Do you find the VRDC documentation/training resources beneficial? 
        5.2. Can you find the data you need? 
        5.3. How did you learn about intricacies of how and where the data can and cannot be transferred?
        5.4. How do you feel about the design and organization?
        5.5. What do you like, or not like about navigating the VRDC?
    6. Did you encounter any problems navigating the remote desktop?
        6.1. What do you think about the browser and other desktop tools within the VRDC?
        6.2. How do you feel about the VRDC’s virtual desktop, speed, organization, applications, etc.? 
    7. How did you learn about the data files?
        7.1. Were you able to find the files within the system?
        7.2. Were you able to find the data documentation?
        7.3. Did you have the access you were expecting (files/software)?
    8. Were you able to locate and use the VRDC training materials?
        8.1. Do you find the VRDC documentation/training resources beneficial?
        8.2. Were there training materials you needed that were not available?
        8.3. Do you have any other suggestions for improving out training offerings?
    9. Have you used the VRDC User Dashboards?
        9.1. Did you find the information on the dashboards easy to locate/follow?
        9.2. Do you have any recommendations for other information you would like to see on the dashboards?
    10. What issues or pain points do you encounter navigating the VRDC?
        10.1. How could the VRDC be more user-friendly?
        10.2.  Any thoughts or suggestions for improvements on navigating the environment?
        10.3.  In a perfect world, what would the VRDC look like to you?
Now let’s review all our comments on the board and identify how we feel about them. Using the agree, disagree, requires more information, or most important icons take a moment to review everyone’s feedback and place your icon rating on the cards for each question. You may use as many icons as you would like, and you may use more than one icon on any specific note. Do you feel this is accurate? Are there any details we should add? Click the icon you feel most identifies with your thoughts and place it on the edge of the sticky note.
Section 4: Analysis/Research [20 Minutes]  
    11. Can you tell us about your experience conducting your analysis in the VRDC? 
        11.1. What has worked well and what has not worked well while using the VRDC for your analysis (please indicate SAS/Databricks/etc.)?
        11.2. What do you think are the benefits and challenges of using the VRDC vs. other CMS systems?
    12. Tell us about your experience uploading/downloading?
        12.1. Any issues uploading data to your VRDC folder?
        12.2. Any concerns with the Output Review process?
        12.3. Any issue downloading data once output review has completed?
    13. What is the biggest challenge you face with the software available in the VRDC?
        13.1. Databricks, Analytic containers, SAS?
        13.2. Is there other software you would like to see in the VRDC?
        13.3. What makes the VRDC difficult to use?
    14. Have you used the CCW Help Desk?
        14.1. Was your issue solved satisfactorily?
        14.2. Was the Help Desk responsive to your request?
    15. Do you have any other suggestions for improving the VRDC analytic environment?
Now let’s review all our comments on the board and identify how we feel about them. Using the agree, disagree, requires more information, or most important icons take a moment to review everyone’s feedback and place your icon rating on the cards for each question. You may use as many icons as you would like, and you may use more than one icon on any specific note. Do you feel this is accurate? Are there any details we should add? Click the icon you feel most identifies with your thoughts and place it on the edge of the sticky note.
Section 5: Final Thoughts [20 Minutes]  
    16. How would you like to receive or be notified of maintenance outages?
    17. How would you like to receive information about new enhancements or features to the VRDC?
        17.1. Do you feel you are receiving the information you need about VRDC changes and outages?
    18. Is there anything else you would like to add or ask that we haven't covered?
        18.1. Tell me about your ideas or any recommendations you may have? 
        18.2. If you could decide on the changes, what would you want?
        18.3. Is there anything else I should have asked?
Now let’s review all our comments on the board and identify how we feel about them. Using the agree, disagree, requires more information, or most important icons take a moment to review everyone’s feedback and place your icon rating on the cards for each question. You may use as many icons as you would like, and you may use more than one icon on any specific note. Do you feel this is accurate? Are there any details we should add? Click the icon you feel most identifies with your thoughts and place it on the edge of the sticky note.
Conclusion [5 Minutes]  
Thank you very much for taking the time to talk with us and for sharing your thoughts! We have several other focus group sessions to complete, once we have collated all participants feedback from every session, we will take time to digest what you all are saying and provide you an update. Once you are finished recording you feedback and rating all the feedback documented on the Mural board, please take a moment to leave us feedback on the board. Once you are finished leaving use feedback feel free to exit the session. If you think of anything after today, please send any additional thoughts to _Kristy Johnson, her email was reposted in the chat [email protected].
Instrument: Mural board template screen shot below








Announcement “Join Our Focus Group” 
Sender: CCW Mailbox
Subject: Help Make the CCW VRDC Better by Joining Our Focus Group!
Body: Hi VRDC User,

To gain a better perspective on our user’s needs, we are asking for your participation in a VRDC focus group. The focus group will last no longer than two hours and will provide us with an opportunity to learn more about your experience as a VRDC user. We would like to discover what is going well, what could be improved, and what future enhancements you would like to see. Your views and the information collected from the focus groups will help design and prioritize future VRDC enhancements.  
 
If you would like to participate in a VRDC focus group please contact Kristy Johnson ([email protected]) by COB MM/DD/YYYY.
 
See the schedule of dates and times for our focus group sessions below. Please indicate so in your response to Kristy Johnson which session you would like to attend. 
 
 
Thank you, 

CCW Help Desk
[email protected] 
1-866-766-1915 | Toll Free
1-515-440-3159 | Fax
www.ccwdata.org

Email Invitation “Focus Group Session Invitation”
Sender: Kristy Johnson 
Subject:  CCW VRDC Focus Group Invitation for DATE from TIME 
Body: Hi,

Thank you for volunteering to participate in our VRDC Focus Group! Our focus group session will consist of a Mural board exercise that gives users a chance to document and see all responses from the participants in the session. At the end, each participant will be able to rate and add feedback on all recorded comments. 

We have a few recommendations listed below to prepare you for this focus group session. Please take some time to review so this focus group session can be as seamless as possible.

    1. Become briefly familiar with Mural.
    2. Review the topics we plan to ask you.
    3. Review the Mural Board to get familiar.

Not familiar with Mural? Click here https://www.mural.co/ or review the quick tutorial links below:

https://support.mural.co/s/
https://www.youtube.com/watch?v=zA1RcPp19w4. 
https://www.youtube.com/watch?v=4IT_Q9Pg9jU

During the session, you will provide your feedback on color-coded sticky notes.  All feedback recorded will be kept confidential and transcribed as an anonymous user for the Mural board exercise. We can only spend a certain amount of time on each topic and suggest taking time to review the Mural board and questions asked so you have an idea of the agenda and topics covered.

See the list below for the topics covered in this focus group session:
    • Onboarding experience.
    • Experience navigating the VRDC.
    • What you would like to see in the VRDC.
    • Using the VRDC to conduct your research.
    • VRDC challenges or pain points.
    • Recommendations for improving the VRDC.

If you would like to get a sneak peak of the Mural board, click here INSERT LINK 

If you have any questions, please do not hesitate to reach out to me directly.

Thank you,

Kristy Johnson