CMS-10800 Gen IC

Request for Approval under the “Generic Clearance for the Center for Clinical Standards and Quality IT Product and Support Teams” (OMB Control Number: 0938-1397)

TITLE OF INFORMATION COLLECTION: Single Ease Question for User Insights on Hospital Quality Reporting (CMS-10800)

PURPOSE OF COLLECTION:

What are you hoping to learn / improve? How do you plan to use what you learn? Are there artifacts (user personas, journey maps, digital roadmaps, summary of customer insights to inform service improvements, performance dashboards) the data from this collection will inform? 

Hospital Quality Reporting has moved from a delivery-oriented approach to a product management approach for feature prioritization and roadmap development. As part of this shift, we want to provide our product managers with user experience (UX) feedback regarding our products' ease of use. We would like to use TouchPoints to serve our users a Single Ease Question (SEQ) as a pop-up survey in a modal window after completing major activities in the system. This SEQ will ask them to rate how easy or difficult it was to complete the task according to a Likert scale. Our goal is to benchmark these ratings over time and flag specific user flows that, based on user ratings, may need further qualitative research or analysis.

TYPE OF COLLECTION: (Check one)

[ ] Card Sorting [ ] Cognitive Testing

[ ] Field Studies [ ] First Click Tests

[ ] Focus Groups [ ] Participatory Design

[X] Survey [ ] Tree Testing

[ ] User Interviews [ ] Usability Testing

[ ] Other: _______________________

CERTIFICATION:

I certify the following to be true:

1. The collection is voluntary.

2. The collection is low-burden for respondents and low-cost for the Federal Government.

3. The collection is non-controversial and does not raise issues of concern to other federal agencies.

4. The results are not intended to be disseminated to the public.

5. Information gathered will not be used for the purpose of substantially informing influential policy decisions.

6. The collection is targeted to the solicitation of opinions from respondents who have experience with the program or may have experience with the program in the future.

Name: __Adebola Adeleye (Product Lead for HQR HCD)_____________________

To assist review, please provide answers to the following question:

PERSONALLY IDENTIFIABLE INFORMATION

1. Is personally identifiable information (PII) collected? [ ] Yes [ X] No

2. If Yes, will any information that is collected be included in records that are subject to the Privacy Act of 1974? [ ] Yes [X] No

3. If Yes, has an up-to-date System of Records Notice (SORN) been published? [ ] Yes [ ] No

GIFTS OR PAYMENTS

Is an incentive (e.g., money or reimbursement of expenses, token of appreciation) provided to participants? [ ] Yes [ X ] No

If Yes, describe:

BURDEN HOURS

Category of Respondent	No. of Respondents	Participation Time	Burden
(2) Private Sector (HQR End Users)	2,000	.166 hour	6 hours
Totals	2,000	.166 hour	6 hours

FEDERAL COST

The estimated annual cost to the Federal government is ____________
There is no additional cost for this survey implementation, as it is part of the Hospital Quality Reporting contract scope that has already been awarded.

ACTIVITY DETAILS

1. How will you collect the information? (Check all that apply)

[ X] Web-based or other forms of social media

[ ] Telephone

[ ] In-person

[ ] Mail

[ ] Other, Explain.

2. Will interviewers or facilitators be used? [ ] Yes [ X] No

3. Who will you collect the information from?

Describe the people you will interact with or collecting information from and why the group is appropriate for the program / service to connect with. Please provide a description of how you plan to identify your potential group of respondents and if only a sample will be solicited for feedback, how you will select them (e.g., anyone who provided an email address to a call center representative, a representative sample of administrators who downloaded a report in May 2021, intercept interviews at a particular field office, a list of customers, e.g., a CRM database that has contact information, to reach out to that defines the universe of potential respondents and have a sampling plan for selecting from this universe). Attach a copy of your sampling plan if applicable.

We are proposing to implement the Touchpoints tool on Hospital Quality Reporting (HQR) so that we can collect feedback from our users about how they rank their ease of use after completing key activities in the system. These users interact directly with HQR and consist of hospital quality directors and specialists who oversee HQR program requirements, third party vendors who submit data on behalf of these hospitals, and internal CMS contractors who perform data validation and administrative tasks for various HQR programs.

Collecting their feedback data will allow us to benchmark high traffic user flows and identify interactions that create unnecessary burden for users. We will sample approximately ten percent of users who log into HQR and complete a key function in the system. Below are some example activities that may trigger an SEQ modal to appear.

Example flows:

• Submitting data via a data form

• Submitting data via file upload

• Downloading a public report

• Submitting/updating a notice of participation

• Adding a new user to the user’s organization

• Modifying an existing user’s permissions

• Adding a new vendor to your organization

• Modifying vendor permissions

4. How will you ask a respondent to provide this information?

For example, after an inquiry is submitted online, the final screen will present the opportunity to provide feedback by presenting a link to a feedback form / an actual feedback form.

When a randomly selected HQR user logs in to the HQR system and completes a key task, they will be presented with a pop-up modal that says “Overall, how easy or difficult did you find this task using Hospital Quality Reporting?”

5. What will the activity look like?

Describe the information collection activity – e.g., what happens when a person agrees to participate? Will facilitators or interviewers be used? What is the format of the interview/focus group? If a survey, describe the overall survey layout/length/other details. If User Testing, what actions will you observe / how will you have respondents interact with a product you need feedback on. 

As described above, once the user is presented with the Single Ease Question (SEQ) in the pop-up modal, they will be asked to provide a rating on a scale from 1 (Very difficult) to 5 (Very easy) by selecting the radio button that corresponds to their ease of use. Once their rating is submitted, they will see a new message in the modal thanking them for their feedback. Users also have the option to close out of this modal if they wish not to provide their feedback.

6. Please provide your question list.

Paste here the questions or prompts presented to participants in your activity. If you have an interview / facilitator guide, that can be attached to the submission and referenced here.

Our Touchpoints implementation will feature a single ease question (SEQ):

• “Overall, how easy or difficult did you find this task using the Hospital Quality Reporting system?”

  • 1- Very difficult

  • 2- Somewhat difficult

  • 3 - Neither difficult nor easy; neutral

  • 4 - Somewhat easy

  • 5 - Very easy

Please make sure that all instruments, instructions, and scripts are submitted with the request.

7. When will the activity happen?

Describe the time frame or number of events that will occur (e.g., We will conduct focus groups on May 13,14, 15; We plan to conduct customer intercept interviews over the course of the Summer at the field offices identified in response to #2 based on scheduling logistics concluding by Sept. 10; or This survey will remain on our website in alignment with the timing of the overall clearance.)

We will be implementing the Touchpoints SEQ with HQR end users beginning in the spring of 2022 (March/April timeframe). This timeline is contingent on PRA approval and developer capacity to complete the Touchpoints implementation in production. We will keep the survey live on HQR until we hit our target goal of 2,000 responses (approximately ten percent of HQR’s number of registered users). Once we attain 2,000 responses we will analyze the responses and incorporate the results into our benchmarks for user experience and customer satisfaction improvement metrics.

Instructions for completing Request for Approval under the “Generic Clearance for the Center for Clinical Standards and Quality IT Product and Support Teams”

TITLE OF INFORMATION COLLECTION: Provide the name of the collection that is the subject of the request (e.g. Comment card for soliciting feedback on xxxx).

PURPOSE: Provide a brief description of the purpose of this collection and how it will be used. If this is part of a larger study or effort, please include this in your explanation.

DESCRIPTION OF RESPONDENTS: Provide a brief description of the targeted group or groups for this collection of information. These groups must have experience with the program.

TYPE OF COLLECTION: Check one box. If you are requesting approval of other instruments under the generic, you must complete a form for each instrument.

CERTIFICATION: Please read the certification carefully. If you incorrectly certify, the collection will be returned as improperly submitted or it will be disapproved.

Personally Identifiable Information: Provide answers to the questions. Note: Agencies should only collect PII to the extent necessary, and they should only retain PII for the period that is necessary to achieve a specific objective.

Gifts or Payments: If you answer yes to the question, please describe the incentive, and provide a justification for the amount.

BURDEN HOURS:

Category of Respondents: Identify who you expect the respondents to be in terms of the following categories: (1) Individuals or Households; (2) Private Sector; (3) State, local, or tribal governments; or (4) Federal Government. Only one type of respondent can be selected per row.

No. of Respondents: Provide an estimate of the Number of respondents.

Participation Time: Provide an estimate of the amount of time required for a respondent to participate (e.g., fill out a survey or participate in a focus group).

Burden: Provide the Annual burden hours: Multiply the Number of responses and the participation time and divide by 60.

FEDERAL COST: Provide an estimate of the annual cost to the Federal government.

If you are conducting a focus group, survey, or plan to employ statistical methods, please provide answers to the following questions:

The selection of your targeted respondents. Please provide a description of how you plan to identify your potential group of respondents and how you will select them. If the answer is yes, to the first question, you may provide the sampling plan in an attachment.

Activity Details: Identify how the information will be collected. More than one box may be checked. Indicate whether there will be interviewers (e.g. for surveys) or facilitators (e.g., for focus groups) used.

Submit all instruments, instructions, and scripts are submitted with the request.

8