CMS-10682.CMS_OCS_GenericClearanceRequest_SitemapTesting

Request for Approval under the “Generic Clearance for the Collection of Routine Customer Feedback” (OMB Control Number: 0938-NEW)

TITLE OF INFORMATION COLLECTION:

Unmoderated Online User Testing of a Proposed New CMS.gov Website Structure

PURPOSE:

The Web and New Media Group within the Office of Communications at the Center for Medicare and Medicaid (CMS) has recently drafted a new site structure for their flagship website, CMS.gov. They want to gather feedback on this site structure from the users of CMS.gov, who include health care providers, health insurance brokers, and individuals who help beneficiaries select health insurance in state exchanges. The objectives of this round of testing are:

• Determine the effectiveness of the new CMS.gov site structure and navigation labels

• Ensure that end users of CMS.gov can complete specific core tasks efficiently and effectively

DESCRIPTION OF RESPONDENTS:

End users of the website CMS.gov, who include health care providers, health insurance brokers, and individuals who help beneficiaries select health insurance in state exchanges. We will send an e-mail invitation to those who have signed up for updates on CMS.gov, i.e., those who have subscribed to the GovDelivery e-mail newsletter service on CMS.gov. Participation is purely voluntary and based on each respondent’s interest in contributing to the improvement of CMS.gov, which many of the target audience visit frequently and rely on to perform job-related tasks.

TYPE OF COLLECTION: (Check one)

[ ] Customer Comment Card/Complaint Form [ ] Customer Satisfaction Survey

[X] Usability Testing (e.g., Website or Software) [ ] Small Discussion Group

[ ] Focus Group [ ] Other:

CERTIFICATION:

I certify the following to be true:

1. The collection is voluntary.

2. The collection is low-burden for respondents and low-cost for the Federal Government.

3. The collection is non-controversial and does not raise issues of concern to other federal agencies.

4. The results are not intended to be disseminated to the public.

5. Information gathered will not be used for the purpose of substantially informing influential policy decisions.

6. The collection is targeted to the solicitation of opinions from respondents who have experience with the program or may have experience with the program in the future.

Name: Michael McCann_________________________________

To assist review, please provide answers to the following question:

Personally Identifiable Information:

1. Is personally identifiable information (PII) collected? [ ] Yes [X] No

2. If Yes, will any information that is collected be included in records that are subject to the Privacy Act of 1974? [ ] Yes [ ] No

3. If Yes, has an up-to-date System of Records Notice (SORN) been published? [ ] Yes [ ] No

Gifts or Payments:

Is an incentive (e.g., money or reimbursement of expenses, token of appreciation) provided to participants?

[ ] Yes [X] No

BURDEN HOURS

Category of Respondent	No. of Respondents	Participation Time	Burden
Individuals in the private sector (who use CMS.gov)	50	15 minutes	12.5 hours
Totals	50	15 minutes	12.5 hours

FEDERAL COST: The estimated annual cost to the Federal government is: $25,000. This is the cost of the contracted labor costs. There is no cost for licensing of software.

If you are conducting a focus group, survey, or plan to employ statistical methods, please provide answers to the following questions:

The selection of your targeted respondents

1. Do you have a customer list or something similar that defines the universe of potential respondents and do you have a sampling plan for selecting from this universe? [X] Yes [ ] No

If the answer is yes, please provide a description of both below (or attach the sampling plan)? If the answer is no, please provide a description of how you plan to identify your potential group of respondents and how you will select them?

We will send an e-mail invitation to those who have signed up for updates on the CMS.gov website. That is, those who have subscribed to CMS’s GovDelivery e-mail newsletter service via the CMS.gov website. Those who have done so are usually those who visit the CMS.gov website frequently and who are interested in keeping up with the information that the site contains.

The test we want to conduct is an unmoderated, online test that can be completed by any person to whom we send an e-mail invitation. Respondents will be asked to respond to two multiple choice questions upfront, then they will complete a series of 7 tasks by navigating our proposed site hierarchy using a tool called Treejack (from Optimal Workshop). Tasks are completed by finding the specific page in the hierarchy where they believe they can get what they need. The last part of the test includes three optional questions that prompt users to provide brief written commentary about the new site structure. Each test should take no longer than 10–20 minutes.

Theoretically, everyone on the e-mail list could respond to our invitation and take the test. We expect, however, to get about 30 responses and not more than 50 responses. There is no need for us to personally select individual participants via a sampling plan or otherwise.

Administration of the Instrument

1. How will you collect the information? (Check all that apply)

[X] Web-based unmoderated testing platform

[ ] Telephone

[ ] In-person

[ ] Mail

[ ] Other, Explain

2. Will interviewers or facilitators be used? [ ] Yes [X] No

Please make sure that all instruments, instructions, and scripts are submitted with the request.

Instructions for completing Request for Approval under the “Generic Clearance for the Collection of Routine Customer Feedback”

TITLE OF INFORMATION COLLECTION: Provide the name of the collection that is the subject of the request. (e.g., Comment card for soliciting feedback on xxxx)

PURPOSE: Provide a brief description of the purpose of this collection and how it will be used. If this is part of a larger study or effort, please include this in your explanation.

DESCRIPTION OF RESPONDENTS: Provide a brief description of the targeted group or groups for this collection of information. These groups must have experience with the program.

TYPE OF COLLECTION: Check one box. If you are requesting approval of other instruments under the generic, you must complete a form for each instrument.

CERTIFICATION: Please read the certification carefully. If you incorrectly certify, the collection will be returned as improperly submitted or it will be disapproved.

Personally Identifiable Information: Provide answers to the questions. Note: Agencies should only collect PII to the extent necessary, and they should only retain PII for the period of time that is necessary to achieve a specific objective.

Gifts or Payments: If you answer yes to the question, please describe the incentive and provide a justification for the amount.

BURDEN HOURS:

Category of Respondents: Identify who you expect the respondents to be in terms of the following categories: (1) Individuals or Households;(2) Private Sector; (3) State, local, or tribal governments; or (4) Federal Government. Only one type of respondent can be selected per row.

No. of Respondents: Provide an estimate of the Number of respondents.

Participation Time: Provide an estimate of the amount of time required for a respondent to participate (e.g. fill out a survey or participate in a focus group)

Burden: Provide the Annual burden hours: Multiply the Number of responses and the participation time and divide by 60.

FEDERAL COST: Provide an estimate of the annual cost to the Federal government.

If you are conducting a focus group, survey, or plan to employ statistical methods, please provide answers to the following questions:

The selection of your targeted respondents. Please provide a description of how you plan to identify your potential group of respondents and how you will select them. If the answer is yes, to the first question, you may provide the sampling plan in an attachment.

Administration of the Instrument: Identify how the information will be collected. More than one box may be checked. Indicate whether there will be interviewers (e.g. for surveys) or facilitators (e.g., for focus groups) used.

Submit all instruments, instructions, and scripts with the request.

7