CMS-10710.SupportingStatementPartA(30-day)

CMS-10710; 0938-1382

SUPPORTING STATEMENT A

FOR PAPERWORK REDUCTION ACT SUBMISSION

Generic Clearance for Improving Customer Experience

(OMB Circular A-11, Section 280 Implementation)

CMS-10710; OMB control number 0938-1382

1. Explain the circumstances that make the collection of information necessary. Identify any legal or administrative requirements that necessitate the collection. Attach a hard copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information, or you may provide a valid URL link or paste the applicable section. Specify the review type of the collection (new, revision, extension, reinstatement with change, reinstatement without change). If revised, briefly specify the changes. If a rulemaking is involved, make note of the sections or changed sections, if applicable.

On September 11, 1993, President Clinton issued Executive Order 12862, “Setting Customer Service Standards” which clearly define his vision that the Federal agencies will put the people first. Executive Order 12862 directs Federal agencies to provide service to the public that matches or exceeds the best service available in the private sector. Section 1(b) of Executive Order 12862 requires government agencies to “survey customers to determine the kind and quality of services they want and their level of satisfaction with existing services” and Section 1(e) requires agencies “survey front-line employees on barriers to, and ideas for, matching the best in business.”

On March 30, 2016, President Obama established the Core Federal Services Council, which again emphasized the need to deliver world-class customer service to the American people. The Council, composed of the major high-volume, high-impact Federal programs that provide transactional services directly to the public, were encouraged “to improve the customer experience by using public and private sector management best practices, such as conducting self-assessments and journey mapping, collecting transactional feedback data, and sharing such data with frontline and other staff.”

In March 2018, the Administration of President Trump launched the President’s Management Agenda (PMA) and established new Cross-Agency Priority (CAP) Goals. Excellent service was established as a core component of the mission, service, stewardship model that frames the entire PMA, embedding a customer-focused approach in all of the PMA’s initiatives. This model was also included in the 2018 update of the Federal Performance Framework in Circular A-11, ensuring ‘excellent service’ as a focus in future agency strategic planning efforts. The PMA included a CAP Goal on Improving Customer Experience with Federal Services, with a primary strategy to drive improvements within 25 of the nation’s highest impact programs. This effort is supported by an interagency team and guidance in Circular A-11 requiring the collection of customer feedback data and increasing the use of industry best practices to conduct customer research.

These Presidential actions and requirements establish an ongoing process of collecting customer insights and using them to improve services. This new request will enable the Centers for Medicare and Medicaid Services (hereafter “the Agency”) to act in accordance with OMB Circular A-11 Section 280 to ultimately transform the experience of its customers to improve both efficiency and mission delivery, and increase accountability by communicating about these efforts with the public.

2. Indicate how, by whom, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection.

The Agency will collect, analyze, and interpret information gathered through this generic clearance to identify services’ accessibility, navigation, and use by customers, and make improvements in service delivery based on customer insights gathered through developing an understanding of the user experience interacting with Government.

For the purposes of this request, "customers" are individuals, businesses, and organizations that interact with a Federal Government agency or program, either directly or via a Federal contractor. Specifically, primary audiences include: Medicare beneficiaries, Health Insurance Marketplace consumers, advocates, stakeholders, media, researchers, and medical professionals.

"Service delivery" or "services" refers to the multitude of diverse interactions between a customer and CMS. For example, searching and applying for Medicare or Health Insurance Marketplace insurance coverage, finding a health care provider, accessing information on current coverage, processing online payments, accessing program guidance or program data, researching coverage policies, accessing news or reports, or seeking information such as public health notices.

Under this request, three types of activities will be conducted to generate customer insights:

Customer Research (E.g., User Persona and Journey Map Development): A critical first component of understanding customer experience is to develop customer personas and journey maps. This process enables the Agency to more deeply understand the customer segments they serve and to organize the processes customers interact with throughout their engagement with the Federal entity to accomplish a task or meet a need. In order to adequately capture the perspective of the customer and the barriers or supports that exist as they navigate these journeys, it is necessary to directly interact with customers rather than relying solely upon the Agency’s stated policy of how a process should work or employees’ interpretation of how services are delivered. This can occur through a variety of information collection mechanisms that include focus groups, individual intercept interviews at a service site, shadowing a user as they navigate a Federal service and documenting their reactions and frustrations, customer free-response comment cards, or informal small discussion groups.

Regardless of the format, the Agency will apply Human Centered Design (HCD) Discovery methods to generate personas and journey maps, ultimately identifying customer insights. An approach to recruiting participants, resources for preparing and structuring interviews, and a consent form for interviewees can be found at https://www.gsa.gov/cdnstatic/HCD-Discovery-Guide-Interagency-v12-1.pdf. This document is also included in the package.

Insights documented, summarized and presented in customer personas and journey maps can then be shared across the program, the Agency, other Federal, State, and Local government stakeholders and even with the public to validate and discuss common themes identified. These products can be used as “indicator lights” for where more rigorous qualitative and quantitative research can be conducted to improve Federal service delivery.

Publicly shared personas and journey maps will include language that qualifies their use (see question #16), and high-level, non-identifying descriptive statistics of the population(s) interviewed to develop it (ex. “25 Service members that transitioned to civilian employment within the last decade, 14 female, 11 male, 21 enlisted and 4 officers) to ensure that the perspective represented is understood. Quotes or insights will never be associated with an actual individual unless they have signed a release form (see link above for template) and this was included in the specific collection request.

Customer Feedback (Satisfaction Survey): Surveys to be considered under this generic clearance will only include those surveys modeled on the OMB Circular A-11 CX Feedback survey to improve customer service by collecting feedback at a specific point during a customer journey. This could include upon submitting a form online on a Federal website, speaking with a call center representative, paying off a loan, or visiting a Federal service center.

In an effort to develop comparable, government-wide scores that will enable cross-agency or industry benchmarking (when relevant) and a general indication of an agency’s overall customer satisfaction, OMB Circular A-11 Section 280 requires high impact services to measure their touchpoint/transactional performance in as a real-time manner as possible. These feedback surveys include:

• A required overall trust score of the entity in response to a Likert-scale question (preferred 5 point) Agencies should use the statements below as a sentence base and make only minor edits. Any requested modification to the wording of these statements must first be discussed with OMB prior to implementation in order to maintain reporting comparability government-wide.

  • This interaction increased my trust in [Program/Service name].

  • I trust [Agency/Program/Service name] to fulfill our country’s commitment to [relevant population].

• A required overall score of the transaction (satisfaction) in response to a Likert-scale question (preferred 5 point)

  • Agencies should use the statements below as a sentence base and make only minor edits, using a 5-point Likert scale. Any requested modification to the wording of these statements must first be discussed with OMB prior to implementation in order to maintain reporting comparability government-wide.

    • Please rate your experience [5 star option].

    • I am satisfied with the service I received from [program/service name].

• A required series of questions or choices to assess relationships between the overall score and experience drivers relevant to the service (see chart below)

• 1 question allowing but not requiring free response

• If applicable and necessary, as few questions as possible that enable the agency to make use of the data, such as a question regarding the purpose of a visit or call, for a total survey length of no more than 15 questions

Agencies are to submit these surveys for approval under the OMB A-11 Section 280 Umbrella Clearance that can be established at the department level using the templates provided on CX MAX Community page. Individual collection requests under this clearance will be reviewed by a customer experience subject matter expert in an effort to further streamline the review of these types of collections.

Baseline statements for each driver are provided, and the Agency may make adjustments to the wording as necessary for mission and circumstance-specific customization based on the type of service (see 280.4) or transaction type (for example, it may not involve an interaction with an employee). It is important that whenever possible, agencies ask the fewest number of questions, and map each question to one of the categories below, so that we might identify at a government-wide level which elements are most impactful on satisfaction and trust within different types of Federal services.

Customer Experience Drivers	Driver Sub-Categories
Service Quality	Service Effectiveness / Perception of Value
	My need was addressed / My issue was resolved. / I found what I needed. / My question was answered.
Process	Ease / Simplicity
	It was easy to complete what I needed to do. / It was easy to find what I needed.
	Efficiency / Speed
	It took a reasonable amount of time to do what I needed to do. / I found what I needed on the site quickly.
	Equity / Transparency
	I was treated fairly / I understand what was being asked of me throughout the process.
People (If applicable for a transaction)	Employee Interaction / Warmth / Helpfulness / Competence
	Employees I interacted with were helpful. / The Call Center Representative was committed to solving my problem.

Assess experience drivers one of three ways:

1. Individual questions for each driver on a 5-point Likert scale (strongly agree) to (strongly disagree)

2. A multiple choice question asking “How can we improve” or an equivalent question with a set of drivers to which the participant may choose all that apply

3. A multiple choice ‘choose all that apply’ questions, example one for selecting which drivers had a positive impact and one for those that had a negative impact

The Agency may add a few additional questions to those listed above to clarify type of service received, inquiry type, service center location, or other program-specific questions that can help program managers to filter and make use of the feedback data.

In an effort to increase transparency to drive accountability, the feedback data collected through the A-11 standard feedback surveys are meant to be shared with the public. This collection is part of the government-wide effort to embed standardized customer metrics within high-impact programs to create government-wide performance dashboards. Data collected from the questions listed above will be submitted by the Agency to OMB at a minimum quarterly for updating of customer experience dashboards on performance.gov. This dashboard will also include the total volume of customers that passed through the transaction point at which the survey was offered, the number of customers the survey was presented to, the number of responses, and the mode of presentation and response (online survey, in-person, post-call touchtone, mobile, email). This will help to qualify the data’s representation by showing both the response rate and total number of actual responses.

User Testing of Services and Digital Products: Agencies should continually review, update and refine their service delivery, including communication materials, processes, supporting reference materials, and digital products associated with a Federal program. This often requires “field testing” program informational materials, process updates, forms, or digital products (such as websites or mobile applications) by interacting with past, existing, or future customers and soliciting feedback. These activities can include cognitive laboratory studies, such as those used to refine questions on a program form to ensure clarity, demo kiosks at a service center where customers can provide informal feedback while waiting for a service, or more formally scheduled in-person observation testing (e.g., website or software usability tests). These information collection activities are more specific than broad customer research and related to a particular artifact / product of a Federal program. As such, there will be a more structured interview / set of questions than more open-ended customer research. Findings from these activities are meant to support the design and implementation of Federal program services and digital products, and may only be shared in an anonymized / in aggregate if a particular insight is useful to include as part of a customer persona, journey map, or common lesson learned for improving service delivery.

The Agency will only submit under this generic clearance if it meets the following conditions:

• The collections are voluntary;

• The collections are low-burden for respondents (based on considerations of total burden hours or burden-hours per respondent) and are low-cost for both the respondents and the Federal Government;

• The collections are non-controversial and do not raise issues of concern to other Federal agencies;

• Any collection is targeted to the solicitation of opinions from respondents who have experience with the program or may have experience with the program in the near future;

• Personally identifiable information (PII) is collected only to the extent necessary and is not retained;

• Information gathered is intended to be used for general service improvement and program management purposes

• Upon agreement between OMB and the agency all or a subset of information may be released as part of A-11, Section 280 requirements only on performance.gov. Additionally, summaries of customer research and user testing activities may be included in public-facing customer journey maps and summaries.

• Additional release of data must be done coordinated with OMB.

This clearance will help the Agency to establish a process where customer experience is regularly monitored and measured. The results will assist the Agency in the planning and decision-making processes to improve the quality of the Agency’s products and services.

Results from feedback activities and surveys will be used to measure against established baseline standards and for measuring the Agency’s progress toward defined goals.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or forms of information technology, e.g. permitting electronic submission of responses, and the basis for the decision of adopting this means of collection. Also describe any consideration given to using technology to reduce burden.

There are neither legal nor technical obstacles to the use of technology in these information collection activities. The determination to use technology, and which technology to use, will be based on the type of information collected and the utility and the availability of specific technology to each respondent in a proposed customer research activity or feedback survey.

4. Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Item 2 above.

The Agency will work to ensure the streamlining of all customer research and feedback surveys under this clearance. The Agency will also work to reduce existing customer feedback surveys and questions into alignment with the A-11 Standard CX Feedback survey as part of a coordinated Agency-wide customer program. The information to be supplied on these surveys will not be duplicated on any other information collection.

5. If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden. A small entity may be (1) a small business which is deemed to be one that is independently owned and operated and that is not dominant in its field of operation; (2) a small organization that is any not-for-profit enterprise that is independently owned and operated and is not dominant in its field; or (3) a small government jurisdiction, which is a government of a city, county, town, township, school district, or special district with a population of less than 50,000.

The information collected in these surveys will represent the minimum burden necessary to evaluate customer experience with the Agency’s programs and processes. The Agency will minimize the burden on respondents by sampling as appropriate, asking for readily available information, and using short, easy-to-complete information collection instruments.

6. Describe the consequences to Federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.

Without regular mechanisms for collecting and generating customer insights, the Agency is not able to provide the public with the highest level of service. These activities will be coordinated to ensure that most individual respondents will not be asked to respond to more than one survey instrument per transaction or to participate in more than one qualitative feedback or testing activity.

7. Explain any special circumstances that would cause an information collection to be conducted in a manner:

• requiring respondents to report information to the agency more often than quarterly;

• requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;

• requiring respondents to submit more than an original and two copies of any document;

• requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records for more than three years;

• in connection with a statistical survey, that is not designed to produce valid and reliable results than can be generalized to the universe of study;

• requiring the use of a statistical data classification that has not been reviewed and approved by OMB;

• that includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or that unnecessarily impedes sharing of data with other agencies for compatible confidential use; or

• requiring respondents to submit proprietary trade secrets, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information’s confidentiality to the extent permitted by law.

These surveys will be consistent with all the guidelines in 5 CFR 1320.5, especially those provisions in subsection (g) which require that a statistical survey be designed to produce results that can be generalized to the universe of study. There are no special circumstances that would cause this information collection to be conducted in an unusual or intrusive manner. All participation will be voluntary. Should the Agency need to deviate from the requirements outlined in 5 CFR 1320, individual justification will be provided to OMB on a case-by-case basis.

8. As applicable, state that the Department has published the 60 and 30 Federal Register notices as required by 5 CFR 1320.8(d), soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the agency in response to these comments. Specifically address comments received on cost and hour burden.

Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instruction and record keeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or reported.

Consultation with representatives of those from whom information is to be obtained or those who must compile records should occur at least once every 3 years – even if the collection of information activity is the same as in prior periods. There may be circumstances that may preclude consultation in a specific situation. These circumstances should be explained.

The 60-day public comment notice published in the Federal Register on February 14, 2020, (85 FR 8596). No comments were received. The 30-day public comment notice published in the Federal Register on May 14, 2020 (85 FR 28948).

9. Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees with meaningful justification.

The standard will be no payment or gift to respondents for participation. If any payments are proposed the Agency will submit specific justification for each proposed use as part of the completed package submitted to OMB.

10. Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statute, regulation, or agency policy. If personally identifiable information (PII) is being collected, a Privacy Act statement should be included on the instrument. Please provide a citation for the Systems of Record Notice and the date a Privacy Impact Assessment was completed as indicated on the IC Data Form. A confidentiality statement with a legal citation that authorizes the pledge of confidentiality should be provided.¹ If the collection is subject to the Privacy Act, the Privacy Act statement is deemed sufficient with respect to confidentiality. If there is no expectation of confidentiality, simply state that the Department makes no pledge about the confidentially of the data.

This generic information collection request proposes to collect information via a variety of voluntary information collection activities (see section 12).   Although the agency is not invoking statutory support for confidentiality, the quality of this type of information requires respondent candor and anonymity. Therefore, the agency pledges to keep the information collected private unless otherwise required by law. Respondents will be notified on the data collection forms that their information will only be reported in aggregated form and no personally identifiable responses will be publicly released.

11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private. The justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

No questions will be asked that are of a personal or sensitive nature.

12. Provide estimates of the hour burden of the collection of information. The statement should:

• Indicate the number of respondents by affected public type (federal government, individuals or households, private sector – businesses or other for-profit, private sector – not-for-profit institutions, farms, state, local or tribal governments), frequency of response, annual hour burden, and an explanation of how the burden was estimated, including identification of burden type: recordkeeping, reporting or third party disclosure. All narrative should be included in item 12. Unless directed to do so, agencies should not conduct special surveys to obtain information on which to base hour burden estimates. Consultation with a sample (fewer than 10) of potential respondents is desirable. If the hour burden on respondents is expected to vary widely because of differences in activity, size, or complexity, show the range of estimated hour burden, and explain the reasons for the variance. Generally, estimates should not include burden hours for customary and usual business practices.

• If this request for approval covers more than one form, provide separate hour burden estimates for each form and aggregate the hour burdens in the ROCIS IC Burden Analysis Table. (The table should at minimum include Respondent types, IC activity, Respondent and Responses, Hours/Response, and Total Hours)

• Provide estimates of annualized cost to respondents of the hour burdens for collections of information, identifying and using appropriate wage rate categories. The cost of contracting out or paying outside parties for information collection activities should not be included here. Instead, this cost should be included in Item 14.

A variety of instruments and platforms will be used to collect information from respondents. The annual burden hours requested (300,000) are based on the number of collections we expect to conduct over the requested period for this clearance.

Customer Research: The respondent burden averages from five (5) minutes to one hundred and twenty (120) minutes for interviews or focus groups, dependent on the individual activity. This estimation has been based on previous customer research activities conducted by Federal customer experience teams.

The Agency estimates that across all sub-components and services, there will be up to 500 individual interviews averaging 30 minutes each.

This clearance estimates that each member of a focus group is expected to spend an average of 1.5 hours per group. The Agency estimates that there will be 15 segmented focus groups held each year with an average of thirty participants per group for an average of 450 participants.

Customer Feedback (Satisfaction Survey): Industry best practice is to present every customer the opportunity to provide feedback at each instrumented touchpoint/transaction in a customer journey (ex. After submitting an application, completing a call at a call center, or visiting an in-person service center). The Agency will specify the total possible number of respondents based on estimated annual volume, but this information collection sets a ceiling estimate of 1,000,000 annually.

Testing of Services and Digital Products: Simple user testing (15 minutes or less) can be useful for a variety of digital product and service initiatives across the Agency (ex. basic website design updates, communications revisions). Expecting each instance to have on average of 20 users, and 30 services/products tested each year, this clearance expects 600 respondents.

Additionally, more extensive cognitive laboratory or in-person observations can occur for newly launched products (such as mobile applications or re-designs of process steps) requiring an average of 30 minutes per respondent. Expecting each instance to have an average of 20 users, and 10 new services/products tested each year across the Agency, this adds 200 respondents to the clearance.

The Agency will keep track of the above activities in order to accurately update burden calculations year to year.

Type of Information Collection	Number of Respondents	Number of responses per respondent	Hours per response	Total Burden hours
Customer Interviews
Focus Groups
Feedback Survey	2,740,000	1	3/60	133,933
User Testing (Rapid Feedback)
User Testing (Deep Dive)
Total	2,740,000	na	na	133,933

Respondents for the aforementioned types of information collection activities will represent various demographic profiles with respect to age, occupation, and income (hourly wage). Therefore, we are using the May 2022 All Occupation code 00-0000² provided by the Bureau of Labor and Statistics. We also have not adjusted for fringe and overhead.

The median hourly wage is $22.26. We estimate the total cost to be as follows.

Total Annual Cost = (133,933 hours) x ($22.26/hour) = $ 3,075,101.68

13. Provide an estimate of the total annual cost burden to respondents or record keepers resulting from the collection of information. (Do not include the cost of any hour burden shown in Items 12 and 14.)

• The cost estimate should be split into two components: (a) a total capital and start-up cost component (annualized over its expected useful life); and (b) a total operation and maintenance and purchase of services component. The estimates should take into account costs associated with generating, maintaining, and disclosing or providing the information. Include descriptions of methods used to estimate major cost factors including system and technology acquisition, expected useful life of capital equipment, the discount rate(s), and the time period over which costs will be incurred. Capital and start-up costs include, among other items, preparations for collecting information such as purchasing computers and software; monitoring, sampling, drilling and testing equipment; and acquiring and maintaining record storage facilities.

• If cost estimates are expected to vary widely, agencies should present ranges of cost burdens and explain the reasons for the variance. The cost of contracting out information collection services should be a part of this cost burden estimate. In developing cost burden estimates, agencies may consult with a sample of respondents (fewer than 10), utilize the 60-day pre-OMB submission public comment process and use existing economic or regulatory impact analysis associated with the rulemaking containing the information collection, as appropriate.

• Generally, estimates should not include purchases of equipment or services, or portions thereof, made: (1) prior to October 1, 1995, (2) to achieve regulatory compliance with requirements not associated with the information collection, (3) for reasons other than to provide information or keep records for the government or (4) as part of customary and usual business or private practices. Also, these estimates should not include the hourly costs (i.e., the monetization of the hours) captured above in Item 12

No costs for respondents are anticipated.

14. Provide estimates of annualized cost to the Federal government. Also, provide a description of the method used to estimate cost, which should include quantification of hours, operational expenses (such as equipment, overhead, printing, and support staff), and any other expense that would not have been incurred without this collection of information. Agencies also may aggregate cost estimates from Items 12, 13, and 14 in a single table.

The Agency may incur costs in setting up focus group, small discussion, or testing environments to include such things as hiring contractors, facilitators or moderators, travel to customer sample locations, renting meeting space, in providing remuneration, etc. Costs will be determined on an individual project basis and will be included in the ICR provided to OMB for each project to be conducted.

15. Explain the reasons for any program changes or adjustments. Generally, adjustments in burden result from re-estimating burden and/or from economic phenomenon outside of an agency’s control (e.g., correcting a burden estimate or an organic increase in the size of the reporting universe). Program changes result from a deliberate action that materially changes a collection of information and generally are result of new statute or an agency action (e.g., changing a form, revising regulations, redefining the respondent universe, etc.). Burden changes should be disaggregated by type of change (i.e., adjustment, program change due to new statute, and/or program change due to agency discretion), type of collection (new, revision, extension, reinstatement with change, reinstatement without change) and include totals for changes in burden hours, responses and costs (if applicable).

We are submitting a revision of the currently approved information collection request. One small change was made to Healthcare.gov Web Sat Survey (CMS-10710.GenIC#4) to mirror the Medicare.gov Web Sat Survey (CMS-10710.GenIC#5) by asking consumers for an email address if they'd like to opt in to be contacted for future research. This data collection supports our overall web experience improvement efforts by providing a pool of potential research participants for our qualitative web experience & usability research (covered by another PRA clearance package).

We are also retaining all of the previously approved GenICs associated with this information collection request.

IC Title	Hours
CMS-10710.GenIC#4. Healthcare.gov Web Satisfaction Survey	1,600
CMS-10710.GenIC#5 - Medicare.gov Web Sat Survey (A11 Section 280) (30-day)	24,000
CMS-10710.GenIC#6 - Marketplace IVR Customer Sat Survey (A11 Section 280) (30-day)	25,000
CMS-10710.GenIC#7 - Medicare IVR Customer Sat Survey (A11 Section 280)	83,333
Total	133,933

16. For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.

No attempt will be made to generalize the findings from these three groups of activities to be nationally representative or statistically valid. They are meant to compliment and help to contextualize performance and evaluation data as part of a three-pronged approach to understanding Federal program implementation and opportunities for improvement (Performance, Evaluation, and “Feedback” data³).

Customer Research: Insights gleaned from qualitative customer research may be presented publicly in the format of a conceptual user persona or customer journey map. Customer research can take anywhere from 6 weeks for a short sprint to a full fiscal year, depending on the specific project. The Agency expects most journey mapping efforts to last approximately 6 months, with a user persona and journey maps ready for feedback (both from internal and external to government stakeholders) within one month of completing customer research.

Publicly available Journey maps will include specific language to contextualize their use and will be included in specific requests. This language can include something like:

What should I know about journey maps?

Journey maps are living documents—continually refined and revisited. There is never

a “final” version, and these maps are meant to serve as a summary of the voices of

actual customers of U.S. Government services. A map may not precisely document

the way a Government program is meant to be navigated, accessed, or used. It might

not capture every government program or resource available to a customer segment.

However, it is the product of a qualitative research approach to gather insights from

customers’ actual experiences. These findings can help us to identify areas for

high-impact improvements across delivery channels and organizational silos.

Customer Feedback: Once touchpoint surveys are implemented at transaction points along the customer journey interacting with Federal services, data from the A-11 Standard CX Feedback survey will be submitted to OMB quarterly for review and publication in a summary dashboard on performance.gov.

This data will include:

• Specific transaction point at which the survey was administered

• Total volume of customers that interacted at this transaction point during the given quarter

• Total volume of customers that were presented the survey

• Total number of customers who completed the survey

• Mode(s) of collection (ex. online, over mobile, over the phone, paper form)

• Specific survey instrument that shows the Agency’s wording of standard A-11 CX Feedback survey

• Distribution of the responses across the 5 point Likert scale for each of the standard questions

The purpose of collecting volume and response numbers is to share customer feedback measures in context of the response rate and total volume of responses to qualify interpretation of the CX feedback data.

Testing of Services and Digital Products: Similar to Customer Research, this can range from a short two-day rapid feedback from users within an Agile product development sprint or longer effort to gather more extensive feedback from multiple physical locations.

17. If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

The Agency will include the OMB Control Number and collection expiration date at any qualitative feedback or testing activity and on each survey.

18. Explain each exception to the certification statement identified in the Certification of Paperwork Reduction Act.

The Agency is not requesting an exception to the certification statement identified in Item 20, “Certification for Paperwork Reduction Act Submissions, “ of OMB Form 83-I.

1 Requests for this information are in accordance with the following ED and OMB policies: Privacy Act of 1974, OMB Circular A-108 – Privacy Act Implementation – Guidelines and Responsibilities, OMB Circular A-130 Appendix I – Federal Agency Responsibilities for Maintaining Records About Individuals, OMB M-03-22 – OMB Guidance for Implementing the Privacy Provisions of the E-Government Act of 2002, OMB M-06-15 – Safeguarding Personally Identifiable Information, OM:6-104 – Privacy Act of 1974 (Collection, Use and Protection of Personally Identifiable Information)

2 https://www.bls.gov/oes/current/oes_nat.htm#00-0000

3 https://ssir.org/articles/entry/time_for_a_three_legged_measurement_stool

12