Form CMS-10415 Evaluation of Stakeholder Training

On-site Post-Training Survey

Below is a sample Onsite Post-Training Survey evaluating one day of on-site training. The

surveys are provided to event participants electronically and on paper. Surveys evaluating multiple modules will include the dates and titles of all sessions to be evaluated. Surveys are voluntary and training participants can opt out of completing evaluations for sessions they did not attend.

Stakeholder Training Evaluation Form

[Session T itle] [Dates]  [Location] Day [X]

Please take a few minutes to complete the relevant section(s) of this evaluation form. Your feedback will assist CMS in determining the content and direction of subsequent training sessions. Your responses are voluntary and confidential.

Section A — Modules (Day [3])

Please indicate your level of agreement with the following statements regarding the Module

[1]: [Module T itle]

Module [1] : [Module Title]	Strongly Agree	Agree	Disagree	Strongly Disagree	Not Applicable
The information was presented in an organized manner.					
The materials provided enhanced my training experience.					
In general, the module met my expectations.					
In my opinion, the module met the stated learning objectives.					

(Select one response per statement.)

Section B— Session Logistics How satisfied were you with each of the following aspects of the [Session T itle] training session?

Aspect	Very Satisfied	Satisfied	Dissatisfied	Very Dissatisfied	Not Applicable
The helpfulness of onsite staff					
The registration check-in process					
The session site					
The break(s) provided during the session(s)					
The visibility of presentation slides and visual aids					
The audibility of the speaker(s)					

(Select one response for each aspect.)

Section C - General Comments and Recommendations What recommendations, if any, do you have for future [Session Title] session topics?

Do you have any general comments regarding the [Session T itle] session?

Section D – Background Information

Which of the following best describes your organization? (Select one response only.)

^{ Qualified Health Plan/Issuer  Centers for Medicare & Medicaid Services (CMS)}

 Non-exchange Issuer  Third Party Administrator (TPA)

 Industry Association  Small Business Health Options Program (SHOP)

 State Based Exchange (SBE)  State Reinsurance Entity

^{ Pharmacy Benefit Manager (PBM)  Marketplace Assister/Navigator}

 Issuer Vendor  Other (Specify):

 Consulting Organization

Which of the following best describes your role within your organization? (Select one response only.)

 Chief Executive Officer  Chief Financial Officer  Compliance Staff

 Agent  Broker  CMS Staff

^{ Business/Program Analyst  Third Party Submitter  Finance/Revenue Staff}

 Coder/Data Analyst  Operations Staff  Risk Adjustment Staff

 Program/Project Manager  Information Technology Staff  Consultant

 Industry Association Representative  Quality Assurance/Quality Control Staff

 Other (Specify):

Evaluation forms will be collected at the conclusion of the session.

Thank you for completing the [Session Title] Evaluation Form.

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0938-NEW. The time required to complete this information collection is estimated to average 15 minutes per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: CMS, 7500 Security Boulevard, Attn: PRA Reports Clearance Officer, Mail Stop C4-26-05, Baltimore, Maryland 21244-1850.

Webinar Post-Training Participant Survey

Below is a sample Webinar Post-Training Participant Survey evaluating one webinar (e.g., the

first webinar in a series of webinars). Surveys evaluating multiple webinars will be emailed to participants and include the dates and titles of all sessions to be evaluated. ARDX will customize the email language and the survey instructions for each participant to reflect only those sessions that the participant attended.

Sample Email Language for Distribution of Evaluation Tool Link

Thank you for attending the <mm/dd/yy> Stakeholder Training Webinar session on

<Title of Session>. Please take a moment to complete the relevant sections of this evaluation form. Your feedback will assist CMS in determining the content and direction of the subsequent sessions.

To evaluate the webinar session please click here. Evaluations will be accepted through

<mm/dd/yyyy> at <hh:mm> EST.

If you have questions regarding logistics please contact the registrar at 1-###-###-#### or email the registrar at <registrar email address>.

_{Thank you for your time and your interest.}

_{Instructions:}

[Session Title] Webinar Evaluation Form

Please take a moment to answer the following questions in regards to the Stakeholder Training webinar on <Title of Session> held on <mm/dd/yyyy.>

Your feedback will assist CMS in determining the extent to which we achieved the goals of the session and help CMS to make improvements for future webinars. Your responses are voluntary and confidential.

Section A: Session Logistics

1. Please rate your level of satisfaction with each of the following aspects of the webinar.

1a. The ease of the webinar log-in

a. Very satisfied b. Satisfied

c. Dissatisfied

d. Very Dissatisfied e. Not Applicable

* If the respondent selects c or d, the following request will appear along with an empty text box for a response. (No response required)

Please provide a brief description of why you were dissatisfied or very dissatisfied with webinar log in.

1b. The ease of the webinar navigation

a. Very satisfied

b. Satisfied

c. Dissatisfied

d. Very Dissatisfied e. Not Applicable

*If the respondent selects c or d, the following request will appear along with an empty text box for a response. (No response required)

Please provide a brief description of why you were dissatisfied or very dissatisfied with webinar navigation.

2. The visibility of the slides(s)

a. Very satisfied b. Satisfied

c. Dissatisfied

d. Very Dissatisfied e. Not Applicable

*If the respondent selects c or d, the following request will appear along with an empty text box for a response. (No response required)

Please provide a brief description of why you were dissatisfied or very dissatisfied with visibility of the speaker(s).

3. The audibility of the speaker(s)

a. Very satisfied b. Satisfied

c. Dissatisfied

d. Very Dissatisfied e. Not Applicable

*If the respondent selects c or d, the following request will appear along with an empty text box for a response. (No response required)

Please provide a brief description of why you were dissatisfied or very dissatisfied with the audibility of the speaker(s).

4. The functionality of the question and answer feature

a. Very satisfied b. Satisfied

c. Dissatisfied

d. Very Dissatisfied

e. Not Applicable

*If the respondent selects c or d, the following request will appear along with an empty text box for a response. (No response required)

Please provide a brief description of why you were dissatisfied or very dissatisfied with the functionality of the questions and answer feature.

Section B: Session Facilitation and Content

1. Please rate your level of satisfaction with the facilitation of the webinar.

a. Very satisfied b. Satisfied

c. Dissatisfied

d. Very Dissatisfied e. Not applicable

*If the respondent selects c or d, the following request will appear along with an empty text box for a response. (No response required)

Please provide a brief description of why you were dissatisfied or very dissatisfied with the facilitation of the webinar.

2. Please rate your level of satisfaction with the materials provided for the webinar. a. Very satisfied

b. Satisfied

c. Dissatisfied

d. Very Dissatisfied e. Not applicable

*If the respondent selects c or d, the following request will appear along with an empty text box for a response. (No response required)

Please provide a brief description of why you were dissatisfied or very dissatisfied with the materials provided.

Section C: Overall Satisfaction

1. Please rate your level of overall satisfaction with the webinar session.

a. Very satisfied b. Satisfied

c. Dissatisfied

d. Very Dissatisfied e. Not applicable

* If the respondent selects c or d, the following request will appear along with an empty text box for a response. (No response required)

Please provide a brief description of why you were dissatisfied or very dissatisfied with the webinar session.

Section D: Learning Objectives

1. Please rate your agreement with the following statement: The webinar session met the stated learning objectives.

a. Strongly Agree b. Agree

c. Neither Agree nor Disagree

d. Disagree

e. Strongly Disagree

*If the respondent selects d or e, the following request will appear along with an empty text box for a response. (No response required)

Please provide a brief description of why you believe the webinar session did not meet the stated learning objectives.

Section E: Comments and Suggestions

1. What suggestions do you have for future [Session title] topics?

2. Do you have any additional comments regarding the [Session Title] webinar?

Section F: Background Information

1. Which of the following best describes your organization? (Survey will accept only one

response.)

a. Qualified Health Plan/Issuer b. Non-Exchange Issuer

c. Industry Association

d. State Based Exchange (SBE)

e. Pharmacy Benefit Manager (PBM)

f. Issuer Vendor

g. Centers for Medicare & Medicaid Services (CMS)

h. Third Party Administrator (TPA)

i. Small Business Health Options Program (SHOP)

j. State Reinsurance Entity

k. Marketplace Assister/Navigator l. Other (Specify):

2. Which of the following best describes your role within your organization? (Survey will accept only one response)

a. Chief Executive Officer b. Chief Financial Officer c. Compliance Staff

d. Agent e. Broker

f. CMS Staff

g. Business/Program Analyst h. Third Party Submitter

i. Finance/Revenue Staff

j. Coder/Data Analyst k. Operations Staff

l. Risk Adjustment Staff

m. Program/Project Manager

n. Information Technology Staff o. Consultant

p. Industry Association Representative

q. Quality Assurance/Quality Control Staff r. Other (specify):

Thank you completing the [Session Title] webinar evaluation form.

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0938-NEW. The time required to complete this information collection is estimated to average 15 minutes per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: CMS, 7500 Security Boulevard, Attn: PRA Reports Clearance Officer, Mail Stop C4-26-05, Baltimore, Maryland 21244-1850.

User Group Post-Training Participant Survey

Below is a sample User Group Post-Training Participant Survey evaluating one user group

session (e.g., the first user group in a series, for example). Surveys evaluating multiple user groups will include the dates and titles of all sessions to be evaluated. ARDX will customize the email language and the survey instructions for each participant to reflect only those sessions that the participant attended.

Sample Email Language for Distribution of Evaluation Tool Link

Thank you for attending the <mm/dd/yy> Stakeholder Training User Group session on

<Title of Session>. Please take a moment to complete the relevant sections of this evaluation form. Your feedback will assist CMS in determining the content and direction of the subsequent sessions. Your responses are voluntary and confidential.

To evaluate the webinar session please click here. Evaluations will be accepted through

<mm/dd/yyyy> at <hh:mm> EST.

If you have questions regarding logistics please contact the registrar at 1-###-###-#### or email the registrar at <registrar email address>.

_{Thank you for your time and your interest.}

_{Instructions:}

[Session Title] User Group Evaluation Form

Please take a moment to answer the following questions in regards to the Stakeholder Training

User Group session on <Title of Session> held on <mm/dd/yyyy.>

Your feedback will assist CMS in determining the extent to which we achieved the goals of the session and help CMS to make improvements for future user groups. Your responses are voluntary and confidential.

Section A: User Group Logistics

Please rate your level of satisfaction with each of the following aspects of the user group.

1. The ease of the user group log-in. a. Very satisfied

b. Satisfied

c. Dissatisfied

d. Very Dissatisfied e. Not Applicable

*If the respondent selects c or d, the following question will appear along with an empty text box for a response. (No response required)

Please provide a brief description of why you were dissatisfied or very dissatisfied with user group log in.

2. The audibility of the speaker(s)

a. Very satisfied b. Satisfied

c. Dissatisfied

d. Very Dissatisfied e. Not Applicable

* If the respondent selects c or d, the following question will appear along with an empty text box for a response. (No response required)

Please provide a brief description of why you were dissatisfied or very dissatisfied with the audibility of the speaker(s).

3. The visibility of the presentation slides a. Very satisfied

b. Satisfied

c. Dissatisfied

d. Very Dissatisfied e. Not Applicable

*If the respondent selects c or d, the following question will appear along with an empty text box for a response. (No response required)

Please provide a brief description of why you were dissatisfied or very dissatisfied with the visibility of the speaker(s).

Section B: Session Facilitation and Content

1. Please rate your level of satisfaction with the facilitation of the user group.

a. Very satisfied b. Satisfied

c. Dissatisfied

d. Very Dissatisfied e. Not applicable

*If the respondent selects c or d, the following question will appear along with an empty text box for a response. (No response required)

Please provide a brief description of why you were dissatisfied or very dissatisfied with the facilitation of the user group.

2. Please rate your level of satisfaction with the materials provided for the user group. a. Very satisfied

b. Satisfied

c. Dissatisfied

d. Very Dissatisfied e. Not applicable

*If the respondent selects c or d, the following question will appear along with an empty text box for a response. (No response required)

Please provide a brief description of why you were dissatisfied or very dissatisfied with the materials provided for the user group.

Section C: Overall Satisfaction

1. Please rate your level of overall satisfaction with the user group session.

a. Very satisfied b. Satisfied

c. Dissatisfied

d. Very Dissatisfied e. Not applicable

*If the respondent selects c or d, the following request will appear along with an empty text box for a response. (No response required)

Please provide a brief description of why you were dissatisfied or very dissatisfied with the user group session.

Section D: Learning Objectives

1. Please rate your agreement with the following statement: The user group session met the

stated learning objectives. a. Strongly Agree

b. Agree

c. Neither Agree nor Disagree d. Disagree

e. Strongly Disagree

*If the respondent selects d or e, the following request will appear along with an empty test box for a response. (No response required).

Please provide a brief description of why you believe the user group session did not meet the stated learning objectives.

Section E: Comments and Suggestions

1. What suggestions do you have for future [Session title] topics?

2. Do you have any additional comments regarding the [Session Title] user group?

Section F: Background Information

1. Which of the following best describes your organization? (Survey will accept only one

response).

a. Qualified Health Plan/Issuer b. Non-Exchange Issuer

c. Industry Association

d. State Based Exchange (SBE)

e. Pharmacy Benefit Manager (PBM)

f. Issuer Vendor

g. Centers for Medicare & Medicaid Services (CMS)

h. Third Party Administrator (TPA)

i. Small Business Health Options Program (SHOP)

j. State Reinsurance Entity

k. Marketplace Assister/Navigator l. Other (Specify):

2. Which of the following best describes your role within your organization? (Survey will accept only one response)

a. Chief Executive Officer b. Chief Financial Officer c. Compliance Staff

d. Agent e. Broker

f. CMS Staff

g. Business/Program Analyst h. Third Party Submitter

i. Finance/Revenue Staff j. Coder/Data Analyst

k. Operations Staff

l. Risk Adjustment Staff

m. Program/Project Manager

n. Information Technology Staff o. Consultant

p. Industry Association Representative

q. Quality Assurance/Quality Control Staff r. Other (specify):

Thank you completing the [Session Title] user group evaluation form.

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0938-NEW. The time required to complete this information collection is estimated to average 15 minutes per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: CMS, 7500 Security Boulevard, Attn: PRA Reports Clearance Officer, Mail Stop C4-26-05, Baltimore, Maryland 21244-1850.

Web Portal Evaluation Form

Below is a sample Web Portal Evaluation Form that will be available on the registration web

portal, www.REGTAP.info, and available to users 24-hours per day, 7 days per week. A pop-up invitation to participate in the survey will be available to a random sampling of users. The pop- up will occur every 10^th time the user logs into REGTAP.

Web Portal Evaluation Form

Thank you for taking time to complete the Web Portal Evaluation Form. Your responses will be

utilized for on-going improvement of the Registration and Technical Assistance web portal and electronic communications. Your responses are voluntary and confidential.

1. How did you hear about the CMS Registration and Technical Assistance web portal?

a. My employer b. A colleague

c. Listserv email

d. Newsletter

e. Social Media Outlet (twitter, etc.)

f. Other: (specify)

2. Please rate your overall satisfaction with the CMS Registration and Technical

Assistance web portal. a. Very Satisfied b. Satisfied

c. Neither satisfied nor dissatisfied d. Dissatisfied

e. Very dissatisfied

*If the respondent selects D or E, the following question will appear along with an empty text box for a response. (No response required)

What can we do to improve your experience with the CMS Registration and

Technical Assistance web portal?

3. Have you used the web portal to register for a training event (onsite training, webinar, or user group)?

A. Yes

i. Rate your level of satisfaction with the registration process a Very satisfied

b Satisfied

c Neither satisfied nor dissatisfied

d Dissatisfied

e Very dissatisfied

*If the respondent selects d or e, the following question will appear along with an empty text box for a response. (No response required)

B. No

What can we do to improve your registration experience?

i. Question 4 will appear

4. Have you used the issues tracking portion of the web portal?

_{A. Yes}

i. Rate your level of satisfaction with issues tracking:

a Very satisfied b Satisfied

c Neither satisfied nor dissatisfied

d Dissatisfied

e Very dissatisfied

*If the respondent selects d or e, the following questions will appear along with an empty text box for a response. (No response required)

B. No

What can we do to improve your experience with issues tracking?

i. Question 5 will appear

5. Do you receive electronic communications (daily tips, weekly bulletins, monthly newsletters) from CMS concerning Stakeholder Training and Technical Assistance?

_{A. Yes}

i. Rate you level of satisfaction with the electronic communications you receive.

a Very satisfied b Satisfied

c Neither satisfied nor dissatisfied d Dissatisfied

e Very dissatisfied

*If the respondent selects d or e, the following question will appear along with an empty text box for a response. (No response required)

What can we do to improve your experience with electronic communications?

ii. Do you have any suggestions for improving these electronic communications?

B. No

i. Question 6 will appear

6. Have you reviewed any of the documents in the library section of the CMS Registration and Technical Assistance portal?

_{A. Yes}

i. Please rate your level of satisfaction with the library. a Very satisfied

b Satisfied

c Neither satisfied nor dissatisfied d Dissatisfied

e Very dissatisfied

ii. What additional resources (if any) would you suggest we include in our library?

B. No

i. Question 7 will appear

7. Please provide any general comments you may have concerning the CMS Registration and Technical Assistance web portal.

8. Which of the following best describes your organization? (Survey will accept only one response).

a. Qualified Health Plan/Issuer b. Non-Exchange Issuer

c. Industry Association

d. State Based Exchange (SBE)

e. Pharmacy Benefit Manager (PBM)

f. Issuer Vendor

g. Centers for Medicare & Medicaid Services (CMS)

h. Third Party Administrator (TPA)

i. Small Business Health Options Program (SHOP)

j. State Reinsurance Entity

k. Marketplace Assister/Navigator l. Other (Specify):

9. Which of the following best describes your role within your organization? (Survey will accept only one response)

a. Chief Executive Officer b. Chief Financial Officer c. Compliance Staff

d. Agent e. Broker

f. CMS Staff

g. Business/Program Analyst h. Third Party Submitter

i. Finance/Revenue Staff

j. Coder/Data Analyst k. Operations Staff

l. Risk Adjustment Staff

m. Program/Project Manager

n. Information Technology Staff o. Consultant

p. Industry Association Representative

q. Quality Assurance/Quality Control Staff r. Other (Specify):

Thank you for completing the Registration and Technical Assistance Web Portal

Evaluation Form.

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0938-NEW. The time required to complete this information collection is estimated to average 15 minutes per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: CMS, 7500 Security Boulevard, Attn: PRA Reports Clearance Officer, Mail Stop C4-26-05, Baltimore, Maryland 21244-1850.

INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW: This information has not

been publicly disclosed and may be privileged and confidential. It is for internal government use only and must not be disseminated, distributed, or copied to persons not authorized to receive the information. Unauthorized disclosure may result in

prosecution to the fullest extent of the law

1