CMS-10393 Revised PRA Supporting Stmt-Pt B Passback Clean Final 07 14 2016

CMS-10393 Revised PRA Supporting Stmt-Pt B Passback Clean Final 07 14 2016.docx

Medicare Beneficiary and Family-Centered Satisfaction Survey

OMB: 0938-1177

Document [docx]
Download: docx | pdf

Supporting Statement Part B

Medicare Beneficiary and Family-Centered Satisfaction Survey


  1. Collection of Information Employing Statistical Methods


    1. Respondent Universe and Sampling Methods


The sampling and data collection methodology used for the Beneficiary Satisfaction survey has to be efficient based on the sample size, minimally burdensome for beneficiary respondents, frequent enough for use in on-going quality improvement efforts, and rigorous enough to permit for scoring and reporting at the QIO-level. To achieve all of the above described goals, CMS will collect and report the data quarterly. While CMS has included two formal evaluations of QIOs during the 11th SOW, providing quarterly data on beneficiary satisfaction with the complaints and appeals processed will permit for interim corrective action to be taken as needed.

The sample for the 11th SOW Beneficiary Satisfaction Survey will include beneficiaries who have filed quality of care complaints and those who have filed appeals about discharge. The proposed approach for the complaints and appeals cases are each described in this section.


Sample of Complaints Cases. The data collection methodology for the 10th SOW relies on obtaining survey response by mail. Based on this data collection methodology, CMS anticipates obtaining a response rate of approximately 60 percent. A 60 percent response rate would be in line with the response rates obtained on other similar surveys conducted by mail, including the Medicare CAHPS survey. During the 9th SOW, Beneficiary Satisfaction Survey data were collected using computer assisted telephone interviewing. Through this data collection, a seventy-five percent response rate was achieved. Because of the larger sample size required to include appeals cases in the data collection, a shift to a mail methodology is proposed for cost efficiency.


Table 1 presents the estimated quarterly number of complaint cases by state. The estimates are based on the average number of complaint cases over a 3-month period during the 9th SOW. Table 1 also presents estimated completes by state, based on a 60 percent response rate. These numbers represent a census of the complaints cases.


Sample of Appeals Cases. The appeals sampling methodology is based on the most efficient means of achieving the analytic goals of providing quarterly state-level scores and a robust data set of annual data analysis. During previous SOW, an average of 25,599 appeals cases was received per quarter. There is no known reason to believe that the volume of appeals cases will change notably in the 11th SOW.


As shown in Table 1, the sample will be drawn by state. Due to the differing volume of appeals by state, in some states with lower volumes, such as Alaska, Wyoming, Arkansas, Washington DC, Delaware and others, CMS will select a census of appeals cases. In other states including Alabama, Connecticut, California, Florida, and New York, CMS will use a simple random sample to draw 50 quarterly cases per state.


Based on a methodology that relies on data collection by mail, CMS anticipates obtaining a response rate of approximately 60 percent. There is no known reason to believe that response to the appeals survey will differ from other similar surveys including the Medicare CAHPS.


Table 1: Estimated Quarterly Numbers of Cases and Estimated Quarterly Completes





State


Appeals Estimated Quarterly Cases


Appeals Proposed Quarterly Sample

Appeals Estimated Quarterly Completes


Complaints Estimated Quarterly Cases


Complaints Proposed Quarterly Sample

Complaints Estimated Quarterly Completes

AK

4

4

2

1

1

0

AL

308

50

30

8

8

5

AR

31

31

19

9

9

5

AZ

228

50

30

7

7

4

CA

2,637

50

30

36

36

22

CO

82

50

30

14

14

8

CT

236

50

30

8

8

5

DC

11

11

7

1

1

0

DE

23

23

14

4

4

2

FL

1,473

50

30

47

47

28

GA

108

50

30

7

7

4

HI

14

14

8

3

3

2

IA

60

50

30

3

3

2

ID

41

41

25

3

3

2

IL

266

50

30

15

15

9

IN

131

50

30

7

7

4

KS

44

44

26

2

2

1





State


Appeals Estimated Quarterly Cases


Appeals Proposed Quarterly Sample

Appeals Estimated Quarterly Completes


Complaints Estimated Quarterly Cases


Complaints Proposed Quarterly Sample

Complaints Estimated Quarterly Completes

KY

81

50

30

4

4

2

LA

28

28

17

5

5

3

MA

428

50

30

6

6

4

MD

217

50

30

9

9

5

ME

28

28

17

2

2

1

MI

376

50

30

17

17

10

MN

216

50

30

9

9

5

MO

187

50

30

9

9

5

MS

61

50

30

5

5

3

MT

10

10

6

2

2

1

NC

147

50

30

2

2

1

ND

27

27

16

0

0

0

NE

95

50

30

1

1

0

NH

28

28

17

1

1

0

NJ

472

50

30

17

17

10

NM

30

30

18

5

5

3

NV

78

50

30

23

23

14

NY

1,458

50

30

45

45

27

OH

345

50

30

23

23

14

OK

36

36

22

14

14

9

OR

161

50

30

5

5

3

PA

1,146

50

30

14

14

9

PR

49

49

29

7

7

4

RI

51

50

30

3

3

2

SC

51

50

30

2

2

1

SD

18

18

11

0

0

0





State


Appeals Estimated Quarterly Cases


Appeals Proposed Quarterly Sample

Appeals Estimated Quarterly Completes


Complaints Estimated Quarterly Cases


Complaints Proposed Quarterly Sample

Complaints Estimated Quarterly Completes

TN

187

50

30

18

18

11

TX

277

50

30

27

27

16

UT

52

50

30

2

2

1

VA

251

50

30

5

5

3

VI

0

0

0

0

0

0

VT

16

16

10

0

0

0

WA

331

50

30

14

14

8

WI

176

50

30

9

9

5

WV

61

50

30

2

2

1

WY

2

2

1

0

0

0

National

12,737

50

30

0

Total

25,603

2,190

1,314

478

478

287



    1. Procedures for Data Collection


The data collection methodology used for the Beneficiary Satisfaction Survey flows from the proposed sampling approach and the need for on-going data for quality improvement. Based on recent literature on survey methodology and response rates by mode, including Dillman’s Tailored Design method1, CMS will use a data collection that is primarily mail. A mail-based methodology will achieve the goals of being efficient, effective, and minimally burdensome for beneficiary respondents. A single mode data collection will also reduce the known mode effects seen particularly in satisfaction surveys.2

Data will be collected quarterly during the QIO 11th SOW. That is to say, data will be collected 4 times per year from the time of OMB approval through the end of the contract period. Over the course of this period, CMS anticipates being able to conduct up to




1 Dillman, D. A. (2007). Mail and Internet Surveys; the tailored design method. New Jersey, United States: John Wiley & Sons Inc.

2 Dillman, D. A., Sangster, R. L., Tarnai, J. and Rockwood, T. H. (1996), Understanding differences in people’s answers to telephone and mail surveys. New Directions for Evaluation, 1996: 45–61. doi: 10.1002/ev.1034


10 rounds of data collection. The sample for each round of data collection will include appeals and complaint cases closed in the previous quarter such that no beneficiary should be sampled more than once. Re-appeals cases will not be included in the universe of eligible cases for sampling. In order for CMS and the QIOs to assess the degree to which patient-centered care is being delivered


Since data will be collected and reported for internal quality improvement on a quarterly basis, the data collection methodology must strive to minimize the data collection field period while maximizing the response rate. The desired data collection field period is 8 to 10 weeks. To achieve these goals, CMS will format the survey materials per Dillman’s Tailored Design principles and use a three-staged approach to data collection:


  1. Mailout of a covering letter, the paper survey questionnaire, and a postage-paid return envelope.


  1. Mailout of a post card that thanks respondents and reminds the non-respondents to please return their survey.


  1. Mailout of a follow-up covering letter, the paper survey questionnaire, and a postage-paid return envelope.


First a pilot study will be conducted (described in detail in section 4). Through the pilot test, CMS will determine the response rate that can be achieved using this approach. If it is deemed necessary, a prenotification letter or additional mailout reminders can be added to the protocol, a telephone non-response step can be added to the protocol as needed to achieve the desired response rate. Additional information on maximization of response rates is included in section 3.


Using the 3-step mail approach described above, CMS anticipates that data collection would occur over an 8 to 10 week period. This is to say, if the first survey mailing were dropped on January 1, we would anticipate completing data collection at the end of February or early March. Data would then be cleaned, scores would be generated, and data would be delivered for CMS and QIO quality improvement review. The precise timing required to achieve an acceptable response rate will be determined through the pilot test. The aim is to complete sampling, data collection complete and scoring within a 12-week period.


Survey Material: The Beneficiary Satisfaction survey will capture beneficiary satisfaction with the appeals review process as well as quality of care complaints.


Table 2 provides a summary of the Survey composites and questions. Each of the survey composites represents an important aspect of patient-centeredness. Data from the survey will be used by CMS and the QIOs to support the quality improvement effort, make changes to improve processes, and review overall success of each QIO at executing the complaints and appeals process in a manor aligned with CMS goals for patient and family centeredness.


NOTE: The questionnaire will be available in English and Spanish.


Table 2: Composite Measures from the Survey of Beneficiary Satisfaction with QIOs


10th SOW and Picker Institute Principles

Composite Label and Questions

Promoting effective coordination of care including helping communities support better health; transitions and continuity

Coordination

Q13. The QIO representative talked with you about programs and services in your community that were available to help you with your health and wellbeing.

Information, communication and education

Beneficiary-Centered Communication

Q7. How satisfied were you that the QIO representative explained things in a way you could understand?

Q8. How satisfied were you that the QIO representative spent enough time with you?

Q9. How satisfied were you that the QIO representative listened carefully to you?


Communication of Written Materials

Q15. How satisfied were you that the forms or letters you got about your [quality of care complaint / appeal] explained things in a way you could understand?

Q16. How satisfied were you that the forms or letters you got about your [quality of care complaint / appeal] had all the information you needed?

Q17. How satisfied were you that the forms or letters you got about your [quality of care complaint / appeal] showed respect for your concerns?

Q18. How satisfied were you that the forms or letters you got about your [quality of care complaint/appeal] were consistent with the information you were told in telephone conversations with the QIO?

Emotional support and alleviation of fear and anxiety

Courtesy and Respect

Q6. How satisfied were you that the QIO representative was as helpful as you thought he or she should be?

Q10. How satisfied were you that the QIO representative showed respect for what you said?

Access to care

Access and Responsiveness

Q11. The QIO representative was as responsive to your [quality of care complaint / appeal] as you thought he or she should be.

Q12. The QIO representative understood the situation related to your [quality of care complaint / appeal].



Analysis: On a monthly basis, each QIO will receive interim data results based on data collected to date for the current round of collection. Item level frequencies will be produced for each QIO such that QIOs can review and monitor survey results from beneficiaries in their state on an on-going basis. Both QIOs and CMS will use the data to identify areas of concern with the processes and institute immediate corrective action.


On an annual basis, data collected over the previous four quarters will be used for detailed analysis and reporting. Annual analysis will include univariate analysis for each of the survey variables as well as multivariate analysis to explore drivers of beneficiary satisfaction. Analytic findings will be presented to the QIO community to help QIOs improve the process utilized in processing beneficiary appeals and complaints. Additionally, CMS will use results to help re-shape the infrastructure supporting QIOs.


Satisfaction Scoring: Standardized scoring for the Beneficiary Satisfaction Survey will permit CMS and the QIOs to assess the process used in resolving beneficiary complaints and appeals without undertaking lengthy analysis on a quarterly basis. Satisfaction scoring is linked to the QIO scope of work and on-going evaluation of QIO performance. Based on the parameters of the QIO scope of work, calculations will be performed to determine the percent of beneficiaries who report being satisfied with the complaint or appeal process. Proposed scoring methodology is presented below. All questions proposed for inclusion in the scoring calculation use the satisfaction scale, or the agreement scale. The details of each of these scales are presented below.


Satisfaction Scale:

    1. Very Satisfied

    2. Satisfied

    3. Neither Satisfied nor Dissatisfied

    4. Dissatisfied

    5. Very Dissatisfied


Agreement Scale:

  1. Strongly Agree

  2. Agree

  3. Neither Agree nor Disagree

  4. Disagree

  5. Strongly Disagree


Survey responses of 1 and 2 (very satisfied and satisfied; and strongly agree and agree) will be counted as 1 point. Survey responses of 3, 4 and 5 (neither satisfied nor dissatisfied; neither agree nor disagree dissatisfied and strongly dissatisfied; and disagree and strongly disagree) will be counted as 0 points, as well as missing responses will not be included in the denominator for scoring – with the goal of producing standardized satisfaction scores for QIO evaluation.


Scores will be calculated at the case level for each of the survey composites. The case level scores will then be rolled up to quarterly QIO level scores for each of the survey composites. Per the QIO 10th Scope of Work, QIOs are to achieve the goal of 80 percent satisfied beneficiaries by the mid-point of the contract and 90 percent satisfied beneficiaries by the end of the contract.


    1. Methods to Maximize Response Rates.


Efforts to maximize response rates will take many forms, including multiple contacts and survey design principles.


As described in section 2 on data collection procedures, outreach to respondents will occur over three separate mailouts. All mailouts will be sent via first class mail. Timing of the mailouts will ensure that respondents are reminded of the request for their participation in the survey.


The survey design team will use Dillman’s Tailored Design principles in preparing the survey and mailout materials. These design principles have been shown to increase response rates to mailout surveys using formatting and layout principles. Covering materials will stress the importance of the respondent’s input and the use of survey findings to improve processes and make them more patient- centered, leveraging Dillman’s social exchange theory.


The mail methodology proposed here has been used successfully on other CMS surveys (Medicare CAHPS).

    1. Testing


As CMS is requesting an extension of an existing survey, we do not propose any new testing of the survey instrument and are relying on the testing done in association with the development of the existing survey.


At the beginning of the 10th SoW, a round of cognitive testing was completed. CMS did not conduct a full pilot test of the data collection and methodology. We cognitively tested the instruments for clarity of language and responses. At the time we submitted the OMB package for the survey currently in use, we relied on experience regarding response rate on similar surveys of Medicare beneficiaries. Consequently, CMS used the first two years of the survey data to evaluate how QIOs processed cases, including how QIOs encouraged beneficiaries to agree to take the survey. CMS used satisfaction scores collected in the first two years as a process monitoring tool and not a tool used for contract evaluation. The satisfaction score data were used in year 3 for contract evaluation purposes.


    1. Survey Development Consultants


The following individuals were consulted in the development of the surveys.



Table 3. Individuals Consulted During Development of Surveys


Organization

Name

Contact Information

CMS

David Russo

617.565.1310

[email protected]

CMS

Sally Berko

410.786.6211

[email protected]

Attachment 1 - CMS Medicare Beneficiary and Family Centered Care Satisfaction Survey

Quality of Care Complaints


Q1. Our records show that on [DATE] you filed a quality of care complaint about your or another person’s Medicare benefits. Is that right?

Q2. Have you received the results or findings in response to your quality of care complaint?

Q3. How satisfied are you with the results or findings in response to your quality of care complaint?
Q4. Please give us your comments on the results or findings in response to your quality of care complaint.

Q5. Did you speak to a QIO representative about your quality of care complaint?

Q6. How satisfied were you that the QIO representative was as helpful as you thought he or she should be?

Q7. How satisfied were you that the QIO representative explained things in a way you could understand?

Q8. How satisfied were you that the QIO representative spent enough time with you?

Q9. How satisfied were you that the QIO representative listened carefully to you?

Q10. How satisfied were you that the QIO representative showed respect for what you said?

Q11. The QIO representative was as responsive to your quality of care complaint as you thought he or she should be.

Q12. The QIO representative understood the situation related to your [quality of care complaint.

Q13. The QIO representative talked with you about programs and services in your community that are available to help with your health and wellbeing.

Q15. How satisfied were you that the forms or letters you got about your quality of care complaint explained things in a way you could understand?

Q16. How satisfied were you that the forms or letters you got about your quality of care complaint had all the information you needed?

Q17. How satisfied were you that the forms or letters you got about your quality of care complaint showed respect for your concerns?

Q18. How satisfied were you that the forms or letters you got about your quality of care complaint were consistent with the information you were told in telephone conversations with the QIO?

Q19. In responding to your appeal, [QIO NAME], the QIO in your state, gathered information about your quality of care complain, explained the complaint steps, and gave you the results or findings of your case. We are referring to this as the “quality of care complaint process”. Using any number from 0 to 10 where 0 is the worst quality of care complaint process possible and 10 is the best quality of care complaint process possible, what number would you use to rate the overall quality of care complaint process?
Q20. Please give us your comments on the process that was used in responding to your quality of care complaint. Include any comments you have on what worked well, and suggestions you have on ways to improve the process.



Attachment 2 - CMS Medicare Beneficiary and Family Centered Care Satisfaction Survey

Appeals.


Q1. Our records show that on [DATE] you filed an appeal about your or another person’s Medicare benefits. Is that right?

Q2. Have you received the results or findings in response to your appeal?

Q3. How satisfied are you with the results or findings in response to your appeal?
Q4. Please give us your comments on the results or findings in response to your appeal.

Q5. Did you speak to a QIO representative about your appeal?

Q6. How satisfied were you that the QIO representative was as helpful as you thought he or she should be?

Q7. How satisfied were you that the QIO representative explained things in a way you could understand?

Q8. How satisfied were you that the QIO representative spent enough time with you?

Q9. How satisfied were you that the QIO representative listened carefully to you?

Q10. How satisfied were you that the QIO representative showed respect for what you said?

Q11. The QIO representative was as responsive to your appeal as you thought he or she should be.

Q12. The QIO representative understood the situation related to your appeal.

Q13. The QIO representative talked with you about programs and services in your community that are available to help with your health and wellbeing.

Q15. How satisfied were you that the forms or letters you got about your appeal explained things in a way you could understand?

Q16. How satisfied were you that the forms or letters you got about your appeal had all the information you needed?

Q17. How satisfied were you that the forms or letters you got about your appeal showed respect for your concerns?

Q18. How satisfied were you that the forms or letters you got about your appeal were consistent with the information you were told in telephone conversations with the QIO?

Q19. In responding to your appeal, [QIO NAME], the QIO in your state, gathered information about your appeal, explained the appeal steps, and gave you the results or findings of your case. We are referring to this as the “appeal process”. Using any number from 0 to 10 where 0 is the worst appeal process possible and 10 is the best appeal process possible, what number would you use to rate the overall appeal process?
Q20. Please give us your comments on the process that was used in responding to your appeal. Include any comments you have on what worked well, and suggestions you have on ways to improve the process.



Shape1

1


File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
File TitleMedicare Beneficiary and Family-Centered Satisfaction Survey Supporting Statement Part B
SubjectSupporting Statement Part B
AuthorCMS
File Modified0000-00-00
File Created2021-01-24

© 2024 OMB.report | Privacy Policy