Form CMS-10591 Marketplace Dental Survey Cognitive Testing Eligibility

Attachment A – Draft Survey Instrument

Marketplace Dental Survey (MDS) Overview

Exhibit 1 provides a quick overview of what is measured in this survey. It is NOT meant to list hypothesized composite items, and it does NOT list every item. Rather, the exhibit covers unique constructs to provide an overview of what is included in the Marketplace Dental Survey (MDS).

Exhibit 1. Overview of Constructs in the Marketplace Dental Survey (MDS)

Symbol Legend
*	In original composite for existing D-CAHPS composite1
δ	In QHP survey domain2
Ϭ	Existing D-CAHPS survey item but not in a D-CAHPS composite
	New item (not in original D-CAHPS survey)
Care from Dentists and Staff (Existing D-CAHPS Composite)
*	Regular dentist explained things in a way that was easy to understand
*	Regular dentist listened carefully to you
*	Regular dentist treated patient you with courtesy and respect
*	Regular dentist spent enough time with you
*	Dentists or dental staff did everything to make you as comfortable as possible
*	Dentists or dental staff explained what they were doing during treatment
	Regular dentist knew of dental health history
	Regular dentist addressed prevention
Access to Dental Care (Existing D-CAHPS Composite)
*	How often were dental appointments as soon as you wanted
*	How often were emergency appointments as soon as you wanted
*	How often were specialty appointments as soon as you wanted
*	How often was wait time over 15 minutes
*	How often did anyone explain why you were waiting for more than 15 minutes
	Disability accommodations if needed
Dental Plan Costs and Services (Existing D-CAHPS Composite)
*	How often dental plan covered all of the services you thought were covered
*	How often 800 number, written materials, or website provided the information you wanted
*	How dental plan’s customer service gave you the information or help you needed
*	How often dental plan’s customer service staff treated you with courtesy and respect
*	Did your dental plan cover what you and your family needed to get done
*	Did dental plan help you find a dentist you were happy with
	Forms were easy to fill out
	Understood cost of care prior to treatment
	Delay or not obtain care because you were worried about cost
Ϭ	Plan is worth the cost
Cultural Competance (QHP Composite)
δ	Need interpreter at dentist’s office
δ	How often got an interpreter
δ	Forms available in preferred language
δ	Forms available in preferred format, such as large print or braille
Facility Quality (New Composite, New QUestions)
	Received administrative reminders
	Facility cleanliness
	Global rating of facility quality
OTHER ITEM CATEGORIES
Patient Ratings
*	Rating [0-10] of personal dentist
*	Rating [0-10] of all dental care
*	Rating [0-10] of ease finding a new dentist
*	Rating [0-10] of dental plan
*	Would respondent recommend dental plan to friends and family
	Rating [0-10] of whether will recommend dental plan to friends and family
	Rating [0-10] of facility
Respondent Characteristics
Ϭ	Overall condition of teeth and gums
Ϭ	Age
Ϭ	Sex
Ϭ	Education status
Ϭ	Ethnicity
Ϭ	Race
Ϭ	Someone help you complete this survey
Ϭ	How did someone help you complete this survey
Ϭ	Preventative (flossing) behavior
Ϭ	Rating of overall health
Ϭ	Rating of overall mental or emotional health
Ϭ	Knowledge of dental insurance terms
Ϭ	Knowledge of how dental plan works

MDS Survey Legend

Enclosed, please find the current draft of the Marketplace Dental Survey (MDS. Each item has been labeled to indicate the domain, the formative research efforts that supported the inclusion of the construct, and item source; the lists below provide the abbreviations used. For example, the label (AC/L/DCAHPS-3) means the survey item came from the Access to Care domain, the construct came from the literature review, and the question wording is the original version of the Dental Consumer Assessment of Healthcare Providers and Systems (D-CAHPS) question #3. The headings in this survey are from the D-CAHPS and are meant for respondent navigation, not domain headings.

Survey Indicator

DCAHPS-# = Dental CAHPS

QHP-# = Qualified Health Plan Survey 2016 Survey

CAHPSPCMH-# = CAHPS Patient-Centered Medical Home Item Set

CAHPSCGS-# = CAHPS Clinician and Group Survey 3.0

CCMDS-# = Care Coordination Quality Measure for Primary Care. Not yet publically available.

NewItem = New item

Note that the letter “m” before the question # indicates the item has been modified. Simple alterations to make the question relevant to dental care (e.g., changing the word “doctor” to “dentist”) are NOT listed modifications. In addition, the reference period was changed to 6 months for all items. This is NOT listed as a modification.

Dental Marketplace Domain Name

ADMIN=Administrative Question

CD=Care from Dentists

AC=Access to Dental Care

PCS=Dental Plan Costs and Services

PR=Patient Ratings

CuC=Cultural Competence

FQ=Facility Quality

RC=Respondent Characteristics

All the questions have a domain label.

Construct Source from Formative Research

L=Lit Review

C=Consumer Focus Groups

S=Stakeholder Interviews

T=Technical Expert Panel

Q=Similar Item in QHP

NA=Screener, no construct source is relevant

Marketplace Dental Draft Survey

We are asking you to complete this survey about your experiences with [DENTAL PLAN NAME] in the last 6 months. If you are enrolled in a different dental plan for 2016, please answer the questions in the survey thinking about your experiences in your previous health plan from July through December 2015.

Your Privacy is Protected. What you have to say is private and will be used only for this study. Your answers will be part of a pool of information. We will not share your name or answers with anyone, except if required by law.

Your Participation is Voluntary. You do not have to answer any questions that you do not want to answer. If you choose not to answer, it will not affect the benefits you receive.

What To Do When You’re Done. Once you complete the survey, place it in the envelope that was provided, seal the envelope, and return the envelope to [SURVEY VENDOR ADDRESS].

What To Do If You Have Questions. [DENTAL PLAN] contracted with [SURVEY VENDOR NAME] to conduct this survey. If you have any questions about the survey, call [SURVEY VENDOR NAME] toll free at (XXX) [XXX-XXXX] between XX:XX a.m. and XX:XX p.m. [SURVEY VENDOR LOCAL TIME], Monday through Friday or e-mail [SURVEY VENDOR EMAIL].

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid Office of Management and Budget (OMB) control number. The valid OMB control number for this information collection is XXXX-XXXX. The time required to complete this information collection is estimated to average 20 minutes per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: CMS, 7500 Security Boulevard, Attn: PRA Reports Clearance Officer, Mail Stop C4-26-05, Baltimore, Maryland 21244-1850.

Survey Instructions

Answer each question by marking the box to the left of your answer.

You are sometimes told to skip over some questions in this survey. When this happens you will see an arrow with a note that tells you what question to answer next, like this:

Yes  If Yes, go to #1 on page 1

No

Survey Introduction

1. Our records show that you are now in the {insert dental plan name}. Is that right? (Admin/NA/DCAHPS-1)

¹ Yes ® If Yes, go to #3

² No

2. What is the name of your dental plan? This may be the name of your health plan if it also covers your dental care. (Admin/NA/DCAHPS-m2)

Please print:

3. In the last 6 months, did you go to a dentist’s office or clinic for care? (Admin/NA/DCAHPS-3)

¹ Yes

² No ® If No, please stop and
return this survey in the postage-paid envelope. Thank you.

Your Regular Dentist

4. A regular dentist is one you would go to for check-ups and cleanings or when you have a cavity or tooth pain. Do you have a regular dentist? (CD/NA/DCAHPS-4)

¹ Yes

² No ® If No, go to #14

5. Have you seen your regular dentist in the last 6 months? (CD /NA/DCAHPS-5)

¹ Yes

² No, I’ve seen someone else ® If No, go to #14

6. In the last 6 months, how often did your regular dentist explain things in a way that was easy to understand? (CD/LSCTQ/DCAHPS-6)

¹ Never

² Sometimes

³ Usually

⁴ Always

7. In the last 6 months, how often did your regular dentist listen carefully to you? (CD/LSCTQ/DCAHPS-7)

¹ Never

² Sometimes

³ Usually

⁴ Always

8. In the last 6 months, how often did your regular dentist treat you with courtesy and respect? (CD/LSCTQ/DCAHPS-8)

¹ Never

² Sometimes

³ Usually

⁴ Always

9. In the last 6 months, how often did your regular dentist spend enough time with you? (CD/LSCTQ/DCAHPS-9)

¹ Never

² Sometimes

³ Usually

⁴ Always

10. Using any number from 0 to 10, where 0 is the worst regular dentist possible and 10 is the best regular dentist possible, what number would you use to rate your regular dentist? (PR/LSCTQ/DCAHPS-10)

0 Worst regular dentist possible

1

2

3

4

5

6

7

8

9

10 Best regular dentist possible

11. In the last 6 months, how often did you and your regular dentist agree on the best way to take care of your dental health? (CD/LSCTQ/CCMDS-m46)

¹ Never

² Sometimes

³ Usually

⁴ Always

12. In the last 6 months, would you say your regular dentist knew about the dental health issues you have had in the past? (CD/LSCTQ/CCMDS-m33)

¹ Definitely yes

² Somewhat yes

³ Somewhat no

⁴ Definitely no

13. In the last 6 months, how often did you and someone from this dentist’s office talk about how to care for your teeth and gums at home? (CD/LSCTQ/ CAHPSCGS-m20)

¹ Never

² Sometimes

³ Usually

⁴ Always

Your Dental Care in the Last 6 Months

So far, the questions on this survey have been about your regular dentist. The next set of questions asks about any dental care you had in the last 6 months, including dental care with your regular dentist or with someone else.

14. In the last 6 months, how often did the dentists or dental staff do everything they could to help you feel as comfortable as possible during your dental work? (CD/LSCT/DCAHPS-11)

¹ Never

² Sometimes

³ Usually

⁴ Always

15. In the last 6 months, how often did the dentists or dental staff explain what they were doing while treating you? (CD/LSCTQ/DCAHPS-12)

¹ Never

² Sometimes

³ Usually

⁴ Always

16. In the last 6 months, how often were your dental appointments as soon as you wanted? (AC/LSCTQ/DCAHPS-13)

¹ Never

² Sometimes

³ Usually

⁴ Always

17. In the last 6 months, did you need to see a dentist right away because of a dental emergency? (AC/NA/NewItem)

¹ Yes

² No ® If No, go to #19

18. In the last 6 months, when you needed to see a dentist right away because of a dental emergency, how often did you get to see a dentist as soon as you wanted? (AC/LSCTQ/DCAHPS-m14)

¹ Never

² Sometimes

³ Usually

⁴ Always

19. In the last 6 months, have you tried to get an appointment for yourself with a dentist who specializes in a particular type of dental care (such as root canals or gum disease)? (AC/NA/NewItem)

¹ Yes

² No ® If No, go to #21

20. In the last 6 months, when you tried to get an appointment for yourself with a dentist who specializes in a particular type of dental care (such as root canals or gum disease), how often did you get an appointment as soon as you wanted? (AC/LSCTQ/DCAHPS-15)

¹ Never

² Sometimes

³ Usually

⁴ Always

21. In the last 6 months, how often did you have to spend more than 15 minutes in the waiting room before you saw someone for your appointment? (AC/LCT/DCAHPS-16)

¹ Never  If Never, go to #23.

² Sometimes

³ Usually

⁴ Always

22. If you had to spend more than 15 minutes in the waiting room before you saw someone for your appointment, how often did someone tell you why there was a delay or how long the delay would be? (AC/L/DCAHPS-17)

¹ Never

² Sometimes

³ Usually

⁴ Always

23. Using any number from 0 to 10, where 0 is the worst dental care possible and 10 is the best dental care possible, what number would you use to rate all of the dental care you personally received in the last 6 months? (PR/LSCTQ/DCAHPS-18)

0 Worst dental care possible

1

2

3

4

5

6

7

8

9

10 Best dental care possible

24. An interpreter is someone who helps you talk with others who do not speak your language. In the last 6 months, did you need an interpreter to help you speak with anyone at your dentist’s office or clinic? (CuC/NA/QHP-12)

¹ Yes

² No ® If No, go to #26

25. In the last 6 months, when you needed an interpreter at your dentist’s office or clinic, how often did you get one? (CuC/LSQ/QHP-13)

¹ Never

² Sometimes

³ Usually

⁴ Always

26. A disability is a physical or mental impairment that substantially limits one or more major life activities. In the last 6 months, did you need extra help or assistance during your dental appointment because you had a disability? (AC/NA/NewItem)

¹ Yes

² No ® If No, go to #28

27. In the last 6 months, how often did you receive the help or assistance you needed because you had a disability? (AC/LT/NewItem³)

¹ Never

² Sometimes

³ Usually

⁴ Always

28. Some offices remind patients between visits about tests, treatment, or appointments. In the last 6 months, did you get any reminders from this dental office between visits? (FQ/LCT/CAHPS-PCMH5)

¹ Never

² Sometimes

³ Usually

⁴ Always

29. In the last 6 months, how often was the dental office where you receive care clean? (FQ/LSCT/NewItem⁴)

¹ Never

² Sometimes

³ Usually

⁴ Always

30. Using any number from 0 to 10, where 0 is the worst dental facility possible and 10 is the best dental facility possible, what number would you use to rate the facilities where you personally received care in the last 6 months? (PR/LCT/NewItem⁵)

0 Worst dental facility possible

1

2

3

4

5

6

7

8

9

10 Best dental facility possible

Dental Plan

The next set of questions asks about your dental plan. For these questions, answer only about your dental plan.

31. In the last 6 months, how often did your dental plan cover all of the services you thought were covered? (PCS/LSCTQ/DCAHPS-19)

¹ Never

² Sometimes

³ Usually

⁴ Always

32. In the last 6 months, did your dental plan cover what you and your family needed to get done? (PSC/LSCTQ/DCAHPS-20)

¹ Definitely yes

² Somewhat yes

³ Somewhat no

⁴ Definitely no

33. In the last 6 months, did you look for any information in written materials or on the Internet about your dental plan?? (PSC/NA/DCAHPS-m21)

¹ Yes

² No ® If No, go to #35

34. In the last 6 months, how often did the written materials or the Internet provide the information you needed about how your dental plan works? (PCS/LSCTQ/DCAHPS-m22)

¹ Never

² Sometimes

³ Usually

⁴ Always

35. In the last 6 months, did you use any information from the dental plan to help you find a new dentist? (PSC/NA/DCAHPS-23)

¹ Yes

² No  If No, go to #38

36. Did this information help you find a dentist you were happy with? (PCS/LC/DCAHPS-24)

¹ Definitely yes

² Somewhat yes

³ Somewhat no

⁴ Definitely no

37. Using any number from 0 to 10, where 0 is extremely difficult and 10 is extremely easy, what number would you use to rate how easy it was for you to find a dentist? (PR/LC/DCAHPS-25)

0 Extremely difficult

1

2

3

4

5

6

7

8

9

10 Extremely easy

38. In the last 6 months, did you try to get information or help from your dental plan’s customer service? (PSC/NA/DCAHPS-26)

¹ Yes

² No ® If No, go to #40

39. In the last 6 months, how often did your dental plan’s customer service give you the information or help you needed? (PCS/LSCTQ/DCAHPS-27)

¹ Never

² Sometimes

³ Usually

⁴ Always

40. In the last 6 months, how often did your dental plan’s customer service staff treat you with courtesy and respect? (PCS/LSCTQ/DCAHPS-28)

¹ Never

² Sometimes

³ Usually

⁴ Always

41. In the last 6 months, did your dental plan give you any forms to fill out? (PSC/NA/QHP-46)

¹ Yes

² No ® If No, go to #46

42. In the last 6 months, how often were the forms from your dental health plan easy to fill out? (PCS/LCTQ/ QHP-47)

¹ Never

² Sometimes

³ Usually

⁴ Always

43. In the last 6 months, how often were the forms that you had to fill out available in the language you prefer? (CuC/LSQ/QHP-49)

¹ Never

² Sometimes

³ Usually

⁴ Always

44. In the last 6 months, did you need the forms in a different format, such as large print or braille? CuC/NA /QHP-50)

¹ Yes

² No  If No, go to #46

45. In the last 6 months, how often were the forms that you had to fill out available in the format you needed, such as large print or braille? (CuC/LSQ/QHP-51)

¹ Never

² Sometimes

³ Usually

⁴ Always

46. Using any number from 0 to 10, where 0 is the worst dental plan possible and 10 is the best dental plan possible, what number would you use to rate your dental plan? (PR/LSCTQ/DCAHPS-29)

0 Worst dental plan possible

1

2

3

4

5

6

7

8

9

10 Best dental plan possible

47. Would you say that your dental plan is worth the cost? (PSC/LCTQ/DCAHPS-30)

¹ Definitely yes

² Somewhat yes

³ Somewhat no

⁴ Definitely no

48. Would you recommend this dental plan to people who want to join? (PR/LCQ/DCAHPS-31)

¹ Definitely yes

² Probably yes

³ Probably no

⁴ Definitely no

49. Using any number from 0 to 10, where 0 is not at all likely and 10 is extremely likely, how likely is it that you would recommend this dental plan to a friend or family member? (PR/LCQ/QHP-53)

0 Not at all likely

1

2

3

4

5

6

7

8

9

10 Extremely likely

50. In the last 6 months, how often did someone at your dental office tell you what your costs would be before you received treatment? (PSC/LSCT/NewItem⁶)

¹ Never

² Sometimes

³ Usually

⁴ Always

51. In the last 6 months, how often did you delay visiting or not visit a dentist because you were worried about the cost? (PSC/STQ/QHP-56)

¹ Never

² Sometimes

³ Usually

⁴ Always

About You⁷

52. In general, how would you rate the overall condition of your teeth and gums? (RC/STQ/DCAHPS-32)

¹ Excellent

² Very good

³ Good

⁴ Fair

⁵ Poor

53. In the last week, how often did you floss your teeth? (RC/TQ/NewItem)

¹ More than once a day

² Once a day

³ Twice or more a week

⁴ Once a week

⁵ Not at all

54. In general, how would you rate your overall health? (RC/Q/QHP-58)

¹ Excellent

² Very good

³ Good

⁴ Fair

⁵ Poor

55. In general, how would you rate your overall mental or emotional health? (RC/Q/QHP-59)

¹ Excellent

² Very good

³ Good

⁴ Fair

⁵ Poor

56. How confident are you that you understand dental insurance terms? (RC/STQ/QHP-82)

¹ Not at all confident

² Slightly confident

³ Moderately confident

⁴ Very confident

57. How confident are you that you know most of the things you need to know about using dental insurance? (RC/STQ/QHP-83)

¹ Not at all confident

² Slightly confident

³ Moderately confident

⁴ Very confident

58. What is your age? (RC/STQ/DCAHPS-33)

¹ 18 to 24

² 25 to 34

³ 35 to 44

⁴ 45 to 54

⁵ 55 to 64

⁶ 65 to 74

⁷ 75 or older

59. What is your sex? (RC/STQ/QHP-75)

¹ Male

² Female

60. What is the highest grade or level of school that you have completed? (RC/STQ/DCAHPS-35)

¹ 8th grade or less

² Some high school, but did not graduate

³ High school graduate or GED

⁴ Some college or 2-year degree

⁵ 4-year college graduate

⁶ More than 4-year college degree

61. Are you Hispanic, Latino/a, or Spanish origin? (RC/STQ/QHP-78)

¹ Yes, Hispanic, Latino/a, or Spanish origin

² No, not of Hispanic, Latino/a, or Spanish origin  If No, go to #63

62. Which group best describes you? (RC/STQ/QHP-79)

¹ Mexican, Mexican American, Chicano/a

² Puerto Rican

³ Cuban

⁴ Another Hispanic, Latino/a, or Spanish Origin

63. What is your race? Mark one or more. (RC/STQ/QHP-80)

¹ White

² Black or African American

³ American Indian or Alaska Native

⁴ Asian Indian

⁵ Chinese

⁶ Filipino

⁷ Japanese

⁸ Korean

⁹ Vietnamese

¹⁰ Other Asian

¹¹ Native Hawaiian

¹² Guamanian or Chamorro

¹³ Samoan

¹⁴ Other Pacific Islander

64. Did someone help you complete this survey? (RC/STQ/DCAHPS-38)

¹ Yes

² No ® Thank you.
Please return the completed survey in the postage-paid envelope.

65. How did that person help you? Check all that apply. (RC/STQ/DCAHPS-39)

¹ Read the questions to me

² Wrote down the answers I gave

³ Answered the questions for me

⁴ Translated the questions into my language

⁵ Helped in some other way

Please print:

Thank you.

Please return the completed survey in the postage-paid envelope.

Attachment B – Cognitive Testing Eligibility Screener

Marketplace Dental Survey Cognitive Testing Eligibility Screener

Recruitment Specifications

AIR will conduct two rounds of cognitive testing of the Marketplace Dental Survey in both Washington, D.C. and Austin, TX. Each round of cognitive testing will be conducted with 12 English-speaking and 12 Spanish-speaking participants. Thus, we will test the survey with a total of 48 participants. Our goal is to interview individuals who have purchased dental insurance through the Marketplace—both individuals purchasing a stand-alone dental plan (dental insurance for dental care only) and individuals who purchased dental insurance through an embedded plan (a health insurance plan that includes dental benefits).

Table 1. Number of Cognitive Testing Participants

Round #	Dates	Number of participants in Washington, D.C.		Number of participants in Austin, TX		Total number of participants
		English speakers	Spanish speakers	English speakers	Spanish speakers
1	Week of 12/7/2015	6	6	6	6	24
2	Week of 2/1/2016	6	6	6	6	24
Total		12	12	12	12	48

• Dates

  • The first round of cognitive testing will be held during the week of 12/7/2015.

  • The second round of cognitive testing will be held during the week of 2/1/2016.

• Locations

  • AIR Washington D.C. office: 1025 Thomas Jefferson St. NW, Washington, D.C., 20007

  • AIR Austin Office: 4700 Mueller Blvd., Austin, TX, 78723

• Interview length

  • Each cognitive interview session will last approximately 60 minutes.

  • Participants will receive $100 for their time and participation.

• Eligibility Criteria

To be eligible to participate in the cognitive interview, a person must meet the following four criteria:

• Has purchased a dental plan through a Marketplace

  • Washington D.C. participants must be Washington D.C. residents and they must have purchased a dental plan from DC Health Link

  • Austin, TX participants must have purchased a dental plan from HealthCare.gov

• Has received dental care within the past 6 months

• Is between the ages of 18 and 64 years old

• Is an English speaker OR is an individual with limited English proficiency who speaks Spanish as his or her primary language

• Among those who are eligible to participate, we also seek diversity in terms of the following characteristics:

  • Race and ethnicity

  • Education level

  • Type of dental plan purchased

    • A dental insurance for dental care only

    • A health insurance plan that includes dental benefits

  • Health literacy level

  • Spanish, Hispanic, or Latino background

  • Having a disability

Our cognitive testing eligibility criteria and targets for each round of cognitive testing are listed in Table 2, on the next page.

Table 2. Eligibility criteria and recruitment targets per round (N=24)

Eligibility Criteria	Recruitment Targets (# of people)
Participants must meet the following four criteria:
Purchased a dental plan through a Marketplace	12 from DC Health Link (all must be Washington, D.C. residents) 12 from Healthcare.gov (Austin, TX participants only)
Received dental care within the past 6 months	All 24 must have received dental care within the past 6 months.
Age	All 24 must be between ages 18 and 64 years old
Primary language	12 English-language speakers 12 Spanish-language speakers
Among those who are eligible to participate, we also seek diversity in terms of the following characteristics. Please note that recruitment targets listed below are approximate targets:
Race/ethnicity	12 Hispanic or Latino individuals who speak Spanish as their primary language 0 – 2 Hispanic or Latino individuals who speak English as their primary language 2 – 4 Asian / Pacific Islander 2 – 4 Black / African American 2 – 4 White 0 – 1 Other, American Indian, or Alaskan Native
Education level	2 – 6 No high school diploma and no GED 2 – 6 High school diploma or GED 2 – 6 Some post high school (technical / vocational school, associate’s degree, or some college, no Bachelor’s degree) 1 – 5 Bachelor’s degree 0 – 1 Graduate degree (master’s or doctorate)
Type of dental plan purchased	Note that AIR will provide lists of dental insurance and health insurance plans to the recruitment firm. Recruiters will use these lists to verify plan information provided by potential participants. 18 – 20 Dental insurance for dental care only 4 – 6 Health insurance that includes dental care
Health literacy level	Health literacy level will be determined based on responses to Questions #16 and #17. 12 - 20 Individuals with low health literacy A person is considered to have low health literacy if he/she responded: “not at all confident” or “slightly confident” with health insurance terms (Question #17), OR “moderately confident” with health insurance terms (Question #17) AND this is the first time that he/she has ever had dental insurance (Question # 16) 4 - 8 Individuals with high health literacy A person is considered to have high health literacy if he/she respondes: “very confident” with health insurance terms (Question #17)
For Spanish-language participants only: Spanish, Hispanic, or Latino background	12 – 15 Mexican or Mexican American 9 – 12 Other Spanish, Hispanic, or Latino background
Having a disability	A disability is a physical or mental impairment that substantially limits one or more major life activities. At least 2 individuals who report having a disability

Screening Script

Introduction When a Person Answers the Phone

Hello, may I please speak with [FIRST AND LAST NAME]? My name is [RECRUITER’S FIRST AND LAST NAME], and I'm calling to see if you would like to participate in a research project.

 IF RESPONDENT WAS ON ENGLISH (SPANISH) RECRUITMENT LIST:

We can also speak in Spanish (English), if you prefer. Would you like that?

I’m calling today about a research project sponsored by the Centers for Medicare & Medicaid Services. This federal government agency runs the federal Health Insurance Marketplace, Medicare, and Medicaid. We would like to talk to you about participating in an interview to help us develop a new survey about dental insurance available through [DC HEALTH LINK or HEALTHCARE.GOV]. Would this be a good time for us to talk?

 IF RESPONDENT IS AVAILABLE, EXPLAIN THE REASON FOR CALLING:

The research project I am calling you about will help us learn about people’s experiences with dental insurance available on [DC HEALTH LINK or HEALTHCARE.GOV], also known as Obamacare. By answering questions about the survey being developed, you will help us improve a survey that can use to better meet the needs of people who buy dental insurance from [DC HEALTH LINK or HEALTHCARE.GOV].

If you would like to help with this project and your background and experiences are a match for the topics we are studying, we will invite you to come for an individual interview at the American Institutes for Research offices located in [GEORGETOWN or AUSTIN]. It would take about 60 minutes of your time, and you would receive $100 dollars for participating.

IF NEEDED, EXPLAIN FURTHER:

Since we need to include people who are a mix of different backgrounds and experiences, there are some requirements that I have to check on for all of the people that we bring in to participate. I need to ask you a few questions to see if you meet the requirements for participating in the interview.

IF YES, CONTINUE WITH QUESTION #1.

IF NO:

Thank you for taking the time to speak with me. I hope you have a good day.

 IF THE PERSON IS NOT AVAILABLE, EITHER:

• DETERMINE A GOOD TIME TO CALL BACK or

• LEAVE A VOICEMAIL MESSAGE

Voicemail Script

Hello, my name is [RECRUITER’S FIRST AND LAST NAME]. I’m calling today about a research project sponsored by the Centers for Medicare & Medicaid Services. We would like to talk to you about participating in an interview to help us develop a new survey about dental insurance available through [DC HEALTH LINK or HEALTHCARE.GOV]. If you qualify for the project you will receive $100 dollar for participating. Please call [PHONE NUMBER] at your convenience, to learn more.

Eligibility Screener

Before we start, I want to let you know that all information you provide will be confidential. None of the information that you provide to us will be used for any purpose outside of this study. You can skip any questions that you do not want to answer and you can stop at any time. Just let me know.

1. About how old are you?

Younger than 18  Not eligible, go to End Script

Between 18 and 29

Between 30 and 54

Between 55 and 64

Older than 64  Not eligible, go to End Script

2. When was your last dental appointment?

Within the last six months

More than six months ago  Not eligible, go to End Script

Don’t Know  Not eligible, go to End Script

3. For Washington D.C. only: Are you a Washington D.C. resident?

Yes

No  Not eligible, go to End Script

4. Did you buy dental insurance through [for Washington D.C. only: DC Health Link or for Texas only: HealthCare.gov] in the last year? This can be for dental care only (a stand-alone dental plan) or for dental care that is included as part of your overall health insurance or medical plan.

Yes

No  Not eligible, go to End Script

5. What language do you speak most often at home?

English  Skip to Question #7

Spanish

Other (please specify): ____________________

6. How well do you speak English?

Very well   May be eligible for the English-language interview

Well   May be eligible for English-language interview

Not well   May be eligible for Spanish-language interview

  But, if also answered “other” language to Question #5, not eligible. Go to End Script

7.   But, if also answered “other” language to Question #5, not eligible. Go to End ScriptDid you buy dental insurance for dental care only or did you buy health insurance that includes dental care?

Dental insurance for dental care only  Go to Question #8

Health insurance that includes dental care  Go to Question #9

8. What is the name of the dental insurance that you bought through [DC Health Link or HealthCare.gov] in the last year?

Name of Dental Insurance Plan: ­­­­­­­­­­­­___________________________________________

 Search the dental insurance plan list to see if the plan qualifies.

 If the dental plan is on the list, go to Question #11

 If the dental plan is not on the list, the person is not eligible. Go to End Script

Don’t Know   Ask the person to get their dental insurance card to look at the name

9. What is the name of the health insurance that you bought through [DC Health Link or HealthCare.gov] in the last year?

Name of Health Insurance Plan: ­­­­­­­­­­­­___________________________________________

 Search the health insurance plan list to see if the plan qualifies

 If the health insurance plan is on the list, go to Question #11

 If the health insurance plan is not on the list, the person is not eligible. Go to End Script

Don’t Know  Ask the person to get their health insurance card to look at the name

10. What is your gender?

Female

Male

11. What is the highest grade or level of school you completed?

Less than high school graduate

High school diploma

GED

Technical or vocational school or certificate program

Associate’s degree (2 year college graduate)

Some college (1 to 3 years of college, no degree)

Bachelor’s degree

Graduate degree (master’s or doctorate)

12. Are you Hispanic, Latino/a, or Spanish origin?

[If Question #10=Female, use Latina. If Question #10 Male, use Latino]

Yes

No  Go to Question #14

13. Which of the following groups best describes you?

[If Question #10=Female, use Mexican, Mexican American, Chicana, Latina. If Question #10 Male, use Mexican, Mexican American, Chicano, Latino]

Mexican, Mexican American, Chicano/a, Latino/a

Puerto Rican

Cuban, or

Another Hispanic, Latino/a, or Spanish Origin

14. Read each question below with a yes/no response:

YES NO

A. Are you White?

B Are you Black or African American?

C. Are you American Indian or Alaska Native?

D. Are you Asian?

E. Are you Native Hawaiian or Pacific Islander?

F. Are you another race?

15. We would like to audio record your interview to make sure we do not miss any information. Only the researchers have access to the recording. Would you be willing to be audio recorded?

Yes

No

16. Is this the first time that you have ever had dental insurance?

Yes

No

17. How confident are you that you understand dental insurance terms/language?

Not at all confident

Slightly confident

Moderately confident

Very confident

18. A disability is a physical or mental impairment that substantially limits one or more major life activities. Have you had a disability in the past 6 months?

Yes

No

Invitation for Eligible Individuals

Thank you for answering all of my questions. Based on your answers, your background and experiences are a fit for this study. I would like to invite you to join the interview.

We are having interviews on _________________ . The interview will take about 60 minutes and you will be given $100 for participating.

We have several days and times available. Let me know what works best for you. We have open:

Day 1	9:30 am -10:30am	11:30 am – 12:30 pm	1:30 pm – 2:30 pm	3:30 pm – 4:30 pm	5:30 pm – 6:30 pm
Day 2	10:00 am -11:00 am	12:00 pm – 1:00 pm	2:00 pm – 3:00 pm	4:00 pm – 5:00 pm

Would you be able to come to the interview on one of those dates and times?

IF NO:

Is it okay for me to call you if we have new days and times open up for the interview? Thank you for taking the time to speak with me. I hope you have a good day.

IF YES: CONFIRM APPOINTMENT AND RESPONDENT’S INFORMATION

So, let me confirm that you are agreeing to participate in an interview about your experiences with dental health insurance on [DATE AND TIME].

Let me just mention two things:

1. If you wear reading glasses, please be sure to bring them to the discussion, as there may be some reading involved; and

2. Please be aware that we have a no-smoking policy.

Now, let me just make sure I have your name spelled correctly and ask for your email and/or mailing address so we can send you an interview confirmation letter with directions. We will only use this information for this study and nothing else.

Name: ______________________________________

Email: ______________________________________

Address: _____________________________________________________________________

City, State: ____________________________________________Zip:____________________

Also, please let me make sure that I have all of your correct phone numbers so that we may call and confirm with you.

Cell phone: ___________________________ Other Phone: __________________________

If you have any questions or cannot make it to the group interview, please call us right away at [PHONE NUMBER] so that we can find someone else to help us. Thank you for your time and for agreeing to help.

End Script – For People Who Do Not Meet the Eligibility Criteria

Thank you very much for answering my questions. As I said earlier, we need to invite people with different backgrounds and experiences related to our study. Unfortunately, the information you shared does not match our study OR we already have enough people in our study with backgrounds like yours. I appreciate your taking the time to speak with me and I hope you have a good day.

Attachment C – Cognitive Testing Informed Consent Form

Marketplace Dental Survey

Cognitive Testing Interview Information & Consent Form

Thank you for meeting with us today. Please read the information below that explains the study. Ask me if you have any questions. Then, if you want to participate, please sign your name at the bottom.

What is the project about and what will you ask me to do?

The purpose of this project is to develop a dental survey to collect information about consumer experiences with their dental plans that they bought on [HealthCare.gov or DC Health Link] and their dental care. We are interviewing people to make sure that the survey questions are clear and to understand how people would choose their answers to these questions. The interview will take about 60 minutes. You will receive $100 to thank you for your time and participation.

Who is doing this project?

This project is being done by the American Institutes for Research (AIR), a non-profit research organization. The project is funded by the Center for Medicare & Medicaid Services (CMS), a government agency.

What is involved in the interview?

The interview will last about 1 hour. During that time, you will help us to improve the survey by providing your thoughts about the survey questions.

You will meet with two staff members from AIR. One staff member will be interviewing you and the other staff member will be taking notes. The interviewer will ask you to read the survey questions and to say out loud what you are thinking as you answer them. There are no “right” or “wrong” answers to these questions. Your honest feedback will help us to make sure that the survey will be easy for other people to understand.

Do I have to participate in this project?

No. It is your choice whether or not to participate. Also, you have the right to stop participating at any time, without penalty. If you choose not to participate or stop participating, there are no penalties and you will still receive the $100 to thank you for your time.

Will you be recording the interview today?

Yes. With your permission, we will audio-record the interview. We may share what you say in reports and publications. However, all information you provide today is confidential. This means that when we share or reproduce information, we will never include your name or identifying information (like your address). The recordings will be destroyed at the end of the project.

How will you protect my privacy?

We will not use your name in connection with anything you say, and we will not give your name to anyone outside of the project. Again, all information you provide today is confidential.

What are the risks?

There are no known risks to being part of this interview that are any different from what you might experience in your everyday life: for example, the time spent in the interview or any travel costs. If the discussion makes you uncomfortable, then you do not have to participate and you still will be paid.

Are there any benefits?

Although there is no direct benefit to you, your participation will help us develop a Marketplace Dental Survey to understand how well dental health plans are meeting consumers’ needs.

What if I want more information?

• If you want more information about this project, please contact the Project Director at AIR, Elizabeth Mokyr Horner, M.P.P., Ph.D., [email protected], 1-650-843-8220.

• If you have questions about your rights as a participant, contact the chair of AIR’s Institutional Review Board at [email protected] or toll-free at 1-800-634-0797 or c/o AIR, Attn:  AIR IRB, 1000 Thomas Jefferson Street, NW, Washington, DC 20007.

Please sign below if you agree to participate

Signing your name below means that you are giving your “informed consent” to participate today. This means that you have read and understood the information on this form, you have had a chance to ask questions, and you are willing to participate under the conditions we have described. Your signature does NOT allow AIR or CMS to identify you by name when your comments are used.

Your signature: _______________________________ Today’s date: ______________________

Please print your name: _________________________

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid Office of Management and Budget (OMB) control number. The valid OMB control number for this information collection is XXXX-XXXX.

Attachment D – Marketplace Dental Survey Cognitive Testing Guide

Marketplace Dental Survey Cognitive Interview Guide

Washington D.C. and Austin, TX, December 2015

Goals

We will conduct cognitive interviews with Marketplace dental plan consumers to test the validity of the draft Marketplace Dental Survey (MDS). Our overall purpose will be to evaluate cognitive interview participants’ understanding of the survey items. The data that we collect from cognitive interviews will help ensure the items are reliable, valid, address key domains, and minimize respondent burden. The goals for cognitive interviewing are to:

1. Determine whether items are understood as intended (making sure questions are clear and aligned with the goal of the question)

2. Determine if there are differences in how participants understand survey items based on whether:

   1. They have purchased a stand-alone versus an embedded dental plan

   2. They have purchased a dental plan from a state-based marketplace versus through HealthCare.gov

3. Ensure that the English and Spanish-language survey questions are conceptually equivalent -- that the English-language content has been translated effectively into Spanish.

4. Determine whether participants are having trouble recalling experiences of their dental care which occurred in the past six months.

5. Determine whether there are motivational or sensitivity problems that were not previously apparent to item developers

6. Explore the impact of changing the reference period from 12-months for Dental CAHPS to 6-months, to be congruent with the other CAHPS Surveys and the Qualified Health Plan Consumer Experience Survey.

7. Identify whether response options are appropriate

8. Suggest any revisions necessary to the survey items

Overview of the Cognitive Interview Process

Each interview will involve a two-person team including an interviewer and a note taker. Each interview will last approximately 60 minutes and will be audio recorded. Here are the general steps involved in the cognitive interview process:

• First, the interviewer will review and explain the informed consent form with participants, answer any participant questions, and obtain the participant’s signature if they agree to continue with the interview

• Next, using a question unrelated to the MDS, the interviewer will train participants in the “think aloud” exercise

• Participants will read each question and mark their answer. They will be encouraged to “think aloud” as they answer the survey items. “Think aloud” means to ask the participant to talk out loud about the thoughts that come into their mind when they are thinking about how to answer the question.

• The interviewer will ask general probes during the “think aloud” and then probe on how and why each participant made their response choice.

• If the participant does not answer out loud, the interviewer will ask the participant for their answer to ensure that their responses are recorded in the notes.

• Once the participant has reviewed the entire survey, the interviewer will ask about their overall reactions to the survey.

Probes for Use Throughout the Interview

Think aloud reminders:

• “Remember to tell me your thoughts and reactions as you read the questions.”

• “Can you tell me what you’re thinking about now?”

To elicit further information:

• “And you say that because…”

• “How so?”

• “In what way?”

• “Tell me more about that.”

• “Remember, there aren’t any right or wrong answers. I just want to know your honest opinion. That’s what will help in making improvements to the survey.”

To redirect:

• “And going back to [item/response], I would like to know more about…”

• “Thank you, now I would like to move us on to [next survey item].”

Probe on significant non-verbal communication (e.g., smiles, eye-rolling, etc.). Don’t over-probe on non-verbal communication –i.e., don’t probe to the point that it makes the participant self-conscious.

• “I notice you laughed [paused] as you answered this, can you tell me about that?”

Sample Questions

5. Have you seen your regular dentist in the last 6 months?

¹ Yes

² No, I’ve seen someone else ® If No, go to #14

Purpose of the question: To determine whether the person is eligible to respond to the following questions which refer to visits to the regular dentist.

Goal of the question: To determine whether the respondent has a regular dentist.

Objective: Determine whether the respondent interprets “regular dentist” as intended.

Probes:

1. Who do you think of when you hear the term “regular dentist?” Who does that include?

2. If the respondent answers “No, I’ve seen someone else”: Who were you thinking of when you answered that you’ve seen someone else?

43. In the last 6 months, how often were the forms from your dental health plan easy to fill out?

¹ Never

² Sometimes

³ Usually

⁴ Always

Purpose of the question: To determine whether the forms that the dental plan requires the person to fill out are burdensome. Specifically, this question speaks to the cultural sensitivity of the dental plan.

Goal of the question: To determine whether the forms that the dental plan requires the person to fill out are burdensome.

Objectives:

• Determine what the respondent interprets as “easy to fill out.”

• Determine if the respondent interprets “in the language you prefer” as intended.

• Determine how respondents interpret the term “available.”

Probes:

1. What do you think they mean by “forms?”

2. What do you think they mean by “easy to fill out?”

3. How did you come up with your answer?

1. Tell me about that. What forms have you filled out?

1 Original D-CAHPS composite measures: https://cahps.ahrq.gov/surveys-guidance/dental/about/dental-plan-measures.pdf

2 QHP Survey domains listed in document “English_QHPSurvey2016_091715.” Not publically available.

3 Prabhu, N. T. (2010). Access to dental care--parents' and caregivers' views on dental treatment services for people with disabilities. Special Care in Dentistry, 35-45.

4 Item based loosely on:

Busby, M., Burke, F. J. T., Matthews, R., Cyrta, J., & Mullins, A. (2012). The development of a concise questionnaire designed to measure perceived outcomes on the issues of greatest importance to patients. British Dental Journal, 212(8), E11. doi:10.1038/sj.bdj.2012.315

IPRO. (2007, February). Dental Care Survey Medicaid Managed Care Members. New York State Department of Health Office of Managed Care. Retrieved from Dental Care Survey Medicaid Managed Care Members.

Perera, I. R., & Usgodaarachchi, U. S. (2009). Development of a psychometric scale to assess satisfaction with dental care among Sri Lankans. Community Dental Health, 26(3), 150–156.

5 Closely based on global rating items from DCAHPS (e.g., 10).

6 Based loosely on:

Hurst, op. cit.

Narayanan, A. &. (2014). The Dental Practice Questionnaire: A patient feedback tool for improving teh quality of dental practices. Australian Dental Journal, 334-348.

Mascarenhas, A. K.-H. (2005). Parents' satisfaction with their child's orthodontic care: A comparison of orthodontists and pediatric dentists. Pediatric Dentistry, 451-456.

7 Note that Literature Review and Consumer Focus Groups did not contribute to About You section

24