ID
Number: CMS-10615
OMB Control Number: 0938-1300
Summary of Public Comments on the Healthy Indiana Program (HIP) 2.0 Federal Evaluation
On May 4, 2016 the Centers for Medicare & Medicaid Services (CMS) announced an opportunity for the public to comment on the information collection concerning CMS’ Healthy Indiana Program (HIP) 2.0 Federal Evaluation. Public comments were accepted through June 3, 2016 and will be utilized to finalize the federal beneficiary surveys, focus group moderator guides, and informational interview protocols for the HIP 2.0 federal evaluation. In total we received four public comments from the individual public and the following groups/organizations: Planned Parenthood of Indiana and Kentucky, The State of Indiana’s Office of the Governor, and the National Health Law Program.
The purpose of this document is to provide a summary of these important public comments and our responses to those comments. Our responses to the public comments are outlined below and are organized by general overarching comments, followed by comments specific to particular survey instruments as presented in the public comment documents.
Comment: I totally oppose this survey and this spending. The entire project should be shut down. This unequal provision of services to special groups of people is certainly against the US Constitution that provides equal treatment. This agency is wasteful and its budget needs to be cut on this project.
Response: This evaluation will help inform important policymaking decisions related to Medicaid expansion efforts as provided under the Affordable care Act, and as designed and implemented under the HIP 2.0 demonstration. The HIP 2.0 survey, data collection and federal evaluation are helping test whether and how the policies under HIP 2.0 affect coverage and quality of care for beneficiaries eligible for Medicaid.
(NHeLP) Comment: As a general recommendation, we would like to reinforce that questions directed at a beneficiary audience must be drafted for individuals with a low literacy level. Many beneficiaries read at low literacy levels or have limited proficiency in English (LEP) and may have trouble understanding complex questions. We suggest having literacy experts review the final questions and ensure they do not exceed a 6th grade reading level (or a lower literacy level as may be appropriate for the relevant population).
Response: Prior to submitting the beneficiary survey instruments to CMS and OMB for approval we assessed the English reading grade level of the surveys using an English readability online tool. All instruments reported an average grade reading level of 5. The average grade reading level was calculated based on the Flesch-Kincaid Grade Level.
Where possible we are striving for questionnaire text that meets or is below a 6th grade reading level. We recognize that certain parts of the instruments, such as the burden statement, are above this reading level but must stand as written as required by OMB. Some other triggers for higher grade level ratings are complex sentences, or words that are 3 or more syllables (for example, “enrollment,” “satisfaction” and “transportation”) but are necessary for our inquiry about HIP 2.0 program-related features. When tested with beneficiaries these types of words did not present any problems with beneficiary comprehension.
(NHeLP) Comment: Further, the survey administrator should translate the survey into the top non-English languages in the beneficiary population and ensure sufficient sample size of LEP beneficiaries to obtain validated results. LEP individuals may experience access to healthcare differently than English speakers, particularly if language barriers impede access to care. Thus, the survey must include participation of LEP individuals from all language groups (that is, administering the survey in Spanish would be insufficient if HIP 2.0 also enrolls a significant number of other non-English language speakers). If LEP individuals cannot be identified in advance (to be sent an in-language survey), we recommend that the survey scripts/introductions include tag lines in the top 15 common languages in Indiana with clear instructions how an LEP individual can get assistance responding to the survey. The survey sample should not implicitly omit LEP individuals simply because they are unable to complete a survey in English.
Response: LEP individuals are an important but relatively small proportion of the total population of HIP beneficiaries in Indiana. As of 2012, Hispanics represented 6% of the total population in Indiana. An estimated 3.3% of the total HIP enrollee population in 2012 was Hispanic, and one-third of the Hispanic population in Indiana was estimated to speak English less than very well (American Community Survey, 2012). Due to time and cost considerations for the evaluation, we chose to focus our efforts on English and Spanish speakers only, noting that a very small proportion of Hispanic HIP beneficiaries are anticipated to be LEP (about 1% of the total HIP enrollee population). Nevertheless we are prepared to accommodate Spanish language questionnaires and/or interviewers for up to 10% of survey participants. Proportions of LEP among other racial/ethnic groups are likely not sizeable due to the lower representation of those groups in Indiana compared to Hispanics.
(PP) Comment: CMS should simplify the beneficiary survey and make sure to provide the information in different languages. CMS should ensure that the survey is accessible to all enrollees. That includes simplifying the language so that people who have lower reading levels can meaningfully engage in the survey and provide critical feedback of the program. Currently, the reading level for the beneficiary survey is above a sixth grade level, which could be frustrating for individuals who have lower reading levels. (Footnote: Portions of the survey were run through a Flesch-Kincaid Grade Level Test and the reading level scored above the sixth grade level and instead fell in the college reading level range.) Equally important is providing the beneficiary survey in other languages so that participants who have low English proficiency can participate in the survey and provide clear and concise feedback to CMS.
Response: Please see the response to comments 2 and 3
(PP) Comment: The survey and focus group questions on preventive care should clarify that preventive services includes women’s preventive care, such as family planning services and prenatal care, and assess whether HIP 2.0 enrollee have been charged for women’s preventive care or faced other impediments when accessing women’s preventive services, particularly family planning services.
We ask that the survey and focus group questions on preventive care define preventive care to include examples of women’s preventive care (namely birth control, well-woman exams, and prenatal care) to make it more accessible and inclusive for female enrollees.
For example, the preamble language in the survey could read: “Preventive services are routine health care services that include getting a flu shot, annual checkups, family planning services, prenatal services, blood pressure checks, cholesterol screenings, or cancer screenings to prevent illness, disease, unintended pregnancies, maternal morbidity, and other health-related problems. The following question asks about your experience with preventive services and POWER accounts.”
Likewise, question 30 in the focus group survey could read:
“Have you received preventive care (like health checkups, well-woman exams, or immunizations) more frequently now, with HIP 2.0, than you did in the past? If yes, can you tell me why? How does that feel?
Response: We appreciate the input of Planned Parenthood and their pointing out of the fact that our draft focus group moderator’s guide did not sufficiently include women’s health services within its example definition of “preventive care.” The language suggested by Planned Parenthood is helpful, and will be happy to include “well-woman exams” in question 30 of our moderator’s guide.
Additionally, revisions have also been made to include “family planning services” and “prenatal services” in all places in the survey instruments where examples of preventive services are mentioned. We intended the questions to provide a few examples of preventive services and not necessarily a comprehensive list.
(PP) Comment: We also ask that the survey and focus group questionnaire directly address whether individuals are required to use their POWER account to pay for women’s preventive care. Federal law exempts women’s preventive services (including all 18 FDA-approved birth control methods, well-woman exams, and prenatal care) from cost-sharing for the new adult group. While we appreciate that question 37 in the survey asks if the cost of preventive care is paid for through the POWER account, we urge CMS to include another, more direct question like:
Please tell us whether you agree or disagree with the following statement: I have never been asked or required to use my POWER account to pay for preventive services such as a flu shot, annual checkups, family planning services, prenatal services, blood pressure checks, and cholesterol screenings.
Agree
Disagree
Not sure/Don’t know
Response: We appreciate the focus that Planned Parenthood has placed on our lack of inclusion of examples of women’s health services within our focus group moderator’s guide. We agree that it would be appropriate to include a question or probe in our bank of questions surrounding use of POWER accounts that explores whether or not women have been required or asked to pay for women’s preventive services (such as birth control, well-women exams, or prenatal care). In our revised draft, we will include such a question.
The proposed survey question referenced in comment #6 is intended to obtain beneficiary understanding with regard to their POWER account and preventive services. While we understand the value in gathering this information, we are most interested in assessing beneficiary understanding of the cost of preventive services (including prenatal and family planning services) and POWER accounts. Since the question regarding whether or not beneficiaries were ever asked or required to use their POWER account to pay for preventive services will be included as part of the focus groups, and can also be assessed via administrative data - in the interest of keeping respondent burden low we have opted not to include this question in the federal surveys. Beneficiary experience regarding preventive services is captured more broadly in another survey question where a specific list of preventive services is not provided but asks whether beneficiaries had a routine general checkup.
(PP) Comment: Importantly, there were no questions in the focus group questionnaire that address enrollee’s experience with being charged for preventive services. Because cost is a considerable barrier to care for low-income health consumers – and because the federally-approved HIP 2.0 waiver requires coverage of preventive care and federal law requires coverage of women’s preventive care without cost – we strongly recommend that the focus group questionnaire inquire whether enrollees have been charged for preventive care.
Response: We agree with this observation and thank Planned Parenthood for pointing out this gap in our focus group moderator’s guide. In response, we will add a question (and probes) to directly ask focus group participants whether they have been charged for women’s preventive services, and probe surrounding the circumstances under which these charges were imposed, by whom, and whether they caused women to forego services due to cost.
(PP) Comment: CMS should incorporate a question in the survey that assesses enrollees’ abilities to receive care in a timely manner.
We appreciate that the focus group questionnaire asks for enrollees to speak to their experiences accessing care under the HIP 2.0 program. However, similar questions are missing from the enrollee survey. There are no questions that speak to an enrollee’s potential difficulty finding a provider, or potential experiences with delayed access. Incorporating questions into the survey on whether enrollees have been able to access providers, receive same-day appointments, and appointment wait times will enable CMS to get a detailed understanding of whether HIP 2.0 meets its intended goals and if the state needs to alter the program to address existing barriers to care.
Response: There are some access to care questions embedded within the HIP 2.0 features that the beneficiary survey is focused on (i.e., transportation, emergency room, POWER accounts, contributions, etc.). The impact analysis under the federal evaluation of the HIP 2.0 demonstration will examine access to care using other data sources.
(IN) Comment: Our concerns revolve around several areas. First, there are many examples of where the survey questions are biased, and response options are leading. Secondly, the questions show a lack of understanding of how the HIP program operates today and could produce misleading results. Third, there are flaws within the survey structure and a number of questions phrased in a way that may be confusing to members – both of which could lead to distorted results. Finally, several questions that were already asked in state’s survey were duplicated in the CMS survey while others were omitted. We would like to understand why and the decision-making process that led to it. We would also like to request a copy of the analytical plan, including an overview of the proposed research questions, measures, comparison populations, and statistical methods.
Response: We appreciate the state’s review of the federal survey questions; we find the state’s comments very helpful. We agree with the objective that questions should be clearly stated and unbiased, and have continued to welcome public input as is provided under PRA to improve the surveys for the federal evaluation of the HIP 2.0 demonstration. Each specific comment provided, including any comments about the state’s perception of bias and of lack of understanding of the HIP program has been carefully reviewed. In the majority of cases, the questions have been clarified and adjusted to address the state’s concern. In some instances, we clarify the intent of the question to support understanding of its purpose and focus, and in other instances we clarify how the structure of the survey already clearly addresses the concern raised. The specific details of this review and our responses are provided below, for each specific comment.
Regarding duplication, as described in detailed briefing materials and several presentations provided to the state by CMS and its evaluation contractor, the federal evaluation of the HIP 2.0 demonstration is not intended to duplicate Indiana’s evaluation. Rather the two evaluations are intended to be complementary, with Indiana’s evaluation providing an in-depth look at each of the components of the demonstration, while the Federal evaluation assesses a subset of key elements of the demonstration that are unique to Indiana and examines the impacts of those policies, including as compared to expansion in other states.
The federal evaluation is focusing its surveys on beneficiary understanding and experience under the HIP 2.0 policies. The state’s beneficiary survey questions were reviewed and considered, along with other sources of questions, and we selected or developed the best set of questions for the federal evaluation which will support analysis of beneficiary understanding and experience under the HIP 2.0 policies with an emphasis on premiums, lock out, emergency room copayments when used in non-emergency situations, POWER accounts, and non-emergency medical transportation (NEMT).
Where the research questions in the two evaluations are similar, the federal evaluation will apply alternative data, comparison groups and / or research methods to provide a thorough and reliable understanding of the impacts of these HIP 2.0 policies on Medicaid beneficiaries. CMS expects the draft federal evaluation design report to be completed in late June, 2016. As CMS has indicated to the state in weekly meetings about the HIP 2.0 demonstration, the federal evaluation design report will be provided to the state, and CMS has offered to present the federal evaluation design again to the state at that time.
(IN) Comment: It is critical to allow a “don’t know” or “not sure,” or “prefer not to answer” option for all of the questions in the surveys. Yet a large number lack these options. Forcing an answer, as these surveys do, can lead to mistaken reporting and biased, inaccurate results. For example, one question asks, (CMS HIP 2.0 Enrollee Survey. Question 14, page 4).
What does HIP 2.0 say you should do if you think you need to go to the emergency room? Mark one or more:
Go directly to the emergency room
Call the phone number or hotline provided by HIP 2.0
Call my doctor
Ask my family or friends
By not allowing an option of “don’t know”, “not sure,” or “prefer not to answer”, this question forces members into providing responses (three of which within this example are inconsistent with HIP 2.0 policy) which encourage biased and inaccurate survey results.
Response: Note that we have changed the phrasing of this question in response to your comment 12 below.
What does HIP 2.0 say you should do if you think you may need to go to the emergency room, but are not sure? Mark one or more:
Go directly to the emergency room
Call the phone number or hotline provided by HIP 2.0
Call my doctor
Ask my family or friends
As to including a broader “don’t know” or “prefer not to answer” response, we are concerned about losing useful information regarding what beneficiaries believe HIP 2.0 policy to be. This question tests whether beneficiaries understand how copayments are applied for emergency room services; one answer is the correct answer while the others are actions that a beneficiary might reasonably think are also acceptable to avoid a copayment if they do not understand the policy. Hence the structure of this question will provide more rich information about beneficiary understanding than it would if there were “don’t know” responses.
Concerning the comment to include ‘prefer not to answer’, it is stated in the survey instructions that if respondents do not want to answer particular questions or do not feel comfortable providing a response they may skip that particular question. All survey modes will mirror the mail-in paper version to the extent possible. The online survey questions are not mandatory and respondents may skip any questions they prefer not to answer or refuse.
(IN) Comment: In addition, the CMS surveys do not include questions about how members travel to their healthcare appointments. This omission is curious since CMS insisted that Indiana include a question on mode of transportation within its member survey.
Response: As indicated in comment response #8, the state and federal evaluations are different and complementary to provide a thorough and reliable understanding of the effects of the HIP 2.0 policies; this also applies to the NEMT analysis. Since the Federal evaluation, including on NEMT, is not intended to provide an in-depth assessment of the different elements of Indiana’s demonstration, the federal survey does not include questions about transportation mode as part of the Federal Beneficiary Surveys.
In selecting the NEMT questions for the federal evaluation, we considered transportation-related questions from the Indiana surveys and the Iowa Wellness Plan Survey. We also considered comments from Mathematica Policy Research (MPR) on the Iowa NEMT evaluation proposal as guidance.
Please also see the response to comment 8.
(IN) Comment: Many of the questions are inaccurate, because they do not reflect the current policy and/or operations of the HIP program.
For example, many of the questions throughout the surveys address enrollees’ “choices”.
These questions may confuse members since they imply that members make an active choice on their enrollment form to enroll in HIP Plus versus HIP Basic. HIP Plus is the initial plan designation for all members. If they make their initial POWER account contribution (PAC), they remain in HIP Plus. If they fail to contribute and are below 100 percent of the federal poverty level (FPL), they are automatically transitioned to HIP Basic. The surveys do not contemplate this, and many of the questions in all the surveys are flawed to this end.
In addition, the relationship between the nurse hotline and the emergency room (ER) copay fee are not addressed correctly in the questions. We recommend reviewing our CMS-approved ER protocols to better understand how the program works. Other examples of inaccuracies are outlined below.
Response: Thank you for bringing this to our attention. While we understand that for those at or below 100% of the FPL the distinction between HIP Basic and HIP Plus is the “default option” versus the “preferred plan” respectively, as you know individuals at or below 100% of the FPL would still be eligible for either the HIP Basic or HIP Plus benefits package. However to avoid confusion, we have eliminated use of the word “choice” and omitted the following survey questions from all surveys:
“When you enrolled in HIP 2.0, did you have a choice between HIP Basic or HIP Plus?”
Furthermore, in the New Enrollee Survey Instrument we have revised the language to address the state’s intention with respect to the policy structure in HIP 2.0.
The CMS approved ER protocols state that if a member calls the Nurse helpline prior to seeking emergency care then the member will not be subject to a copayment, even if the ED visit is determined to be non-emergent. The survey question “What does HIP 2.0 say you should do if you think you may need to go to the emergency room, but are not sure? Mark one or more”” is intended to assess beneficiary awareness of the Nurse helpline. The ER protocols also state that beneficiaries will be educated about the copayment responsibilities.
(IN) Comment: There are many examples of where the question, as drafted, is unclear, which could lead to biased results. For example, throughout all of the surveys, there are problems with verb tenses in many of the questions and responses. We recommend that all of the questions and responses be reviewed for this issue. In addition, disenrolled members should be asked if they voluntarily left or were disenrolled involuntarily (for non-payment or non-compliance related to eligibility). Also, understanding what plan the member thought they were in is also important, as the survey asks questions about the Basic plan separate from the Plus plan. The questionnaires should ask which plan the member had and whether they switched from one to the other.
Response: We appreciate your comments and have taken them into consideration during the survey revisions. All of the verb tenses have been noted and updated in all relevant questions.
Reasons for involuntary disenrollment have been added to the disenrollee survey question and the question structure has been revised.
Additionally, during the survey development an expanded list of reasons for disenrollment was explored but ultimately the answer options and reasons for disenrollment were simplified. We primarily are interested in differentiating between “voluntary disenrollees” and those that were “locked out,” and include a question that offers a list of reasons why they left HIP 2.0 (voluntarily or involuntarily). For example, survey question 12 in the disenrollees and lockouts survey asks respondents why they left HIP 2.0. The answer options provided will help distinguish respondents who were locked out due to non-payment and those that left HIP 2.0 due to non-compliance (e.g., ineligibility due to an income increase, did not finish paperwork, , etc.)
We did not include a direct question that asks respondents which HIP 2.0 benefits package they think they have. As pointed out, respondents may not know which plan they are in, or if they were moved from one to another. So that we are able to ask the appropriate set of follow-up questions we ask respondents if they contribute/had contributed monthly or annually to determine which HIP 2.0 benefits package they are/were in and if they are aware of the plans (HIP Basic and HIP Plus) in general.
(IN) Comment: The POWER account definition utilized in the survey instruments is not accurate and refers to POWER accounts as “special savings accounts.” They should be referred to as “health savings accounts.”
Response: We appreciate your comment, the POWER account definition has been updated to refer to them as ‘health savings accounts.’
(IN) Comment: The Demographics section in all surveys should include a “Prefer not to answer” answer option.
Response: The survey instruments do not include any “Prefer not to answer” answer options for two primary reasons. (1) As a point of reference, the Nationwide Medicaid CAHPS survey that the evaluation team has drawn questions from (mail-in paper format version) also does not include “Prefer not to answer” response options in their questions. CAHPS has been the national standard for measuring and reporting on the experiences of consumers with their health plans. The development of CAHPS surveys is governed by a set of principles meant to ensure that the surveys are scientifically sound and provide information that is specific, understandable, and actionable. (2) The survey instructions (across all three instruments) state “You may skip any questions that you do not feel comfortable answering”. The inclusion of this statement in the instructions is sufficient in reminding respondents that they may skip any questions they prefer not to answer.
(IN) Comment: The Demographics survey question asking about family size and total income over the last year should be broken up into two separate survey questions.
Response: This question structure is the same structure used in several beneficiary surveys with various modes that have previously collected FPL data, and those data seem to have matched up well with data from the American Community Survey (ACS).
This question structure was tested and not found to be problematic. Updates to the formatting and language are being made to simplify selection of family size and its associated family income levels. The web version of the survey first asks about family size; the follow up family income question only displays the income categories relevant to the respondent’s indicated family size.
(NHeLP) Comment: Questions 48-52 on Health Coverage Cost: We appreciate the revisions to the previous draft questions on health care coverage, but do not feel they have resolved our previously expressed concerns. Several questions in this draft about preferences for health plan cost structure remain unclear and likely confusing for respondents. Others are highly leading.
First, we appreciate that the script now clearly defines copays, but it should also define other relevant terms (like monthly contribution and benefit package) and explain how they are connected. The script should make clear that the amounts of contributions and copays are related in health plans, with higher contributions corresponding with lower copayments.
Response: We have taken your comments into further consideration and have made additional survey revisions to the Health Coverage Cost and Payment Options survey section. This particular survey section will only include two survey questions:
Q. If you could choose how to pay for your health care services, what would you choose?
I would choose to pay copays at my health care visits
I would choose to make monthly or annual contributions
It does not matter to me
Q. How important are each of the following factors when thinking about enrolling in a benefits package?
Please mark one answer in each row. |
Very important |
Somewhat important |
Not at all important |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The revisions help increase clarity and assess a broad level of beneficiary understanding with respect to their health coverage more generally.
(NHeLP) Comment: Second, question 48 remains fatally flawed to the point where we question its inclusion in the survey. What respondent would say they do not want choice? Furthermore, someone who cannot afford monthly contributions may have no true choice between HIP’s two benefits packages. We recommend eliminating or completely restructuring this question. Perhaps it could be designed as a ranking list to identify what people prioritize in choosing a health care plan (network, cost structure, managed care company, etc.)
Response: We have taken this comment into consideration and have omitted this survey question. Please see the response to comment 16.
(NHeLP) Comment: The hypothetical plan comparison applicable to questions 50 - 52 apparently attempts to measure if individuals would be willing to pay extra in copays in exchange for a shorter lockout. This is more refined than the prior draft, but the scenario still makes it impossible to meaningfully choose between health plans. It is unclear what is meant by “may have copays” in Package Y, and there are no relative costs attached. The answers would likely differ if the copay was $1, $4 or $15, or if the monthly contribution was $5, $15, or $25. Without specific details, it becomes difficult to draw any valid or actionable conclusions from the answers collected (whatever those answers may be.)
A more useful approach would be to test the same questions using different dollar amounts for monthly contributions and premiums, to gauge quantitatively how respondents value the risk of missing a payment.
Response: During the survey development process we developed iterations of questions using the same approach proposed in the comment above. After soliciting feedback from several survey experts and through cognitive testing/debriefing, we learned we needed to keep the questions as simple and few as possible. We have therefore omitted this survey question because of its complexity. Please see the response to comment 16.
(NHeLP) Comment: Recommendations for Health Coverage Cost and Payments
Eliminate or question 48 on “choice” of benefit package.
Define contributions (regular monthly payments made to purchase coverage, paid regardless of whether services are used) and benefit package (set of benefits covered by a health plan along with the cost sharing charged for those services) in the opening script.
Response: Please see the response to comment 16. The survey testing results did not indicate any respondent confusion with the use of the term ‘contributions.’
(IN) Comment: Q50 - Q51. The explanation of benefits is inaccurate. HIP does employ an ER copay, and coverage does not terminate for a member below 100 percent of the FPL if they fail to make their contribution. In addition, the three-month timeframe is not germane to the HIP program and could be confusing to HIP members. It appears that this is a reference to another program. This question is also listed in the other surveys.
Response: We appreciate your comment. To provide more context, this survey section is intended to have respondents think about things they would like to see in any benefits package (as noted in the description at the top of the survey section) and does not ask about their experience in HIP 2.0. The questions included explore hypothetical benefits package options.
To reduce respondent burden and question ambiguity, this section has been revised. Please see the response to comment 16.
(NHeLP) Comment: We also strongly support the questions related to Indiana’s ER copay experiment, but once again recommend adding specific questions to measure whether providers are fulfilling their notice and referral requirements prior to charging copays.
Response: We appreciate this comment. Two survey questions have been added to assess whether providers are fulfilling their notice and referral requirements prior to charging copays. The additional questions include the following:
Were you told the reason for the copay was because your condition was not an emergency?
Yes
No
Not sure/Don’t know
Were you told about another available provider where you could get the care you needed without the emergency room copay?
Yes
No
Not sure/Don’t know
(NHeLP) Comment: Question 1 can easily be misread as a choice between “Healthy Indiana Plan” and “HIP 2.0.” This potential misreading could be easily corrected as follows:
“1. Are you currently enrolled in the “Healthy Indiana
Plan 2.0,” or (also called
“HIP 2.0”)?”
Response: Thank you, those revisions have been made.
(NHeLP) Comment: Question 3: We are concerned that respondents may not understand the phrase “benefits package.” We suggest adding a simple definition and a follow-up question prompting the individual to report whether they are in HIP Plus or HIP Basic (or state plan coverage?).
“3. HIP 2.0 offers different benefits packages. A benefit package is the set of services covered by a health plan along with the cost sharing charged for those services. Are you aware that HIP 2.0 offers:” (HIP Plus/HIP Basic)
Response: During the survey testing we assessed if participants understood the term “benefits package.” Testing participants reported a clear understanding. Therefore we are leaving the question as it is.
(IN) Comment: Q3 - Q4. The survey states that HIP offers different benefit packages and asks members if they are aware of Plus and Basic, but fails to describe the two packages, which may lead to confusion.
Response: A description of the different HIP 2.0 benefits packages was not included for two reasons. First, our survey is designed to test knowledge and understanding of HIP 2.0 before assessing HIP 2.0 experience. Second, providing a description of the HIP 2.0 benefits package could potentially lead respondents and would not provide an accurate assessment of their understanding of their own plan and of HIP 2.0 overall.
(IN) Comment:Q7. Stating “Sometimes people need” for the transportation questions is biased and should be worded in a more neutral manner, such as:
Think about the most recent healthcare visit you had scheduled in the last 6 months. Which of these phrases best describes your transportation for that visit?
I had transportation and went for that visit.
I did not go for that visit for some other reason.
I did not go for that visit mainly because I didn’t have transportation.
Don’t know/not sure
Response: We have revised the question to remove the introductory sentence that contained the comment that the state suggests introduces potential bias and references to taxi and mileage reimbursement. See the response to comment 27 below.
(IN) Comment: Q7. In addition to the bias noted earlier, this question does not accurately reflect the types of transportation services offered by providers in Indiana. None offer mileage or taxi reimbursement.
Response We have revised the question to eliminate the introductory statement (see also comment 25).
(IN) Comment: Q9. This question should distinguish between “scheduled” appointments and “non-scheduled” appointments.
Response: We are not making the distinction between scheduled and non-scheduled appointments in our survey analyses. The survey analyses will look at access to care as a whole and examine unmet health care needs. Additionally, our approach allows inclusion of individuals who choose not to schedule appointments due to potential barriers to care, such as transportation.
(IN) Comment: Q10. This question is confusing because it is mixing two concepts – 1) ability to pay for and 2) ability to get transportation within the same question.
Response: The two transportation concepts are asked about separately regarding transportation barriers with respect to specific types of health care services (see Enrollee Survey Questions 10 and 11).
For this particular question (Q10), we intend to capture overall level of worry associated with transportation be it the ability to pay for transportation or ability to get transportation. Our cognitive testing results indicated that the majority of respondents did not find this question confusing. Therefore we are leaving this question unchanged.
(IN) Comment: Q12. HIP covers transportation to the emergency room when there is a danger to life, so the option for the “emergency room care” should be omitted.
Response: We have revised the item to refer to non-emergency conditions only: “emergency room care for a non-emergency condition.”
(NHeLP) Comment: Questions 15-22 on Emergency Room Copays: We appreciate that CMS has improved some of the questions in this section, but we think structure of questions on ER copays could be strengthened. Namely, the survey should measure whether ER providers are fulfilling their screening, notice and referral requirements included in the ER copay protocol. These requirements, which have not been waived under Indiana’s ER copay protocol, are critical protections in Medicaid cost sharing law and are necessary preconditions an ER must fulfill before charging any copay for ER visits.3 The federal evaluation provides a great opportunity to collect data on whether these important notice and referral requirements have been consistently fulfilled. Also, our recommendations would help indicate if any of the ER copays are being incorrectly levied after the visit based on diagnostic data, which is not permitted under the statute.
Response: Please see the response to comment 21.
(IN) Comment: Emergency Room Section. The protocol was approved a few months ago by CMS and this specific policy is still early on in its’ implementation phase and may impact the data collected in this particular survey section.
Response: While we recognize that the protocol was approved a few months ago and the policy is still early on in its implementation, the purpose of including these questions at this time is to allow for comparisons of beneficiary experience and understanding between the first and second waves of the survey. The planned second wave of the survey is anticipated to launch in the summer of 2017, if approved under the Paperwork Reduction Act, and will provide descriptive information about the beneficiary understanding and experience after the policy has been implemented for at least one year.
(IN) Comment: Q17. This question is confusing as written and could be misinterpreted.
Response: Thank you for pointing this out. We have revised this question (see highlighted below) and the preceding questions as follows.
If you go to the emergency room when your condition is an emergency and you do not call the 24-hour nurse helpline, do you have to pay a copay?
Yes
No
Not sure/Don’t know
If you go to the emergency room when your condition is not an emergency and you do not call the 24-hour nurse helpline, do you have to pay a copay?
Yes
No
Not sure/Don’t know
If you go to the emergency room more than once a year when your condition is not an emergency and you never call the 24-hour nurse helpline, your copay will be:
Higher than $8
$8
Lower than $8
Not sure/ Don’t know
(IN) Comment: Q18. This question is unclear. The member could have thought about visiting the ER and decided he/she did not have an emergency. The purpose of the question is unclear.
Response: This question skips respondents to appropriate questions to identify whether they went to the emergency room in the event that they thought they needed emergency room care. For those who thought about visiting the ER but did not go, deciding they did not have an emergency could be captured in the offered response options “waited to see if I would get better on my own” or “some other reason” of the follow-up question for those who did not go to the ER.
(IN) Comment: Q19. The purpose of the question is unclear and should evaluate the factors that led to the decision to go to the emergency room.
Response: This question is meant to focus on those respondents who say there was a time they thought about going to the emergency room when they needed care, and determines whether or not they went. If they went to the emergency room respondents are then asked if they had to pay a copay or not. If respondents indicate that they did not go to the emergency room they are asked for the main reason why they did not to the emergency room for care (See also response to comment 33).
(NHeLP) Comment: We also note that all of the potential answers to Question 20 are covered in Question 21, making question 20 redundant (as currently phrased). Question 21 includes the additional possible response that a third party paid the copay. In fact, the current question structure would make it impossible to collect accurately data on when a third party ever paid the copay. In such a case, the respondent would correctly answer “No” to question 20 – “Did you pay the copay for the care you received?” – and then be prompted to go to the following section on POWER accounts without ever answering question 21 to indicate a third party paid it. For these reasons, we recommend rewording questions 20 and 21, and adding a follow-up question between 20 and 21, as follows:
“20. Did
you The last time you went to the emergency room, were
you asked to pay a copay?” (Yes, no –
GO TO POWER ACCOUNTS, not sure – GO
TO POWER ACCOUNTS)
Follow up: “20A. (If yes) Were you told the reason for the copay was because your condition was not an emergency? (Yes, No, Not sure)”
20B. (If yes) Were you told about another available provider where you could get the care you needed with lower or no copay?” (Yes, no, not sure)”
“21. How was that copay paid, if at all?...”
Response: Please see response to comment 21. Additionally, ‘the last time you went to the emergency room, were you asked’ was added to the original survey question 20 to help increase clarity.
(IN) Comment: Q20. The wording in this question is unclear, as a person might have been charged a copay but failed to make the payment.
Response: This question has been revised to determine whether beneficiaries were asked to pay a copay, not whether they actually paid a copay: “The last time you went to the emergency room, were you asked to pay a copay for the care you received in the emergency room?”
(NHeLP) Comment: Question 22: We continue to support this question on why individuals avoid the ER and we appreciate the revisions to this question from the last draft. We suggest adding a box to write in “other” reasons for not going.
Response: The federal beneficiary survey will not be including any open-ended answer options. We are not collecting open-ended/free responses for several reasons: (1) The mail survey (and by extension, the online survey) is already lengthy; including several open-ended questions may increase survey non-response. Many respondents likely would not complete a survey that required a lot of writing. (2) Because we will be surveying over 5,000 beneficiaries, there would be significant resource costs (e.g., training for verbatim response coding, and data entry costs) associated with using open-ended questions. In analysis, frequencies of each verbatim response would be generated, and would need to be collapsed into meaningful categories. “Other (specify)” responses would need to be checked to see whether they belonged in existing categories, and recoded if so. This process is subjective and prone to error/bias. (3) Open-ended questions tend to increase respondent burden. (4) Researchers have shown that open-ended questions tend to be less useful in mail surveys than in interviewer-administered surveys, as less-detailed information is provided by respondents than when probed and prompted by an interviewer (de Leeuw, Hox, and Dillman, 2008). This is especially true for low-income and low-education populations (Czaja and Blair, 2005). (5) In-person interviews are best for open-ended questions (Groves and Kahn, 1979).
(NHeLP) Comment: Questions 23-39 on POWER accounts and premium contributions: We support and appreciate the revisions CMS has made to these questions, particularly the additions of questions about the lockout for nonpayment and on the concerns about copay affordability. Data from the HIP 1.0 evaluations indicated that roughly one in four enrollees did not even know what a POWER account was.4 Such data has important implications for interpreting results on how the HIP healthy behavior incentives work (since they are predicated on awareness of the POWER account). We appreciate and support that this survey appears to ask everyone to answer the more detailed questions on how POWER account works.
Response: We appreciate your comment, thank you.
(IN) Comment: Q24. The question, as written, does not fully assess knowledge of the account. Asking whether they check the balance in their account is more appropriate than knowing the actual balance.
Response: The action of “checking your POWER account” was implied in knowing how much is in their POWER account in general. The survey is interested in the overall beneficiary POWER account experience.
(IN) Comment: Q25. This question is inaccurate because no one is required to make an annual contribution.
Response: We understand that beneficiaries are not required to contribute annually; this survey question has been updated to read as the following:
Do you currently contribute?
Yes, I currently contribute
Someone else contributes for me
I do not contribute
(IN) Comment: Q25 and Q26. Individuals are not required to make annual contributions, making the question inaccurate. In addition, response options for Q25 and Q26 overlap.
Response: Please see the response to comment 40.
(IN) Comment: Q27 and Q30. Members are asked whether their contributions or copays are affordable. Responses to this question as stated are leading and biased, and should be revised so that they do not include the word “afford” in the actual response.
Response: We appreciate your comment and have revised the answer options to increase clarity. The answer options provided cover the full range of potential responses.
Inclusion of the word “afford” supports the intent of these particular survey questions and did not pose a problem in testing. In addition, we reviewed the State’s 2010 survey of Healthy Indiana Plan (HIP) participants, and that survey consistently uses the word “afford” throughout in similar questions.
We have updated the neutral answer option to increase clarity. Answer options include the following (Note: The tense changes based on the instrument version):
More than I can/could afford
An amount that I can/could afford
Less than I can/could afford
Not sure/Don’t know
(IN) Comment: The answer options assessing the affordability of monthly or annual contributions and copays are leading and Indiana recommends using the following answer options: “too much” and “too little”
Response: Please see the response to comment 42. The answer options “too much” or “too little” may not accurately capture affordability within the beneficiary experience. For example, respondents could state that they felt their contributions or copays were too much, but this does not imply that they could not/cannot afford (were not able to pay) the amount.
(NHeLP) Comment: Questions 29-31: We strongly support the addition of these questions about why individuals do not pay monthly contributions with follow-up questions on the affordability of copayments.
Response: We appreciate your comment, thank you.
(IN) Comment: Q32 and Q33. The flow of Q32 and Q33 is confusing, and the responses will vary depending on the person’s income level, which the survey doesn’t account for.
Response: We have revised the question to be a test of knowledge in general and not specific to an individual’s income level. The intent of this question is to assess whether beneficiaries understand the consequences of non-payment according to which HIP 2.0 benefits package they have. In analysis we will examine and compare the awareness and understanding of the consequences of non-payment between HIP Basic and HIP Plus beneficiaries.
With respect to the survey flow, the questions first test knowledge of the non-payment consequences. If respondents correctly understood the policy, the survey proceeds with asking them another knowledge question related to the non-payment consequence (lockout). We did not think it would be necessary to ask respondents who indicated incorrect or no awareness of the lockout policy additional questions related to the lockout policy.
(NHeLP) Comment: Question 33: We strongly support the addition of this question to test beneficiaries’ awareness of the potential for a lockout.
Response: We appreciate your comment, thank you.
(IN) Comment: Q34. The question is inaccurate. The account balance is debited whether the person uses their debit card or not as the money still comes out of their account.
Response: The question is interested in examining ease of beneficiary understanding of how their POWER account works and not about procedures around how one’s balance is debited or not. The survey question has been revised for clarity and now reads as: “How easy or hard is it to understand how a POWER account works?”
(NHeLP) Comment: Question 35: This question tests an individuals’ knowledge of how the POWER account works. We appreciate the revisions that include a new question on whether POWER accounts help people understand the cost of health care services. However, sub-questions C and E (on whether the POWER account helps individuals pay for health care services or helps them feel comfortable paying) provide limited value and do not test anything unique to the POWER account structure. Most enrollees will likely agree (since their POWER account card is basically their insurance card, and does pay the costs). However, any health plan or state Medicaid program helps covered individuals pay for services they use, so the POWER account is no different. We recommend deleting statements C and E.
Response: The intention of answer options C and E (POWER accounts help people pay for the health care services they need and make people feel comfortable about paying for their health care services) is to assess the consumer driven health plan features built into the POWER accounts. Additionally, these answer options have been included for two reasons: First, the survey pretest indicated that some respondents did not understand how their POWER accounts worked. Second, data from the HIP 1.0 evaluation indicated that one in four enrollees did not know what a POWER account was. The answer option highlighting if their POWER accounts help them feel comfortable paying for their health care services examines the “consumer empowerment” characteristics of POWER accounts in general.
(NHeLP) Comment: Question 37: We support detailed questions on POWER account rollover and preventive care incentives, and support the revisions in this draft. We also reiterate that earlier evaluations of HIP 1.0 routinely excluded individuals who did not know they had a POWER account from the reported responses about how HIP healthy behavior incentives work.5 We support that this survey asks all respondents to answer all questions on POWER accounts and healthy behavior incentives. We recommend adding a follow up question to test respondents” understanding of what counts as preventive care for them, as follows:
Follow-up: “Do you know what care qualifies as preventive care for yourself?”
Response: While we recognize that beneficiary understanding and awareness of what health care services are considered as preventive care, we did not include this additional question since we provide a general definition. During the survey development we did explore testing beneficiary understanding of preventive care services, but given that preventive services are age- and gender-specific we did not want to impose more respondent burden by adding questions.
(NHeLP) Comment: Questions 40 - 45 on Access: We support the revisions to this section.
Response: We appreciate your comment, thank you.
(NHeLP) Comment: Question 46: Individuals who respond “somewhat satisfied” with HIP 2.0 or neutral should also have the option to offer suggestions on how to improve HIP or identify what makes them only “somewhat” satisfied.
Response: We appreciate your concern and have revised the satisfaction questions across all surveys to read as the following:
Q. Thinking about your overall experience with HIP 2.0, would you say you are:
Very Satisfied
Somewhat Satisfied
Neither Satisfied nor Dissatisfied GO TO NEXT QUESTION
Somewhat Dissatisfied
Very Dissatisfied
Not sure/ Don’t know GO TO NEXT QUESTION
Q. Please tell us how satisfied or dissatisfied you are with each HIP 2.0 item below.
Please mark one answer in each row. |
Very Satisfied |
Somewhat Satisfied |
Neutral |
Somewhat Dissatisfied |
Very Dissatisfied |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The satisfaction questions in the “HIP Basic enrolled, formerly HIP Plus enrolled” section will be revised similarly.
(IN) Comment: Q46 - Q47. Members are asked to think about their overall experience and are then asked about the specific reasons for being dissatisfied. However, there is no question asking about being satisfied or correlating reasons. The lack of balance points to dissatisfaction and is leading. The dissatisfaction question is found in all of three of the surveys, and there are no questions regarding reasons for satisfaction in any of the surveys.
In addition, only the survey of individuals that have left the program or were disenrolled asks whether the person would enroll in the program again. We recommend that the question be asked in all of the surveys.
Response: Please see the response to comment 51.
Additionally, the survey question asking whether individuals would enroll in the program again was included only in the disenrollee and lockout survey due to the time frame in the question. If that same survey question was to be included in the other surveys, the time frame would not be consistent with the time frame of reference in the ‘Satisfaction with HIP’ survey section.
(IN) Comment: Q54. The list of demographics avoids the “four-year” designation for undergraduate college degrees.
Response: The fifth answer option “4-year college graduate” refers to the four-year designation for undergraduate college degrees. Furthermore, this question was taken directly from the Nationwide Medicaid CAHPS survey. All education answer options for this survey question will maintain their original CAHPS format.
(IN) Comment: Q55. Additional response options are likely needed, such as “self-employment,” “unable to work,” and an open response.
Response: We have revised the question to expand the response options as follows:
What best describes your employment status?
Employed full-time
Employed part-time
Self-employed
A homemaker
A full-time student
Unable to work for health reasons
Unemployed
Please also see the response to comment 37 regarding open-ended responses.
(NHeLP) Comment: We support the idea of asking new enrollees separate questions specifically about the enrollment process. However, we would like to see this survey also include questions to test new enrollees’ understandings of the cost sharing structure and use of preventive services. New enrollees should have already been informed of these elements during enrollment, and this is an opportunity to document how effectively Indiana is educating new enrollees on the incentive structures. We do not specify these additional questions in our recommendations below, but they should follow the questions on these topics in the regular enrollee survey.
Response: We excluded these questions for a few reasons, namely budget constraints, respondent burden, and recall period. The primary focus of the new enrollee survey is to obtain beneficiary understanding, experience, and satisfaction with regard to the enrollment process. Increasing the length of the new enrollee survey would increase the burden on respondents. While we understand the importance of both the cost sharing structure and the use of preventive services, it is important that several of the survey questions have a short recall period of “the last 6 months.”
(NHeLP) Comment: Question 1: This question can easily be misread as a choice between “Healthy Indiana Plan” and “HIP 2.0.” This potential misreading could be easily corrected as follows:
“1. Are you currently enrolled in the “Healthy Indiana
Plan 2.0,” or (also
called “HIP 2.0”)?”
Response: Those revisions have been made.
(NHeLP) Comment: Question 4: We are concerned that respondents may not understand the phrase “benefits package.” We suggest adding a simple definition and a follow-up question prompting the individual to report whether they are in HIP Plus or HIP Basic (or state plan coverage?).
“3. HIP 2.0 offers different benefits packages. A benefit package is the set of benefits covered by a health plan along with the cost sharing charged for those benefits. Are you aware that HIP 2.0 offers:” (HIP Plus/HIP Basic).
Response: Please see the response to comment 23.
(IN) Comment: Q5. This question is evaluating written materials and internet resources but does not specify the source of the materials. An individual may receive materials from the State, managed care organizations (MCOs), providers, or other sources.
Response: This survey question was included to provide a broad overall understanding of the beneficiary experience and understanding of the HIP 2.0 enrollment experience. It is not intended to disentangle specifics regarding where the beneficiary received materials from, but rather whether the beneficiary looked for information anywhere regarding their benefits and if they found this information useful in general.
(IN) Comment: Q7. This question evaluates if an individual received information from a customer service representative without identifying the source. There are six separate call centers within the State’s purview (e.g., MCOs, enrollment brokers, eligibility, member services) that could potentially be providing information. In addition, other call centers outside of the State’s purview, such as hospital systems, other provider call centers, or the federal Exchange could also be providing information about HIP.
Response: Please see the response to comment 58. While we understand that individuals may receive information from various sources the federal evaluation is interested in the broad overall understanding of the beneficiary experience and understanding of the HIP 2.0 enrollment experience.
(IN) Comment: Q9 - Q10. It is unclear what the purpose of these questions is and what is being evaluated. HIP members likely fill out a variety of forms before, during, and after enrollment, including applications for enrollment, requests for additional information, health assessments, etc. If a provider assisted a new enrollee, the provider might have also had the individual fill out provider specific forms. Further, forms are completed electronically and by paper, and the survey does not consider this.
Response: We appreciate your comment. These survey questions have been omitted.
(IN) Comment: Q11. This question is misleading as application processing is dependent on many factors, including when the individual submitted all related verifications. In addition, the question could be interpreted in different ways – 1) as time elapsed from application submitted date to enrollment start date or 2) as the amount of time it took the member to get coverage after, for example, losing other coverage, which could include the time it took the member to learn about HIP and to complete an application. Finally, the responses should be quantifiable.
Response: This survey question has been revised to increase clarity:
From the time you submitted your application, how much time did it take for your HIP 2.0 coverage to start?
Less than a month
1 to 3 months
More than 3 months
Not sure/Don’t know
(NHeLP) Comment: Question 11: We support the revision that clarifies the starting point to judge how long an individual waits to get coverage.
Response: We appreciate your comment, thank you.
(NHeLP) Comment: Question 11: The meaning of “enrolled” may be difficult for some individuals to understand. We recommend using “started,” “joined,” or “began coverage with” as alternatives. Also, a respondent who knew they could never afford the monthly contribution might believe they never really had a choice between HIP Plus and HIP Basic. We recommend rewording this question as follows:
“When you
enrolled started in HIP 2.0, did
you have were you offered a choice between HIP
Basic and HIP Plus?
Response: This survey question was adapted from a CAHPS Supplemental Medicaid Enrollment question. Additionally, other beneficiary surveys have also used the term “enrolled” in their survey questions. Please also see the response to comment 11 regarding “choice”.
(IN) Comment: Q12 - Q16. As noted above, these questions may confuse members, because they imply that members can choose to enroll in Plus vs. Basic.
Response: Please see the response to comment 11.
(NHeLP) Comment: Question 12: We support the change that eliminated a vague reference to “fits within my budget.” We continue to recommend sliding this question back to come after question 15. Also, some of the listed factors could be clarified. For example, the statement on “Making a monthly or annual contribution” could suggest the individual did not want regular monthly payments, but it could also mean that the individual found the monthly contributions were too expensive (or, conversely, preferable to more expensive per service copays.)
“How important were the following factors in helping you choose between HIP Basic and HIP Plus?”
“a. The amount of the monthly contributions. (very important, somewhat important, not at all important)”
“a.b.
I would rather make a monthly or annual
contribution payments each month or
annually instead of copays. (very important, somewhat
important, not at all important)”
“b.c. I would rather pay a copay
at each visit instead of monthly payments.”
“c.d. I wanted dental and vision
benefits.” “d. Fits within my budget.”
“e. I wanted to get getting
coverage more quickly.”
Response: We appreciate your comment. Given the revisions in the Health Coverage Cost and Payment Option section and the exclusion of the mention of “choice” in HIP 2.0 during enrollment, this survey question has been omitted. Please see the response to comment 11 regarding “choice” and the response to comment 16 regarding the revisions in the Health Coverage Cost and Payment Option section.
(IN) Comment: Q15. The list of sources is not complete as an individual could have contacted one of the plans (i.e., MDwise, Anthem, or MHS), discussed this at an intake center, or consulted another unknown source.
Response: Thank you for bringing this to our attention. The MCE’s have been included as an answer option. The answer options now read as the following:
I got help from family or friends
I got help from my doctor or health care provider
I got help from a HIP toll free number and/or a HIP representative in-person or online
I got help from my health plan (i.e., Anthem, MDwise, MHS – Managed Health Services)
I got help from another source
I did not get any help
We combined the HIP toll free number and the HIP representatives into one answer option and added in the answer option “I got help from another source.”
(IN) Comment: Q17. This question is very vague. It is unclear what is meant by “something.” We do not believe this question provides useful information.
Response: We appreciate your comment. This survey question has been revised and now reads as the following:
Please tell us whether you agree, disagree, or are not sure about the following statement: You can do something to get coverage while your application is still being processed.
Agree
Disagree
Not sure/Don’t know
The question is trying to get at, in general, whether the beneficiary thinks anything can be done to get coverage while their application is still being processed.
(IN) Comment: Q18. The responses to this question are limited and inaccurate, as they do not reflect all the possible choices, such as returning requests for verifications or information
Response: Thank you for bringing this to our attention. This survey question has been revised and now reads as the following:
Q. Which of the following things could you do to get your HIP 2.0 coverage as soon as possible?
Please mark one answer in each row. |
Yes |
No |
Not sure |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
We have revised the question structure to allow respondents to respond to all possible answer options. Additionally, we have added additional answer options based on the revised survey question. Furthermore, this survey question is now followed by two new additional survey questions with identical answer options in different answer option grids. After respondents are asked about things they could do to get their HIP 2.0 coverage as soon as possible, they are then asked if they did any of those items to get their coverage, and then lastly are asked how easy or hard those particular actions were or could be to do during their HIP 2.0 enrollment process.
(IN) Comment: Q19. The inclusion of “how long” is too subjective and could be leading and convey that the length of time it took for an individual to get coverage was long.
Response: This survey question has been revised and reads as:
“How satisfied were you with how much time it took to get your HIP 2.0 coverage?”
(IN) Comment: Q19. This question is more closely related to Q11. The survey should clarify exactly what timeframe is being referenced. In terms of survey “flow,” it would be reasonable to ask these two questions together.
Response: We appreciate your comment. Given the survey revisions made to the satisfaction questions, the original survey question 19 has been omitted. The answer option ‘a’ in the satisfaction question looks at the length of time for coverage to begin. Please see the response to comment 52.
(NHeLP) Comment: Questions 17-19: We appreciate and support the revisions to these questions on “fast track” coverage.
Response: We appreciate your comment, thank you. Please see the response to comment 68 to see what additional survey revisions have been made to these particular survey questions.
(IN) Comment: Q20. Responses will vary depending on what plan the member has and the survey should ask members which plan they think they have (i.e., Basic, Plus, or “Don’t know”).
Response: Survey question 10 will help determine what HIP 2.0 benefits package the beneficiary has. If respondents select the answer option “I am not required to make contributions” then they are identifying themselves as HIP Basic beneficiaries.
(NHeLP) Comment: Question 20: We are concerned that respondents may not understand the phrase “benefits package.” We recommend including the definition cited above in Question 3.
Response: Please see the response to comment 23.
(NHeLP) Comment: Question 23: We support the addition of this question on how long the lockout would be for non-payment of contributions.
Response: We appreciate your comment, thank you.
(NHeLP) Comment: Question 24: Individuals who respond “somewhat satisfied” or neutral should also have the option to offer suggestions on how to improve/what makes them only “somewhat” satisfied.
Response: Please see the response to comment 51.
(IN) Comment: Q28 - Q29. The explanation of benefits is inaccurate—HIP does employ an ER copay. Coverage does not terminate for members below 100 percent of the FPL if they fail to make their contribution. Also, the three-month timeframe is not germane to the HIP program and could be confusing to HIP members. It appears that this is a reference to another program. These questions are also in other surveys.
Response: Please see the responses to comments 16 regarding the Health Coverage Cost and Payment Option survey section.
(IN) Comment: This survey is lengthy and very complex. We recommend that the federal evaluation considers creating different versions of this particular survey to increase clarity.
Response: To best collect data from individuals who would classify as “HIP Basic enrolled, formerly HIP Plus” we needed to include a separate survey section to capture their experience in HIP 2.0. If the “HIP Basic enrolled, formerly HIP Plus” individuals were included in the Enrollee Survey sample frame we would not accurately capture their beneficiary experience and understanding; we could not be sure if they were reporting on their separate experiences in HIP or on their HIP 2.0 experience as a whole. Separating them out into their own section helps clarify their separate experiences. Additionally, in order to best utilize CMS resources, three versions of this survey were designed to keep the respondent burden low while maximizing data quality.
(NHeLP) Comment: Question 1 can easily be misread as a choice between “Healthy Indiana Plan” and “HIP 2.0.” This potential misreading could be easily corrected as follows:
“Are you
currently enrolled in the “Healthy Indiana Plan 2.0” or
(also called “HIP 2.0”)?”
Response: Those revisions have been made.
(IN) Comment: Q4. As noted previously, this question may confuse members because it implies that members can choose to enroll in Plus vs. Basic.
Response: Please see the response to comment 11.
(IN) Comment: The intent of Question 10-12 is not clear. What comparisons will be made with these particular survey questions since the time frame of reference are for individuals no longer in HIP 2.0.
Response: These survey questions intend to look at access to care and transportation among disenrollees and HIP Plus lockouts. We do not intend to make any comparisons since the time frame asks about after beneficiaries were no longer in HIP 2.0. However, we will be looking at whether they have health insurance coverage after leaving HIP 2.0.
(NHeLP) Comment: Question 12 is a critical question in light of the flawed Lewin evaluation of NEMT and access to care that evaluated whether individuals have missed appointments due to lack of transportation. (For example, the Lewin analysis would not capture how the NEMT waiver impacts the subset of enrollees do not bother making an appointment because they know they cannot get transportation.) Asking about unmet need, rather than missed appointments, does capture this group. We would recommend adding a row to specifically inquire about access to specialists, who are often harder to access because they require longer trips.
Response: During the survey instrument testing we assessed what participants thought of when answering this question. Thinking about the care received from their “doctor,” a few participants included specialists in their notion of “doctor.” Also, this question was adapted from the Behavioral Risk Factor Surveillance System (BRFSS) survey; the specialists that BRFSS specifically includes in the question are dental.
(NHeLP) Comment: Questions 13-14: We support the revision to this question clarifying the other reasons for leaving HIP 2.0. This structure is clearer. The question focusing on paperwork is particularly important, as prior evidence from HIP evaluations showed that incomplete renewal forms was one of the leading causes for disenrollment.
It would be useful to add follow up questions to test individual’s awareness of the 6-month lockout and when they might become eligible again. Possible follow-ups for Question 13 could include:
If YES (to options A or B): “Did you know while you were in HIP 2.0 that you would not be able to reenroll for six months if:
You did not make payments on time? (Yes, No, Not sure)
You did not fill out your renewal forms on time? (Yes, No, Not sure)”
“Are you currently barred from enrolling in HIP 2.0? (Yes, No, Not sure)
“Do you know how long you have to wait, if at all, before you may reapply? (Yes, No, Not sure)”
Response: We appreciate your comment. The POWER account section includes a survey question assessing beneficiary understanding of the length of time associated with the lockout. We are primarily interested in beneficiary understanding with regard to the lockout due to a non-payment.
(IN) Comment: Q13. There are other reasons why a person may no longer be enrolled in HIP, such as qualifying for another Medicaid program or failure to comply with documentation requests. Response choices need to be expanded to reflect the range of disenrollment reasons. In addition, there is no requirement for annual contributions, so the question is inaccurate as stated.
Response: Please see the response to comment 12.
(IN) Comment: Q13 The inclusion of “another year” is awkward and makes the question lengthy. Additionally, the answer options may need to be expanded since they may have been things occurring at different points in time and different processes could have led to a paperwork/redetermination issue.
Response: The answer option for this survey question has been revised and the inclusion of “another year” has been omitted.
The descriptive analyses is intended to focus on beneficiary experience and understanding and this particular question is intended to identify those that were voluntarily disenrolled and those that were locked out of HIP 2.0.
(IN) Comment: Q14. It would appear that the respondents who choose “other reason” at Q13 are the only ones who answer this question, and it is not clear whether the skip pattern will distinguish those who left voluntarily from those who were disenrolled involuntarily. In addition, the inclusion of the transportation option within this question is unclear and confusing.
Response: Thank you for bringing this to our attention. Revisions to both of these questions have been made to increase clarity. These survey questions were integrated into one survey question:
Why did you leave HIP 2.0? Please mark one box in each row.
I left HIP 2.0 because …. |
Yes |
No |
Not sure |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
(NHeLP) Comment: Questions 16-33 on POWER accounts and premium contributions: We support and appreciate the revisions CMS has made to these questions, particularly the additions of questions about the lockout for nonpayment and on the concerns about copay affordability. Data from the HIP 1.0 evaluations indicated that roughly one in four enrollees did not even know what a POWER account was.6 Such data have important implications for interpreting results on how the HIP healthy behavior incentives work (since they are predicated on awareness of the POWER account). We appreciate and support that this survey appears to ask everyone to answer the more detailed questions on how POWER account works.
Response: We appreciate your comments, thank you.
(IN) Comment: Q21. Response options are limited, unbalanced, and could create biased responses. Response options should include a five-point Likert scale.
Response: We have revised the answer options to a five-point Likert Scale. The answer options include the following:
Not at all worried
A little worried
Somewhat worried
Very worried
Extremely worried
(IN) Comment: Q22. The data collected will be variable and depend on when individuals left HIP 2.0.
Response: We understand that this question will be variable and will take that into consideration in the analysis as we expect to be able to determine from enrollment data when beneficiaries were no longer enrolled in HIP 2.0.
(NHeLP) Comment: Questions 23-25: We strongly support the addition of these questions about why individuals do not pay monthly contributions with follow-up questions on the affordability of copayments. Note that the tense should be changed from “do” to “did” in Question 23.
Response: The verb tenses have been updated appropriately.
(NHeLP) Comment: Question 27: We support the addition of this question to test an enrollee’s awareness of the length of the lockout. Evidence that enrollees are actually aware of the lockout threat is a necessary precondition to evaluate the state’s claim that the lockout will reduce nonpayment of premiums.
Response: We appreciate your comment, thank you.
(NHeLP) Comment: Question 29: This question tests an individuals’ knowledge of how the POWER account works. We appreciate the revisions that include a new question on whether POWER accounts help people understand the cost of health care services. However, sub-questions C and E (on whether the POWER account helps individuals pay for health care services or helps them feel comfortable paying) provide limited value and do not test anything unique to the POWER account structure. Most enrollees will likely agree (since their POWER account card is basically their insurance card, and does pay the costs). However, any health plan or state Medicaid program helps covered individuals pay for services they use, so the POWER account is no different. We recommend deleting statements C and E.
Response: Please see the response to comment 48.
(IN) Comment: Q31. This question assumes the member received preventive services and is misleading.
Response: The intention of this question is to test beneficiary understanding of how the POWER account works with respect to preventive services. We have removed the reference to beneficiary “experience” with preventive services, thereby making it clear that we are assessing the beneficiary’s understanding of POWER accounts and preventive services in general, and not assuming that the member received preventive services.
(NHeLP) Comment: Questions 31-33: We support detailed questions on POWER account rollover and preventive care incentives, and support the revisions in this draft. We also reiterate that earlier evaluations of HIP 1.0 routinely excluded individuals who did not know they had a POWER account from the reported responses about how HIP healthy behavior incentives work.7 We support that this survey asks all respondents to answer all questions on POWER accounts and healthy behavior incentives. We recommend adding a follow up question to test respondents’ understanding of what counts as preventive care for them, as follows:
Follow-up: “Do you know what care qualifies as preventive care for yourself?
Response: Please see the response to comment 49.
(NHeLP) Comment: Question 35: We support the inclusion of a definition for copays in this question, but the question itself may not be appropriate for HIP disenrollees. An uninsured respondent who went to a doctor after being disenrolled (and had to self-pay) would not be asked for a copay, but would be asked to pay the whole bill. The wording in question 29 should address this possibility, such as adding a response option:
“C. No, I was asked to pay the whole bill.”
Response: The answer options have been revised and include “No, I was asked to pay the whole bill”
(NHeLP) Comment: Questions 38, 50, and 52: Individuals who respond “somewhat satisfied” or neutral should also have the option to offer suggestions to improve HIP and identify what makes them only “somewhat” satisfied or neutral.
Response: Please see the response to comment 51.
(IN) Comment: Q40. This question assumes that respondents knew that they were previously enrolled in HIP Plus; however, it is likely that some did not know the difference. Also, if the person does not know whether they are in HIP Basic, they should also skip to the end of the survey, since they would be unable to answer the next set of questions (Q41 through Q53).
Response: The survey focuses on beneficiary understanding and experience in HIP 2.0. Beneficiary awareness and understanding of the differences between HIP Basic and HIP Plus is one of the things we are interested in examining. Respondents selecting “Not sure/Don’t know” in this survey question will skip to the end of the survey.
(IN) Comment: Q41. Terminology (“better, same, worse”) may be unclear. We suggest that the question be reworded to tie to coverage.
Response: This question has been revised to increase clarity and reads as the following:
Thinking about your current HIP Basic coverage, how does it compare to HIP Plus? Is it better, about the same, or worse?
Better than HIP Plus coverage
About the same as HIP Plus coverage
Worse than HIP Plus coverage
Respondents will be skipped to the appropriate follow up questions in this survey question.
(IN) Comment: Q50 - Q53. These questions assume that respondents knew they were in Plus or Basic; however, it is likely that some did not know the difference. It is likely that some respondents will not know they were moved from Plus to Basic.
Response: Beneficiary awareness and understanding of the differences between HIP Basic and HIP Plus are things we are interested in examining. Furthermore, from the sample frame and enrollment information we will be able to determine which individuals have shifted to HIP Basic from HIP Plus.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Mendoza, Kristin |
| File Modified | 0000-00-00 |
| File Created | 2021-01-23 |