MIPCD_Supp_State_Part_A_05_08_2013_clean

Under section 4108 of the Patient Protection and Affordable Care Act, the Centers for Medicare & Medicaid Services (CMS) is required to contract with an independent entity or organization to conduct an evaluation of the Medicaid Incentives for Prevention of Chronic Disease (MIPCD) demonstration. RTI International was awarded a contract with CMS to conduct this evaluation. As part of this evaluation, RTI, under contract with CMS, will conduct State site visits, two rounds of focus group discussions, interviews with key program stakeholders, and field a beneficiary satisfaction survey. Both the State site visits and interviews with key program stakeholders will entail one-on-one interviews; however each set will have a unique data collection form. Thus, each evaluation task listed above has a separate data collection form and this proposed information collection encompasses four data collection forms.

The first set of one-on-one interviews will be conducted during the site visits to each of the 10 State MIPCD programs. For these site visits, staff will work with each State to identify and schedule one-on-one interviews with relevant stakeholders that fall into the following categories: managers, educators, recruiters, clinical staffs, and evaluators. In some States, clinical staff members serve as both participant educators by providing appropriate health prevention information and education and program recruiters by assessing patients to determine their eligibility for the State’s initiative. As a result, a protocol has been created specific to clinical staff’s unique role and potentially broad interactions with participants. Interview questions will focus on program implementation progress, including progress recruiting participants and providing participant incentives, special populations, individual State program evaluation progress, and State data collection and tracking. Feedback gleaned from these interviews will be used as background for the MIPCD second Report to Congress and assist in coordinating the logistics for the focus group discussions and Beneficiary Satisfaction Survey—research tasks that will take place after the site visits are completed.

Beneficiary satisfaction with their MIPCD program will be assessed using a combination of qualitative and quantitative assessments. First a qualitative assessment will be conducted using a series of beneficiary focus groups in all 10 States implementing the MIPCD. A total of 30 focus groups will be completed in Round 1. Up to 4 of the 30 groups may be conducted in Spanish with the remaining groups conducted in English. Depending on the nature and scope of the programs offered in each State, the focus groups will focus on one or more program components (i.e., incentive type, chronic disease). For example, if a State has implemented multiple program components or has multiple program sites, we will focus our evaluation in selected areas. We will determine where to conduct the focus groups in consultation with CMS and State program leadership, with a goal of including a diversity of program types and beneficiaries. We will consider factors such as

the health condition(s) targeted (e.g., smoking cessation, weight loss [diabetes prevention], diabetes management);
statewide versus smaller or single site programs; and
programs offering different types and amounts of incentives (e.g., cash or debit card, gift card, or other type of incentive).

A second set of stakeholder interviews will be conducted with no more than 5 individuals in each State for a total of 45 interviews. These interviews will take place when focus group discussions are conducted in each State. The exact number of interviews per State may vary somewhat depending on the scope and nature of the State program. These interviews will target stakeholders that spend 50 percent or more of their time on the program, directly interacting with beneficiaries and have been in their role for 1 year or longer. These interviews will focus on understanding the beneficiary experience and how each program addresses quality of care, accessibility and beneficiary satisfaction. Potential interviewees include program directors, clinicians, educators, and others who provide services and interact directly with beneficiaries on a regular basis. State program leadership will help identify stakeholders who can be most informative about the beneficiary experience and satisfaction. Stakeholders contacted to participate in this second set of interviews may overlap with those interviewed during the site visit; however, the questions will differ greatly from those asked during the site visit.

For the Beneficiary Satisfaction Survey, there will be a cross-sectional survey conducted in English or Spanish with Medicaid beneficiaries who participated in State programs for at least 6 months prior to when the survey is fielded in Year 3 of the evaluation. For States that have an experimental and control arm, we will sample beneficiaries from the experimental arm who are receiving incentives. The purpose of the survey is to assess beneficiaries’ satisfaction with the program—specifically satisfaction with accessibility of program activities, quality of services, and with the incentives. The survey will provide the beneficiary perspective on these aspects of the program. The survey will be administered by mail with telephone follow-up (see Attachments 6.e. & 9.e. for survey telephone follow-up script in English and Spanish).

B. Justification

B.1 Need and Legal Basis

CMS seeks Office of Management and Budget (OMB) approval to conduct an independent assessment of the MIPCD demonstration initiative. Information will be collected from demonstration State staff, partner organizations, providers and contractors, and program participants over a 3-year period. Authorization for CMS to conduct this study is provided under Section 4108(d) (1) of the Patient Protection and Affordable Care Act (Public Law 111-148) (see Attachment 1). Specifically, CMS is required to procure an independent entity to conduct an evaluation and assessment of the MIPCD programs carried out by States. The purpose of the evaluation and assessment includes determining the following:

The effect of such initiatives on the use of health care services by Medicaid beneficiaries participating in the program;
The extent to which special populations (including adults with disabilities, adults with chronic illnesses, and children with special health care needs) are able to participate in the program;
The level of satisfaction of Medicaid beneficiaries with respect to the accessibility and quality of health care services provided through the program; and
The administrative costs incurred by State agencies that are responsible for administration of the program.

This new information collection addresses CMS’ need to evaluate the effectiveness of the MIPCD program. The independent evaluation includes four new data collection instruments—site visit protocols, stakeholder interview protocols, focus group discussion guides, and a beneficiary satisfaction survey.

A related, but separate data collection for MIPCD implementation monitoring was earlier approved by the Office of Management and Budget (OMB). Section 4108(d)(2) of the ACA requires MIPCD States to submit reports to CMS on the specific uses of the grant funds, beneficiary participation and outcomes, program implementation process and lessons learned, and preventive services as part of reporting on quality measures for Medicaid managed care programs. To standardize these State reports, CMS with assistance from its Implementation Contractor, Econometrica, developed the Minimum Data Set (MDS). The MDS is a secondary data collection that assembles information already collected by grantees in the course of tracking beneficiary participation and outcomes and performing their own evaluation activities. On August 10, 2012, CMS published a 60-day notice (77 FR 47851) followed by a 30-day notice on October 19, 2012 (77 FR 64343) for the MDS.OMB approved this collection under Control Number 0938-1184 on December 22, 2012. This earlier PRA package is associated with the MIPCD demonstration, but is tied specifically to grant implementation monitoring.

Both data collections are authorized under Section 4108 of Affordable Care Act. However, these two data collection requests are associated with different provisions in Section 4108 of the ACA. The four new data collection forms in this package—the site visit protocols, the stakeholder interviews, the focus group discussion guides, and the beneficiary satisfaction survey—are associated with the national MIPCD evaluation required under Section 4108(d)(1) of the ACA. The earlier PRA submission for the MDS (OMB Control Number 0938-1184), a secondary data collection instrument designed to monitor program implementation, is associated with Section 4108(d) (2) of the ACA. Given the different purposes of the national evaluation and the implementation monitoring done through the MDS, we believe a separate PRA submission for the evaluation’s new data collection instruments is appropriate.

B.2 Information Users

This is a new collection. CMS will use information collected to produce two reports to congress that respond to all four components of Section 4108(d) (1) of the Patient Protection and Affordable Care Act, the legislation underlying this evaluation. These reports will inform recommendations for legislation and administrative action to expand or extend these initiatives beyond their end date of January 1, 2016, made by the Secretary of Health and Human Services.

The site visits will provide a deeper understanding of the programs and inform the impact evaluation of the MIPCD initiatives’ on Medicaid beneficiaries’ use of health care services and their health outcomes, one component of the legislation. This impact evaluation will also draw on Medicaid claims and encounter data and beneficiary utilization records. For Medicaid claims and encounter data, States will share beneficiary records with the evaluators; no data collection instrument is needed for this portion of the evaluation. The evaluation team will obtain and use already-collected, beneficiary-level data to complement the Medicaid claims and encounter data for its impact analyses. The beneficiary utilization records are collected quarterly by the Implementation Contractor through the MIPCD State Minimum Data Set (MDS). This separate data collection instrument administered by Implementation Contractor has already obtained OMB approval (OMB Control Number 0938-1184). Both Medicaid claims and encounter data and beneficiary utilization records will also be used to examine the extent to which special populations (including adults with disabilities, adults with chronic illnesses, and children with special health care needs) are able to participate in the program, a second component of the legislation.

The focus group discussions, stakeholder interviews and Beneficiary Satisfaction Survey will address third component of the legislation--the level of satisfaction of Medicaid beneficiaries with respect to the accessibility and quality of health care services provided through the program.

Additionally, the Beneficiary Satisfaction Survey instrument will be useful to other evaluators and researchers who are interested in incentive-based health programs. The information to be collected through the focus groups will allow CMS to assess the level of satisfaction different segments of the beneficiary population have with the MIPCD programs. If this data collection is not performed, CMS will not know which programs and incentive levels offer the highest level of beneficiary satisfaction for those with different chronic health conditions. CMS will use this information to better understand how beneficiary groups respond to incentive programs related to tobacco cessation, diabetes, and weight loss. Ultimately, this information will be used to develop Medicaid beneficiary programs that are of higher quality and more accessible to their constituents. The end goal is to engage beneficiaries in primary prevention health programs to reduce their health and financial burden of chronic disease conditions through early diagnosis, proper care, and improved health habits.

Finally, the fourth component of the evaluation mandated by the legislation—to collect data on the administrative costs incurred by State agencies that are responsible for administration of the program—will be addressed by collecting administrative cost data from States using a new data collection instrument that is currently being developed. Once the instrument is complete, it will be submitted for OMB review and approval.

B.3 Use of Information Technology

The data will be collected through in-person and telephone interviews, in-person focus group discussions, and Beneficiary Satisfaction Survey by mail with telephone follow-up of non-respondents. We do not plan to use automated, electronic, mechanical, or other technological collection techniques. The use of electronics in data collection is limited to electronic audio recordings of the interviews and focus group discussions. These audio-recordings will be used to capture all information and assist with the preparation of reports. Beneficiaries and interviewees will provide informed consent before they are audio recorded.

B.4 Duplication of Efforts

This information collection does not duplicate any other effort and the information cannot be obtained from any other source. A review of the literature reveals that there are no existing data collection efforts, no comparable studies, and no available data that provide a cross-state assessment of this CMS demonstration. Moreover, this independent assessment is a federal mandate designed to inform Congress about the use of incentives to improve health outcome and risk for Medicaid beneficiaries participating in State-level prevention programs.

Although some demonstration States may conduct focus group discussions or collect information through beneficiary surveys, none of these data collection efforts are consistently implemented across all States. To avoid any duplication at the State level, we have reviewed all beneficiary satisfaction surveys developed by demonstration States (Hawaii, Montana, and Texas) to assess potential duplication. We will also share the Beneficiary Satisfaction Survey and the focus group discussion guides we develop with the States so they can avoid duplication in any data collection instruments they develop and expand on what is being collected for the overall evaluation.

B.5 Small Businesses

The site visits may involve interviews with physicians who may represent small business or other small entities. Potential physician participants will be contacted via e-mail to schedule an interview. The e-mail will provide background information on the purpose of our study and instruct physicians interested in participating to respond to the e-mail and schedule an interview date/time (see Attachments 2.a. & 2.b. for the email and background information). Physicians who are not interested in participating will not be required to respond in any way and may simply discard the e-mail. Also, prior to each interview, interviewers will read the consent form and obtain verbal informed consent from physicians (Attachment 2.c.). Physician participation in this voluntary study does not involve travel, record-keeping, or preparation for the site visit interviews, and is not expected to have an impact on small business.

B.6 Less Frequent Collection

If the information collection is not conducted, CMS will not have the information necessary to carry out the monitoring and evaluation requirements of the program as stated in Section 4108 of the ACA.

Also, because the site visit protocol, the stakeholder interview protocol, and the Beneficiary Satisfaction Survey are data collection methods that will be implemented only once, it is not possible to reduce the frequency of data collection.

The focus groups will be conducted twice in a select group of States. The second round of focus group discussions are designed to highlight the impact of certain programmatic changes on beneficiary satisfaction. Again, a sub-set of States will be selected as the location for these discussions and we will not include round 1 focus group discussion participants in any round 2 focus group discussions. Thus, we will be collecting information from participants one time only and eliminating this data collection method would make it challenging to address pertinent research questions for the independent evaluation.

B.7 Special Circumstances

There are no special circumstances.

B.8 Federal Register/Outside Consultation

The 60-day Federal Register notice was published on May 17, 2013; the 30-day Federal Register notice was published on July 26, 2013.

No outside consultants contributed to the formation of the study design.

B.9 Payments/Gifts to Respondents

We will give participants in the focus groups a monetary incentive to thank them for their time and participation in the focus group effort. Because many of these participants will have children, the incentive also serves to offset childcare costs related to participating in the study. The incentive amount is $75 (provided as cash) for a 90-minute focus group.

This proposed payment of $75 per participant is intended to recognize the time burden placed on the participants, encourage their cooperation, convey appreciation for contributing to this important activity, and cover any transportation expenses incurred by participants. Numerous empirical studies have shown that incentives can significantly increase research response rates (e.g., Abreu & Winters, 1999; Shettle & Mooney, 1999; Greenbaum, 2000). Professional recruiting firms recommend a minimum payment of $75 per participant. In their experience, they find a drop off in respondent commitment with any lower amount. In response to offering this level of incentive, respondents are much more likely to honor their commitment of participating in the focus group or interview. Lower incentives could actually result in higher recruiting costs because of the need to over-recruit by higher percentages (Krueger & Casey, 2009).

B.10 Confidentiality

Confidentiality of patient-specific data will be maintained as provided by the Privacy Act of 1974 (5 U.S.C. 552a). All electronic project files at RTI will be stored on a limited-access project shared drive on RTI’s secure network servers; only project staff that has been authorized by the project director can access the shared drive. After project completion, all electronic files (e.g., notes, documents, reports) will be archived on RTI’s project shared drive. Also, one year after the evaluation has ended; RTI will destroy all personally identifiable information collected during the evaluation. All RTI employees and contractors working on the project who have access to project data are required to sign a code of conduct that outlines how project staff should conduct research with human subjects which includes ensuring privacy and confidentiality (Attachment 3). The proposed data collection methods are voluntary, and no persons are required to respond to the interviews. In addition, respondents may decline to answer any question. Finally, all data collection instruments, recruitment materials, and consent forms have been reviewed and approved by RTI’s Institutional Review Board (IRB). RTI’s IRB approval of the evaluation’s consent forms indicates that they conform to all informed consent requirements. RTI's IRB is required under the Code of Federal Regulations Title 45, Public Welfare, Part 46, "Protection of Human Subjects," to review biomedical and behavioral research conducted by RTI under contract from the Department of Health and Human Services to protect the rights of human subjects of research. The Office for Human Research Protections (OHRP) has granted a Federal wide Assurance (FWA #3331 effective until June 17, 2014) to RTI that allows it to review and approve studies independently.

These data collection activities are covered under a Centers for Medicare & Medicaid Services System of Records: “Master Demonstration, Evaluation, and Research Studies for the Office of Research, Development and Information” (System No. 09-70-0591). The System of Records Notice was published in the Federal Register on April 19, 2007 (Volume 72, page 19705).

B.10.1 Site Visit and Stakeholder Interviews

During the informed consent process, individuals participating in either set of interviews—site visit (Attachments 5.c. to 5.g.) or stakeholder (Attachments 7.a)--will be assured verbally that their participation in this evaluation will be confidential. All interview data, whether recorded or written, will be saved in a de-identified format. Only evaluation staff members will have the cross-walk to link written or audio files back to individual interviews and this cross-walk will be maintained in a separate and secure location. We will aim to conduct one-on-one interviews to further ensure the confidentiality of our interviewees. In situations where multiple interviewees want to participate in one interview, we will reinforce the importance of confidentiality among interview participants. Finally, we will not identify participants by name in any reports and will provide only summary-level feedback in most instances. For situations where we feel including an individual’s specific feedback is important, we will first obtain permission from a respondent before we include his or her specific feedback and in all cases, we will not attribute quotes directly.

B.10.2 Focus Group Discussions

Both English and Spanish-speaking participants will be responsible for contacting staff via a toll-free number to be screened for focus group eligibility (see Attachments 4.a. & 8.a. for Focus Group Screener in English and Spanish). This toll-free number will be answered with a recorded greeting that requires callers to select their language of preference (e.g. “Thank you for calling the MIPCD focus group screening line. For English, please press 1. Para Español, por favor presione 2.”). Once participants select a language, they will be redirected to the focus group recruitment staff that will conduct their screening in their preferred language, as indicated initially on the call. If focus group recruitment staff are unavailable to answer a participant’s call, the participant will be connected to a recorded message in both English and Spanish that will instruct the participant to leave his/her contact information and a focus group recruiter will call the participant back as soon as possible.

Information gathered during these screening calls will be used to assist the focus group facilitator during the discussions. However, all information gathered will be de-identified for analysis and reported in summary form. In some cases, focus group discussion recruitment strategies may include obtaining participant lists from States and contacting some participants to join a focus group discussion. In these instances, State-provided contact information will be maintained in a separate location from any research findings or notes collected from the focus group discussions.

All focus group participants will be assured of their confidentiality via written informed consent that each individual will complete prior to participating in the focus group discussion (see Attachments 4.e. & 8.e. for Focus Group Consent Forms in English and Spanish). Beneficiaries will be assured that they will incur no penalties if they wish not to respond to the information collection as a whole or to any specific questions. These procedures conform to ethical practices for collecting data from human participants. Focus group notes and transcripts will be stripped of identifying information and saved on secure network servers.

B.10.3 Beneficiary Satisfaction Survey

For the Beneficiary Satisfaction Survey, we will inform beneficiaries that their responses on the survey are confidential and specifically (1) will not be shared with program staff, and (2) names will not be included in reports of survey findings. We will provide information in both English and Spanish about confidentiality in the pre-notification letter (Attachments 6.a. in English & 10.a. in Spanish), the letter to accompany the survey (Attachments 6.b. in English & 9.b. in Spanish), and by phone for those who complete the survey by phone (Attachments 6.e. in English & 9.e. in Spanish).

B.11 Sensitive Questions

The majority of questions asked in the focus groups will not be of a sensitive nature. There will be no requests for a beneficiary’s Social Security number. A few questions on the screener (e.g., age, sex, race, ethnicity, education, see Attachments 4.a. in English & 8.a. in Spanish) are potentially sensitive to a small portion of beneficiaries, but are not considered highly sensitive. These questions are necessary to recruit beneficiaries who represent a variety of demographic groups and to collect information integral to the purpose of this study. Steps to avoid negative reactions from beneficiaries will be taken, including informing them both verbally and on the written consent that they can refrain from answering any question that makes them feel uncomfortable or that they simply do not wish to answer.

The Beneficiary Satisfaction Survey includes questions about the study participants’ sociodemographic characteristics and self-reported health status, including assessment of their overall health and overall mental or emotional health (Attachments 6.c: Beneficiary Satisfaction Survey in English, Section J—“About you” & 9.c.: Beneficiary Satisfaction Survey in Spanish, Sección J—“Acerca de Uste” ). These questions could potentially be perceived as sensitive by some study participants. Study participants are informed that their responses are confidential (also see section B.10 on Confidentiality) and that they can skip any questions they do not want to answer. The purpose of these questions is to understand study participants’ background characteristics and allow us to examine whether and how satisfaction with the program varies by these characteristics.

B.12 Burden Estimates (Hours and Wages)

Tables 1 & 2 provide the burden estimates in hours for all four information collection forms for the different respondent types. The annualized wages presented in Table 2 are based on data from the United States Department of Labor, Bureau of Labor Statistics (May, 2011) for State, local, and private industry earning and assumes an average hourly wage rate for respondents who work an estimated 40-hour work week. There are no direct costs to respondents associated with this information collection.

In the sub-sections that follow, we explain these burden estimates for each information collection form and provide an accompanying table for each form. Tables 3 through 7 are simply sections of Table 1 and 2 repeated for illustrative purposes. Figures in the tables are rounded to the nearest dollar for the total calculation.

B.12.1 Site Visit Interviews

We will aim to interview no more than 20 individuals in each State. We estimate that all 200 interviews will be conducted during one fiscal year. We plan to interview five stakeholder types—(1) managers, (2) recruiters, (3) educators, (4) clinic staff, and (5) evaluators—and have developed protocols tailored to their participation and role in the State’s program (Attachments 5.c. to 5.g.). The questions vary across the stakeholder types with a burden of 1.50 hours for managers, 1.00 hours for educators and clinic staff and 0.75 hours for recruiters and evaluators.

Average hourly wage rate for managers is $47.25¹; recruiters who are community and social service specialists—$20.30²; health educators—$25.66³; evaluators—$38.71⁴; and clinic staff —$44.02⁵. The total estimated annualized cost to respondents is $7,392, as summarized in Table 3.

Table 1. Estimated Response Burden Table (Hours)

Type of Respondent	Form Name	Number of Respondents	Number of Responses per Respondent	Average Burden per Response (in hours)	Total Burden (in hours)
State program participants	Beneficiary Focus Group Round 1 (Total) Screener Moderator’s Guide	660 660 270	1 1 1	0.08 1.50	460 55 405
State program participants	Beneficiary Focus Group Round 2 (Total) Screener Moderator’s Guide	220 220 90	1 1 1	0.08 1.50	153 18 135
State program Participants	Beneficiary Satisfaction Survey (Total) Mail-version Telephone completion	3,561 2,254 1,307	1 1 1	0.25 0.33	1,000 564 436
State program management staff	Site Visit Management interview protocol	40	1	1.50	60
State program recruitment staff	Site Visit Recruiter interview protocol	40	1	0.75	30
State program recruitment staff	Beneficiary Satisfaction Interviews guide	25	1	0.50	13
State program education staff	Site Visit Educator interview protocol	40	1	1.00	40
State program education staff	Beneficiary Satisfaction Interviews guide	20	1	0.50	10
State program evaluation staff	Site Visit Evaluator interview protocol	40	1	0.75	30
State program clinical staff	Site Visit Clinic staff interview protocol	40	1	1.00	40
Total	—	4,686	—	—	1,836

Table 2. Estimated Response Burden Table (Annualized Wages)

Type of Respondent	Form Name	Number of Respondents	Number of Responses per Respondent	Average Burden per Response (in hours)	Average Hourly Wage Rate	Total Cost
State program participants	Beneficiary Focus Group Round 1 (Total) Screener Moderator’s Guide	660 660 270	1 1 1	0.08 1.50	$22.22 $22.22 $22.22	$10,221 $1,222 $8,999
State program participants	Beneficiary Focus Group Round 2 (Total) Screener Moderator’s Guide	220 220 90	1 1 1	0.08 1.50	$22.22 $22.22 $22.22	$3,407 $407 $3,000
State program participants	Beneficiary Satisfaction Survey (Total) Mail-version Telephone completion	3,561 2,254 1,307	1 1 1	0.25 0.33	$22.22 $22.22 $22.22	$22,202 $12,521 $9,681
State program management staff	Site Visit Management interview protocol	40	1	1.50	$47.25	$2,835
State program recruitment staff	Site Visit Recruiter interview protocol	40	1	0.75	$20.30	$609
State program recruitment staff	Beneficiary Satisfaction Interviews guide	25	1	0.50	$20.30	$254
State program education staff	Site Visit Educator interview protocol	40	1	1.00	$25.66	$1,026
State program education staff	Beneficiary Satisfaction Interviews guide	20	1	0.50	$25.66	$257
State program evaluation staff	Site Visit Evaluator interview protocol	40	1	0.75	$38.71	$1,161
State program clinical staff	Site Visit Clinic staff interview protocol	40	1	1.00	$44.02	$1,761
Total	—	4,686	—	—	—	$43,733

Table 3. Estimated Site Visit Interview Response Burden Table (Hours and Wages)

Type of Respondent	Form Name	Number of Respondents	Number of Responses per Respondent	Average Burden per Response (in hours)	Total Burden (in hours)	Average Hourly Wage Rate	Total Cost
State program management staff	Site Visit Management interview protocol	40	1	1.50	60	$47.25	$2,835
State program recruitment staff	Site Visit Recruiter interview protocol	40	1	0.75	30	$20.30	$609
State program education staff	Site Visit Educator interview protocol	40	1	1.00	40	$25.66	$1,026
State program evaluation staff	Site Visit Evaluator interview protocol	40	1	0.75	30	$38.71	$1,161
State program clinical staff	Site Visit Clinic staff interview protocol	40	1	1.00	40	$44.02	$1,761
Total	—	200	—	—	200	—	$7,392

B.12.2 Stakeholder Interviews

Each interview will last 30 minutes. We estimate that all 45 interviews will be conducted over an approximately nine month timeframe. A total of 45 stakeholders will be recruited. Each interview will be conducted one on one to ensure confidentiality and to reduce the time burden on respondents (see Attachment 7.a.). The average burden of the interview is estimated at 30 minutes.

We anticipate interviewing two types of respondents—health educators and recruiters. The average hourly wage rate for health educators is $25.66⁶; for recruiters who are community and social service specialists, it is $20.30⁷. The total estimated annualized cost to respondents is $511, as summarized in Table 4.

Table 4. Estimated Stakeholder Interview Response Burden Table (Hours and Wages)

Type of Respondent	Form Name	Number of Respondents	Number of Responses per Respondent	Average Burden per Response (in hours)	Total Burden (in hours)	Average Hourly Wage Rate	Total Cost
State program recruitment staff	Beneficiary Satisfaction Interviews guide	25	1	0.50	12.5	$20.30	$254.
State program education staff	Beneficiary Satisfaction Interviews guide	20	1	0.50	10	$25.66	$257.
Total	—	45	—	—	23	—	$511

B.12.3 Focus Group Round 1

Interest and eligibility for participation in the focus group component of the study will be evaluated using the Screener (Attachments 4.a. in English & 8.a. in Spanish). Based on previous experience in recruiting participants, we estimate that screening burden will average 5 minutes per response and that the response rate for recruitment into the study will be approximately 50 percent. Each focus group will consist of an average of nine beneficiaries and will last 90 minutes. The focus group discussions will be led by a professional focus group moderator (see Attachments 4.b. in English & 8.b. in Spanish, Round 1& Round 2 Focus Group Discussion Guide). We estimate that 30 focus groups will be conducted over an approximately nine month timeframe. To allow for any last-minute changes in availability, 11 beneficiaries will be recruited to each focus group (i.e., 9 beneficiaries and 2 alternates). Therefore, to ensure participation of 270 for 30 focus groups per 1.5 years (i.e., 30 focus groups x 9 respondents), a total of 330 beneficiaries will be recruited (i.e., 30 focus groups x 11 respondents). In the case that all 11 respondents show up, the two alternates will be asked to leave with payment. Given the expected 50 percent response rate, we estimate that a total of 660 beneficiaries will be screened for participation throughout the year (i.e., 330 respondents x 2). In calculating the burden of the Focus Groups, 660 beneficiaries will be screened for the Focus Groups (a burden of 5 minutes), but only 270 of these 660 will participate in the Focus Groups (a burden of 1.5 hours). Information will be collected over an approximately nine month timeframe. Average hourly wage rate for beneficiaries was calculated using an estimated 40-hour work week and usual hourly earnings of $22.22⁸. Up to four of the 30 focus groups may be conducted in Spanish. Focus groups conducted in Spanish will have the same burden estimates as those conducted in English. The total estimated annualized cost to respondents is $10,221, as summarized in Table 5.

Table 5. Estimated Focus Groups Round 1 Response Burden Table (Hours and Wages)

Type of Respondent	Form Name	Number of Respondents	Number of Responses per Respondent	Average Burden per Response (in hours)	Total Burden (in hours)	Average Hourly Wage Rate	Total Cost
State program participants	Beneficiary Focus Group Screener Form Round 1	660	1	.08	54.98	$22.22	$1,222
State program participants	Moderator’s Guide for Beneficiary Focus Groups Round 1	270	1	1.50	405.00	$22.22	$8,999
Total	—	660	—	—	460	—	$10,221

B.12.4 Focus Group Round 2

Interest and eligibility for participation in the focus group component of the study will be evaluated using the Screener (Attachment 4.a.). Based on previous experience in recruiting participants, we estimate that screening burden will average 5 minutes per response and that the response rate for recruitment into the study will be approximately 50 percent. Each focus group will consist of an average of nine beneficiaries and will last 90 minutes. Similar to round one focus group discussions, round two discussions will be led by a professional focus group moderator. We estimate that 10 focus group discussions will be conducted. To allow for any last-minute changes in availability, 11 beneficiaries will be recruited to each focus group (i.e., 9 beneficiaries and 2 alternates). Therefore, to ensure participation of 90 (i.e., 10 focus groups x 9 respondents), a total of 110 beneficiaries will be recruited (i.e., 10 focus groups x 11 respondents). In the case that all 11 respondents show up, the two alternates will be asked to leave with payment. Given the expected 50 percent response rate, we estimate that a total of 220 beneficiaries will be screened for participation throughout the year (i.e., 110 respondents x 2). In calculating the burden of the Focus Groups, 220 beneficiaries will be screened for the Focus Groups (a burden of 5 minutes), but only 90 of these 220 will participate in the Focus Groups (a burden of 1.5 hours). Average hourly wage rate for beneficiaries was calculated using an estimated 40-hour work week and usual hourly earnings of $22.22⁹. The total estimated annualized cost to respondents is $3,407, as summarized in Table 6.

Table 6. Estimated Focus Groups Round 2 Response Burden Table (Hours and Wages)

Type of Respondent	Form Name	Number of Respondents	Number of Responses per Respondent	Average Burden per Response (in hours)	Total Burden (in hours)	Average Hourly Wage Rate	Total Cost
State program participants	Beneficiary Focus Group Screener Form Round 2	220	1	.08	18.33	$22.22	$407
State program participants	Moderator’s Guide for Beneficiary Focus Groups Round 2	90	1	1.50	135	$22.22	$3,000
Total	—	220	—	—	153	—	$3,407

B.12.5 Beneficiary Satisfaction Survey

We will first mail the survey to 6,139 participants and anticipate that 1,473 (24 percent), roughly a quarter of the sample, will complete the mail survey (see Attachments in English 6.a., 6.b., & 6.c.& in Spanish 9.a., 9.b., & 9.c.). We will send a second mailing to the 4,490 participants who did not respond initially (see Attachments 6.d. & 9.d. for the Reminder Letter in English and Spanish, respectively). Of those who receive a second mailing, we estimate that 781 (13 percent) of the sample will complete the mail survey. Thus, we estimate that a total of 2,254 beneficiaries will complete the survey by mail. Based on pre-testing with fewer than nine participants, the mail survey takes 15 minutes to complete. We will call the 3,885 participants who did not complete a mail survey. We estimate that 1,307 (34 percent) of 3,885 participants who do not complete a mail survey, will complete a survey over the phone once they are called (see Attachments 6.e. & 9.e. for Survey Telephone Script in English and Spanish, respectively). Based on experience with self-administered and telephone surveys, it will take longer (20 minutes) to complete the survey by phone. Average hourly wage rate for beneficiaries was calculated using an estimated 40-hour work week and usual hourly earnings of $22.22¹⁰. The total estimated annualized cost to respondents is $22,202 as summarized in Table 7. Surveys completed in Spanish have the same burden estimates as those completed in English.

Table 7. Estimated Beneficiary Satisfaction Survey Response Burden Table (Hours and Wages)

Type of Respondent	Form Name	Number of Respondents	Number of Responses per Respondent	Average Burden per Response (in hours)	Total Burden (in hours)	Average Hourly Wage Rate	Total Cost
Respondents	Beneficiary Satisfaction Survey mail-version	2,254	1	0.25	564	$22.22	$12,521
Respondents	Beneficiary Satisfaction Survey telephone-completion	1,307	1	0.33	436	$22.22	$9,681
Total	—	3,561	—	—	1,000	—	$22,202

B.13 Capital Costs

Respondents will incur no capital or maintenance costs to complete this data collection.

B.14 Cost to Federal Government

Table 8. Costs to the Federal Government: CMS

Agency	Information Collection Form	Action	Total Hours per Staff	Number of Staff	Total Hours	Total Cost	Cost Description
CMS	Site Visit Interviews (1^st set of interviews)	Start-Up Costs: Review five protocols Provide comments on five protocols	11 10 1	2 2 2	22 20 2	$1,170 $1,063 $106	GS-13staff¹¹: 11 hrs x $49.00 GS-14 staff:11 hrs x$57.33¹²
CMS	Site Visit Interviews (1^st set of interviews)	Reporting Costs: Review site visit notes Review site visit section of the report to congress	18 10 8	2 2 2	36 20 16	$1,914 $1,063 $851	GS-13staff: 18 hrs x $49.00 GS-14 staff:18 hrs x$57.33

Agency

Information Collection Form

Action

Total Hours per Staff

Number of Staff

Total Hours

Total Cost

Cost Description

CMS

Site Visit Interviews

(1^st set of interviews)

Start-Up Costs:

Review five protocols
Provide comments on five protocols

$1,170

$1,063

$106

GS-13staff¹¹: 11 hrs x $49.00 GS-14 staff:11 hrs x$57.33¹²

CMS

Site Visit Interviews

(1^st set of interviews)

Reporting Costs:

Review site visit notes
Review site visit section of the report to congress

$1,914

$1,063

$851

GS-13staff: 18 hrs x $49.00 GS-14 staff:18 hrs x$57.33

(continued)

Table 8. Costs to the Federal Government: CMS (continued)

Agency	Information Collection Form	Action	Total Hours per Staff	Number of Staff	Total Hours	Total Cost	Cost Description
CMS	Stakeholder Interviews (2^nd set of interviews)	Start-Up Costs: Review one protocol Provide comments on the protocol	2.33 2 .33	2 2 2	4.67 4 .67	$248 $213 $35	GS-13staff: 2.33 hrs x $49.00 GS-14 staff:2.33hrs x$57.33
CMS	Stakeholder Interviews (2^nd set of interviews)	Reporting Costs: Review stakeholder interview notes Review stakeholder interview section of the report to congress	4 2 2	2 2 2	8 4 4	$426 $213 $213	GS-13staff: 4 hrs x $49.00 GS-14 staff:4 hrs x$57.33
CMS	Beneficiary Satisfaction Survey	Start-Up Costs: Review survey (3 drafts) Provide comments on survey (3 drafts) Review and provide input on sampling analysis plan	17 6 3 8	2-3 2 2 3	42 12 6 24	$2,344 $638 $319 $1,387	GS-13staff: 17 hrs x $49.00 GS-14 staff:17 hrs x$57.33 GS-15 staff¹³: 8 hrs x $67.00
CMS	Beneficiary Satisfaction Survey	Reporting Costs: Review of initial findings and discussion of analysis Review of reports (2 drafts)	16 8 8	2-3 3 2	40 24 16	$2,238 $1,387 $851	GS-13staff: 16 hrs x $49.00 GS-14 staff:16 3hrs x$57.33 GS-15 staff: 8 hrs x $67.00
CMS	Focus Group Discussions Round 1 & 2	Start-Up Costs: Review discussion guides and materials	10 10	2 2	20 20	$1,063 $1,063	GS-13staff: 10 hrs x $49.00 GS-14 staff:10 hrs x$57.33
CMS	Focus Group Discussions Round 1 & 2	Reporting Costs: 15 minutes per month (for 13 months, for 2 CMS staff) to address issues related to recruitment and general implementation of the focus groups Review, discuss and provide comment and approve 2 drafts of each State topline report	31.25 3.25 28	2 2 2	62.5 6.5 56	$3,323 $346 $2,977	GS-13staff: 31.25 hrs x $49.00 GS-14 staff:31.25hrs x$57.33

Agency

Information Collection Form

Action

Total Hours per Staff

Number of Staff

Total Hours

Total Cost

Cost Description

CMS

Stakeholder Interviews

(2^nd set of interviews)

Start-Up Costs:

Review one protocol

Provide comments on the protocol

2.33

.33

4.67

.67

$248

$213

$35

GS-13staff: 2.33 hrs x $49.00 GS-14 staff:2.33hrs x$57.33

CMS

Stakeholder Interviews

(2^nd set of interviews)

Reporting Costs:

Review stakeholder interview notes

Review stakeholder interview section of the report to congress

$426

$213

GS-13staff: 4 hrs x $49.00 GS-14 staff:4 hrs x$57.33

CMS

Beneficiary Satisfaction Survey

Start-Up Costs:

Review survey (3 drafts)
Provide comments on survey (3 drafts)

Review and provide input on sampling analysis plan

2-3

$2,344

$638

$319

$1,387

GS-13staff: 17 hrs x $49.00 GS-14 staff:17 hrs x$57.33

GS-15 staff¹³:

8 hrs x $67.00

CMS

Beneficiary Satisfaction Survey

Reporting Costs:

Review of initial findings and discussion of analysis

Review of reports (2 drafts)

2-3

$2,238

$1,387

$851

GS-13staff: 16 hrs x $49.00 GS-14 staff:16 3hrs x$57.33

GS-15 staff:

8 hrs x $67.00

CMS

Focus Group Discussions Round 1 & 2

Start-Up Costs:

Review discussion guides and materials

$1,063

$1,063

GS-13staff: 10 hrs x $49.00 GS-14 staff:10 hrs x$57.33

CMS

Focus Group Discussions Round 1 & 2

Reporting Costs:

15 minutes per month (for 13 months, for 2 CMS staff) to address issues related to recruitment and general implementation of the focus groups
Review, discuss and provide comment and approve 2 drafts of each State topline report

31.25

3.25

62.5

6.5

$3,323

$346

$2,977

GS-13staff: 31.25 hrs x $49.00

GS-14 staff:31.25hrs x$57.33

Table 9. Summary Table: CMS Start-Up and Reporting Costs

Cost Type	Information Collection Form	Action	Total Cost
Start-Up Costs	Site Visit Interviews	Review five protocols	$1,063
Start-Up Costs	(1^st set of interviews)	Provide comments on five protocols	$106
Start-Up Costs	Stakeholder Interviews	Review one protocol	$213
Start-Up Costs	(2^nd set of interviews)	Provide comments on the protocol	$35
Start-Up Costs	Beneficiary Satisfaction Survey	Review survey (3 drafts)	$638
Start-Up Costs	Beneficiary Satisfaction Survey	Provide comments on survey (3 drafts)	$319
Start-Up Costs	Beneficiary Satisfaction Survey	Review and provide input on sampling analysis plan	$1,387
Start-Up Costs	Focus Group Discussions Round 1 & 2	Review discussion guides and materials	$1,063
Start-Up Costs Total	—	—	$4,824
Reporting Costs	Site Visit Interviews	Review site visit notes	$1,063
Reporting Costs	(1^st set of interviews)	Review site visit section of the report to congress	$851
Reporting Costs	Stakeholder Interviews	Review stakeholder interview notes	$213
Reporting Costs	(2^nd set of interviews)	Review stakeholder interview section of the report to congress	$213
Reporting Costs	Beneficiary Satisfaction Survey	Review of initial findings and discussion of analysis	$1,387

Reporting Costs	Beneficiary Satisfaction Survey	Review of reports (2 drafts)	$851
Reporting Costs	Focus Group	15 minutes per month (for 13 months, for 2 CMS staff) to address issues related to recruitment and general implementation of the focus groups	$346
Reporting Costs	Discussions Round 1 & 2	Review, discuss and provide comment and approve 2 drafts of each State topline report	$2,977
Reporting Costs Total	—	—	$7,901
Grand Total	—	—	$12,725

Table 10. Costs to the Federal Government: Contractor

Agency	Task	Total Cost Amount
Contractor¹⁴^,¹⁵	Project Development	$84,151
Contractor	Data Collection Activities	$238,429
Contractor	Data Processing and Analysis	$224,403
Contractor	Publication of Results	$14,025
Contractor	Project Management	$93,444
Contractor	TOTAL	$654,452

B.15 Changes to Burden

This is a new data collection.

B.16 Publication/Tabulation Dates

CMS anticipates disseminating the results of the study after the final analyses of the data are completed, reviewed, and cleared. The exact timing and nature of any such dissemination has not been determined, but may include presentations and articles at trade and academic conferences, publications, and Internet posting as well as the Reports to Congress required by the law.

The collected questionnaire data will be edited for cleaning purposes before entering them into an electronic file. The datasets that result from different modes of data collection will be merged into a single SAS dataset with a codebook. The codebook will contain a detailed description of each variable, a description of the response categories, and a frequency count or range of value for each variable.

Sample weights will be computed such that statistical inference can be made regarding the target populations following the RTI Technical Operating Procedure on Sample Weighting (RTI, 2002). These will include design weights, which equal the inverse of the sample selection probability and adjusted weights for nonresponses and post-stratification purpose. Descriptive statistics such as weighted estimates of means, proportions, and rates by States and their standard errors will be produced. The statistical package for survey data, SUDAAN, will be used for computing sample weights and data analysis.

We will develop a detailed analysis plan for review and approval by CMS. The plan will specify the types of analysis to be conducted to address the evaluation questions and will include dummy tables. We anticipate conducting the following types of analyses using statistical tests and logistic regression models for weighted sample survey data:

Across States, compare satisfaction (overall, accessibility, quality) by beneficiary characteristics, including sociodemographic characteristics (e.g., race/ethnicity, education) and health conditions.
Across States, compare satisfaction by program characteristics, including the disease focus areas (e.g., diabetes, smoking cessation) and incentive (e.g., cash vs. noncash, value). For incentives, we will also investigate whether a program’s incentives can be classified as “front-loaded” (with more frequent and larger incentives paid early in a beneficiary’s participation) or “back-loaded.” If so, we will consider whether there is sufficient variation between programs to test whether front-loaded incentives lead to better outcomes than back-loaded incentives. We will also examine whether participation-based incentives (e.g., participation in a weight loss class, attendance at a smoking cessation program) produce better outcomes than outcome-based incentives (e.g., losing weight, stopping smoking).
Across States, examine satisfaction among special populations of interest (adults with disabilities, adults with chronic illness, children with special health care needs [response by proxies]) and how satisfaction compares among beneficiaries generally.
Across States, examine differences in satisfaction between beneficiaries with different levels of participation in the program (e.g., completed program, discontinued program).
Examine the predictors of overall satisfaction and of satisfaction with accessibility and with quality of care specifically.

B.17 Expiration Date

The Beneficiary Satisfaction Survey will be implemented one time only, the focus groups only twice, and each set of stakeholder interviews—site visit interviews and interviews conducted during the same timeframe as focus group discussions are conducted--will be implemented once. The expiration date of OMB approval will be displayed on all information collection instruments.

References

Abreu, D.A., & Winters, F. (1999). Using monetary incentives to reduce attrition in the survey of income and program participation. Proceedings of the Survey Research Methods Section of the American Statistical Association.

Greenbaum, T.L. (2000). Moderating Focus Groups: A Practical Guide for Group Facilitation. Thousand Oaks: Sage Publications, Inc.

Krueger, R. A., & Casey, M. A. (2009). Focus Groups. A Practical Guide for Applied Research (4th edition). Thousand Oaks, CA: Sage Publishers.

Shettle, C., & Mooney, G. (1999). Monetary incentives in U.S. government surveys. Journal of Official Statistics, 15, 231-250.

U.S. Department of Labor, Bureau of Labor Statistics. (2011). May 2011 national occupational employment and wage estimates United States. Washington, DC: U.S. Department of Labor. http://www.bls.gov/oes/current/oes_nat.htm. Last updated on March 29, 2012.

1The Bureau of Labor Statistics estimates that medical and health service manages, on average, earned $46.17 in 2011. The figure is updated to $47.25 in 2012 dollars using the CPI, which is approximately 1.02. http://www.bls.gov/oes/current/oes_nat.htm#11-0000

2The Bureau of Labor Statistics estimates that a community and social service specialists, on average, earned $19.83 in 2011. The figure is updated to $20.30 in 2012 dollars using the CPI, which is approximately 1.02. http://www.bls.gov/oes/current/oes_nat.htm#11-0000

3The Bureau of Labor Statistics estimates that a health educator, on average, earned $25.07 in 2011. The figure is updated to $25.66 in 2012 dollars using the CPI, which is approximately 1.02. http://www.bls.gov/oes/current/oes_nat.htm#11-0000

4The Bureau of Labor Statistics estimates that a social scientist and related worker, on average, earned $37.82 in 2011. The figure is updated to $38.71 in 2012 dollars using the CPI, which is approximately 1.02. http://www.bls.gov/oes/current/oes_nat.htm#11-0000

5The Bureau of Labor Statistics estimates that a physician assistant, on average, earned $43.01 in 2011. The figure is updated to $44.02 in 2012 dollars using the CPI, which is approximately 1.02. http://www.bls.gov/oes/current/oes_nat.htm#11-0000

6The Bureau of Labor Statistics estimates that a health educator, on average, earned $25.07 in 2011. The figure is updated to $25.66 in 2012 dollars using the CPI, which is approximately 1.02. http://www.bls.gov/oes/current/oes_nat.htm#11-0000

7The Bureau of Labor Statistics estimates that a community and social service specialists, on average, earned $19.83 in 2011. The figure is updated to $20.30 in 2012 dollars using the CPI, which is approximately 1.02. http://www.bls.gov/oes/current/oes_nat.htm#11-0000

8The Bureau of Labor Statistics estimates that a first-line supervisors of landscaping, lawn service, and groundskeeping workers, on average, earned $21.71 in 2011. The figure is updated to $22.22 in 2012 dollars using the CPI, which is approximately 1.02. http://www.bls.gov/oes/current/oes_nat.htm#11-0000

9The Bureau of Labor Statistics estimates that a first-line supervisors of landscaping, lawn service, and groundskeeping workers, on average, earned $21.71 in 2011. The figure is updated to $22.22 in 2012 dollars using the CPI, which is approximately 1.02. http://www.bls.gov/oes/current/oes_nat.htm#11-0000

10The Bureau of Labor Statistics estimates that a first-line supervisors of landscaping, lawn service, and groundskeeping workers, on average, earned $21.71 in 2011. The figure is updated to $22.22 in 2012 dollars using the CPI, which is approximately 1.02. http://www.bls.gov/oes/current/oes_nat.htm#11-0000

11The U.S. Office of Personnel Management indicates that the annual salary for a GS-13, Step 5 employee in the Washington/Baltimore/Northern Virginia locality is $100,904 which when divided by 2080 hours is $49.00 an hour. http://www.opm.gov/oca/12tables/GSCalc.asp

12The U.S. Office of Personnel Management indicates that the annual salary for a GS-14, Step 5 employee in the Washington/Baltimore/Northern Virginia locality is $119,238 which when divided by 2080 hours is $57.33 an hour. http://www.opm.gov/oca/12tables/GSCalc.asp

13The U.S. Office of Personnel Management indicates that the annual salary for a GS-15, Step 5 employee in the Washington/Baltimore/Northern Virginia locality is $140,259which when divided by 2080 hours is $69.00 an hour. http://www.opm.gov/oca/12tables/GSCalc.asp

14 Overhead is included in all costs listed.

15 Cost estimates taken from contract budget.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Modified	0000-00-00
File Created	2021-01-28