0990-0223Supporting Statment_CCv4

Supporting Statement for

Evaluation of the Cash and Counseling Demonstration – OMB No. 0990-0223 - Reinstatement with Changes

1. Justification

In this submission, the Office of the Assistant Secretary for Planning and Evaluation (ASPE) of the U.S. Department of Health and Human Services seeks clearance to reinstate and make changes to a study of adult Medicaid beneficiaries in New Jersey. This reinstatement will include a survey of beneficiaries who were eligible to enroll in the state’s Cash and Counseling program, Personal Preference, but did not choose to enroll in the program, in order to inform policy makers about the factors the influence program participation.

1. Need and Legal Basis.

The initial evaluation (OMB NO: 0990-0223, “Evaluation of the Cash and Counseling Demonstration”) of the national Cash and Counseling Demonstration showed that self-directing consumers are more satisfied with their supportive services, report fewer unmet needs, and enjoy greater well-being (Carlson et al. 2004).¹ The original evaluation of the national Cash and Counseling Demonstration was intended to include three groups: self-directing consumers, a control group, and non-participants. When funding was not available to survey all groups, the non-participant sample was removed. Unfortunately, few (8-20%) of the beneficiaries who were eligible to participate in Cash and Counseling demonstrations elected to do so, making the non-participant group an important sample to study. This survey will give federal and state officials a better understanding of which eligible beneficiaries did not participate in self-direction and why they chose not to. This study is mandated by legal code in Section 301 of the Public Health Service Act (42.U.S. C 241).

2. Information Users

Policy makers and program administrators will use this data to understand what factors encourage or constrain the participation of eligible beneficiaries, and whether those factors would be amenable to the way Cash and Counseling programs are designed or explained to potential participants. Although previous efforts to understand reasons for nonparticipation (focus groups of eligible non-participants and brief questionnaires (OMB No. 0990-0223 and 0990-0232) were useful for forming hypotheses, the samples were small and not representative of all eligible non-participants. ASPE, CMS, and other Federal partners will use the information collected through this survey to identify Medicaid beneficiaries with chronic disabilities eligible for self-directed services options who are most likely to be interested in and most likely to benefit from these services, and barriers to participation. This knowledge will inform federal regulatory policy and interpretive guidance provided to states with respect to self-directed services options provided under state plan or HCBS waiver authority.

3. Improved Information Technology

MPR will administer the proposed survey of New Jersey non-participants through telephone interviewing, using a hard copy instrument. Telephone interviews will be used rather than face-to-face interviews because the former are more cost-effective for the government and less burdensome to survey respondents. Telephone surveys were used successfully in the original Cash and Counseling Demonstration and Evaluation, so there is every reason to believe that use of this approach will yield similar high quality data. The interviewers will use hard copy instruments, rather than computer-assisted telephone interviewing (CATI), because a fairly small number of people will be interviewed and because the logical flow of the survey instrument is quite straightforward. Because CATI is resource-intensive, it is generally used in projects with large samples and/or highly complex survey instruments. Data from the interviews will be entered into electronic databases, which will have error checking capabilities (such as checking for skip error logic and out-of-range responses).

3. Duplication of Similar Information

ASPE conducted a search of the terms “Cash and Counseling” and “consumer-directed care,” with the terms “non-respondents” and “non-participants.” No similar studies were found in the PubMed, Ovid, or ProQuest databases.

ASPE also interviewed the directors of the Cash and Counseling program in New Jersey, including William Ditto, the Director of the New Jersey Division of Disability Services in the New Jersey Department of Human Services and Project Director for Personal Preferences (N.J.’s “Cash and Counseling” Medicaid waiver program). His address is NJ Department of Human Services P.O. Box 700, Trenton, New Jersey 08625-0700; his telephone number is (609)-292-7800 and his e-mail address is [email protected]. Mr. Ditto informed us that no other agencies or researchers have collected or are collecting information that could be used to answer the main research questions of the proposed study.

We also interviewed the staff of the “Cash and Counseling” Research and Technical Assistance Center at the Boston College School of Social Work. The Center Director is Kevin Mahoney, Ph.D. His contact information is: (617) -562-4039. Boston College, McGuinn Hall, Room 600, Chestnut Hill MA 02467, [email protected] . The Cash and Counseling Research and Technical Assistance Center has been funded by ASPE, the Administration on Aging, and Centers for Medicare and Medicaid, and private foundations (Robert Wood Johnson Foundation) to provide technical assistance to states that elect to offer “cash and counseling” options to chronically disabled Medicaid of all ages and their family caregivers as well as older Americans and caregivers eligible for Older Americans Act funded services. As such, the Boston College Cash and Counseling Research and Technical Assistance Center is aware of all Cash and Counseling efforts nationwide. Dr. Mahoney informed us that these data are not being collected by any other agencies or organizations.

Information collected during the proposed study will supplement that which was collected, by MPR, during the original evaluation of the New Jersey Cash and Counseling evaluation. Current non-participants in New Jersey will be compared with people who were enrolled in the Cash and Counseling demonstration in New Jersey (participants) who previously completed a survey. Existing survey data will be used for these participants, rather than new data collection for them or their more current counterparts.

3. Small Businesses

No small businesses will be involved in this study.

3. Less Frequent Collection

This is a one time data collection survey. If these data are not collected at all, then the need for the data will not be met.

3. Special Circumstances

None of the special circumstances listed in Item 7 of the instructions for the supporting statement are applicable to this data collection effort. The request fully complies with this regulation.

3. Federal Register Notice/Outside Consultation

A 60-day Federal Register Notice was published in the Federal Register April 28, 2008, vol. 73, No. 82; pp. 22949-22950. There were no comments.

ASPE consulted with Kevin Mahoney, Ph.D., the director of the Cash and Counseling Research and Technical Assistance Center. His contact information is: (617) -562-4039. Boston College, McGuinn Hall, Room 600, Chestnut Hill MA 02467, [email protected] . Dr. Mahoney had no comments.

3. Payment/Gift to Respondents

No payments or gifts will be provided to survey respondents. ASPE does not anticipate any unusual difficulties in gaining cooperation of sample members.

3. Confidentiality

The Certificate of Confidentiality was waived for the original Evaluation of Cash and Counseling study. The changes involved in this reinstatement do not impact the confidentiality so ASPE is not seeking a Certificate of Confidentiality.

3. Sensitive Questions

The proposed data collection instrument does not contain questions that would normally be considered sensitive.

3. Burden Estimate (Total Hours & Wages)

A total of 600 New Jersey Medicaid beneficiaries who did not participate in the initial Cash and Counseling Demonstration will be surveyed. With a sample of 600, there will be sufficient statistical power to test whether participants differ significantly from non-participants on key characteristics (e.g., knowledge of the program, living situation, martial status, race/ethnicity). ASPE estimates that it will take 27 minutes for each participant to complete the survey, based on completion times for nine pretest respondents (For a discussion of pre-testing, see Section B.4.). Completion time could vary because the data collection instrument includes several open-ended questions and some respondents could provide relatively expansive replies. Spanish-language interviews, as well as those in other languages (which would require interpreters), would most likely take a bit longer to complete. The total estimated annualized burden hours are shown in 12A below.

12A. Estimated Annualized Burden Hours

Type of Respondent	Form Name	No. of Respondents	No. Responses per Respondent	Average Burden per Response (in hours)	Total Burden Hours
Non-Participants (or Proxies)	Telephone Interview	600	1	27/60	270

12B. Estimated Annualized Cost to Respondents

Type of Respondent	Total Burden Hours	Hourly Wage Rate	Total Respondent Costs
Non-Participants (or Proxies	270	$10.00 (minimum wage or less)	$2700.00

3. Capital Costs (Maintenance of Capital Costs)

Respondents will incur no monetary costs in completing the interviews. Those respondents who do not have telephones at home will be invited to call a toll-free number if they wish to complete an interview elsewhere.

3. Cost to Federal Government

The study is funded entirely by ASPE. The total budget for the study, including design, data collection, and analysis and reporting, is $499,679; $149,945 for design and $349,734 for implementation.

3. Program or Burden Changes

This is a new data collection.

3. Publication and Tabulation Dates

The main products of the nonparticipation study will be a project report, an article-length version of the report, and a government briefing. The report will be about 30 pages in length (including tables). The report’s intended audience includes federal and state policymakers and program administrators in both the original and the “next-generation” Cash and Counseling states. In preparing the report, MPR researchers will:

• Examine raw frequency distributions of non-participants’ responses to survey questions and code verbatim responses.

• Specify and construct categorical variables and subgroup indicators.

• Analyze frequency distributions for constructed variables and cross-tabulate selected variables with subgroup indicators.

• In the description of non-participants, they will estimate confidence intervals around sample proportions.

• In the comparison of non-participants and participants, they will test for bivariate associates between participation decisions and other characteristics of interest. They will conduct multivariate regression analysis to predict participation decision as a function of demographic and pre-decision characteristics.

A schedule of major study activities is shown in Table 1.

TABLE 1: SCHEDULE OF MAJOR ACTIVITIES

Activity	Timing (in months after OMB approval)
Modify instrument according to OMB comments	1 month
Translate instrument into Spanish	1 month
Acquire sample	2.5 months
Send advance letters	2.5 months
Prepare interviewer training manuals	2.5 months
Train interviewers	2.5 months
Conduct interviews	5.5 months
Enter responses into electronic database	7 months
Construct analytic files	7.5 months
Write draft report, send to ASPE and TAG	9.5 months
Finalize report based on comments	11 months
Write article-length version of report	12 months
Brief government officials	12 months

3. Expiration Date

The expiration date of the OMB approval will be displayed on the questionnaire and in the advance letters to prospective respondents. (See the advance letter in Appendix A.)

3. Certification Statement

There are no exceptions to the certification.

B. Collection of Information Employing Statistical Methods

1. Respondent Universe and Sampling Methods

The sampling frame will be a list of Medicaid beneficiaries in New Jersey who are eligible to enroll in the state’s Cash and Counseling program, Personal Preference, but who did not enroll. MPR will obtain the frame upon resumption of outreach to, and enrollment of, new prospects in New Jersey. Those who were part of the Cash and Counseling demonstration evaluation (in either the treatment or control group) will be excluded. MPR will consult with New Jersey administrators to determine whether the sample frame of current eligible nonparticipants should be drawn from current Medicaid personal care claims data, from contact lists the state used during outreach activities, or from some other source. If using claims data as the frame, they will identify those with claims for cashed-out personal care benefits while it was possible to enroll. MPR will work with state program staff to ensure that they are in compliance with HIPAA regulations when forming the sampling frame, selecting the sample, and contacting sample members.

MPR will select a stratified random sample large enough to yield 600 completed telephone interviews. The assumption is that 90 percent of the sample will be eligible, and that 75 percent of the eligible sample members will respond. That would mean the initial sample release should be about 889. Stratification variables may include age group, gender, race/ethnicity, tenure in Medicaid personal assistance program, and county. MPR may also stratify by whether the beneficiary ever expressed any interest in the program. For these stratification variables, MPR will work with the ASPE project officer to decide whether to oversample certain subgroups, or just to ensure that they are all proportionately represented in the sample, using either explicit (cell-based) or implicit (sort-based) stratification. MPR will most likely use a sequential sampling technique based on a procedure developed by Chromy and available as a procedures in SAS (SurveySelect).²

2. Procedures for the Collection of Information

MPR will be collecting primary data using an interviewer-administered telephone survey over a three-month period approximately five months after OMB approval. The survey includes original questions designed to measure factors related to nonparticipation. It also contains some questions similar or identical to those asked in the evaluation of the Cash and Counseling demonstration surveys. Those questions will allow comparisons between current participants and people who were part of the Cash and Counseling demonstration (participants).

MPR will use a hard copy survey instrument to conduct the telephone interview, and then enter the data into an electronic database. As part of this process, they will check for errors in skip patterns and out-of-range response values. As noted, with only 600 completed interviews, and a relatively straightforward instrument, it is not worth the cost of programming the survey instrument into a computer-assisted telephone interview.

3. Methods to Maximize Response Rates and Deal with Nonresponse

MPR will use several techniques to maximize response rates in the survey sample. As noted, it will distribute advance letters to explain the study and solicit the cooperation of eligible respondents. Then, when it attempts to interview sample members by telephone, MPR will employ proven procedures for locating sample members, and preventing and converting refusals. They expect to use standard locating resources for about three-quarters of the sample members. Interviewers will be given specific suggestions for addressing respondents’ reluctance to participate, thus avoiding refusals in the first place. When sample members are reluctant to participate in an interview, the interviewer will enter the reason for a respondent’s reluctance on the contact sheet. In addition, all refusals will be evaluated by a supervisor who will judge whether an attempt to convert the refusal is warranted and appropriate. If it is, refusal conversions will be conducted by specially trained, highly experienced, persuasive interviewers who enjoy the challenge of overcoming a refusal. When appropriate, MPR will also send out refusal conversion letters targeted to specific situations. (Example of refusal conversion letter included in Appendix 2.)

Inappropriate items and phrasing can lead to break-offs, thus reducing completions and response rates. Different versions of the questionnaire will be developed for different respondents, inserting appropriate pronouns (second person or third person) as appropriate to avoid interviewer errors that may confuse the respondent and/or collect information about the wrong person. In addition, barriers to completion include sample members’ being physically or cognitively unable to use a telephone, linguistic preference, and, less commonly, living in a household with a telephone. To overcome these barriers, MPR will, respectively, conduct interviews with knowledgeable proxy respondents, conduct interview in Spanish (the most common non-English preference for this sample) or make use of interpreters, and provide a toll-free number so that respondents can participate in interviews from a neighbor’s house or public telephone.

4. Tests of Procedures or Methods to be Undertaken

During the first week of October 2004, MPR conducted a pretest with nine Arkansas Medicaid beneficiaries who were eligible for a Cash and Counseling program, IndependentChoices, but who did not sign up. Some had heard of IndependentChoices, and others had never heard of it. The nine respondents were people who called in and volunteered to do the “practice interview” after being sent a letter about the pretest from Arkansas’s Department of Human Services (Division of Aging and Adult Services). As a result of the pretest, MPR found several questions that required wording changes, or additional interviewer probes, to be clearer to respondents. No questions were removed as a result of the pretest. A fuller description of the pretest was sent to the ASPE project officer. The pretest with nine English-speaking respondents revealed that the interviews ranged from 19 to 37 minutes each, averaging 26.8 minutes.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The person responsible for sample acquisition and survey administration is:

• Barbara Lepidus Carlson, Senior Statistician, Mathematica Policy Research, Inc., (609) 275-2374.

The person responsible for overall research design, data analysis, and reporting is:

• Leslie Foster, Researcher, Mathematica Policy Research, Inc., (609) 936-3265.

C. Attachments/Appendix

1. Copies of applicable sections of laws or regulations [ASPE TO PROVIDE]

2. Data collection instrument

3. A copy of the 60-day Federal Register notice [ASPE TO PROVIDE]

4. Introductory and follow-up (refusal conversion) letters to respondents

1 Carlson BL, Dale S, Foster L, Phillips B, Brown R, and Schore J (2004). “Effect on Consumer Direction on Adults’ Personal Care and Well-Being in Three States.” Project report submitted to the University of Maryland. Princeton, NJ: Mathematica Policy Research, Inc.

2 This procedure offers all the advantages of the systematic sampling approach but eliminates the risk of systematic, list-order bias by making independent selections within each of the zones associated with systematic sampling, while controlling the selection opportunities for units crossing zone boundaries.

13