NSLTCP DC Supp State Part B 5-8

Supporting Statement B for Paperwork Reduction Act Submission for

Data Collection for the Residential Care Community and Adult Day Services Center Components of the National Study of Long-Term Care Providers

New OMB Application

May 8, 2012

Lauren Harris-Kojetin

Chief, Long-Term Care Statistics Branch

Division of Health Care Statistics

National Center for Health Statistics

Phone: 301.458.4369

Fax: 301.458.4693

Email: [email protected]

B. Statistical Methods

1. Respondent Universe and Sampling Methods

The National Study of Long-Term Care Providers (NSLTCP) includes nationally representative surveys of residential care communities (RCCs) and adult day services centers (ADSCs). The primary goal of the survey component of NSLTCP is to provide a general purpose database on RCCs and ADSCs which researchers and policymakers can use to address a wide variety of questions. As a general purpose survey, it will provide broad descriptive data and does not presuppose any particular typology of communities/centers or residents/participants. The main focus is on RCCs and ADSCs, with the survey gathering as much information about their residents/participants as possible while keeping response burden low and within budget constraints.

While the survey content is similar and the data collection protocol is the same for RCCs and ADSCs, each provider type has its own universe and sampling methods. The remainder of this section discusses the universe definition, sampling frame, and sampling methods for RCCs followed by a corresponding discussion for ADSCs.

RCCs: NCHS will use the same definition for RCC in NSLTCP as was used for defining a residential care facility in the 2010 National Survey of Residential Care Facilities (NSRCF) (OMB No. 0920-0780, Expired 12/31/12). As such, the following criteria will be used to determine the universe of RCCs which are eligible for selection in the NSLTCP survey:

Places that are licensed, registered, listed, certified, or otherwise regulated by the state to provide room and board with at least two meals a day, around-the-clock on-site supervision, and help with activities of daily living (e.g., bathing, eating, dressing) or health-related services (e.g., medication supervision); serve primarily an adult population; have at least four beds; and are serving at least one resident at the time of the survey.

The eligibility definition encompasses many types of RCCs, including assisted living places that arrange for personal care services from an outside vendor, as in Connecticut and Minnesota. Excluded are nursing facilities; facilities serving exclusively people with intellectual disabilities or developmental disabilities; group homes and residential care facilities serving exclusively people with severe mental illness; and other residential care settings where personal care or health related services are not arranged or provided. Unregulated communities are also excluded.

The sampling frame for the RCC component of the NSLTCP survey will be constructed from lists of licensed RCCs acquired from the licensing agencies in each of the 50 states and the District of Columbia (OMB No. 0920-0912, Expires 1/31/13). State data on the number of licensed beds for each community and the licensure categories will be used to determine the list of eligible communities. The RCC sampling frame for NSLTCP will contain all of the state-licensed RCCs that are licensed for four or more beds. Based on a frame developed by RTI International in 2009 for the Office of the Assistant Secretary for Planning and Evaluation and used for the 2010 NSRCF, we estimate that there are about 39,635 RCCs nationally.

NCHS aims for the sampling design for RCCs to enable reliable estimates to be made for (1) states, including the District of Columbia; (2) MSA status (metropolitan, micropolitan, neither); (3) RCC bed size categories using the same definitions used for the 2010 NSRCF (small - 4 to 10 beds, medium - 11 to 25 beds, large - 26 to 100 beds, and extra large - more than 100 beds); and (4) Census geographic regions (Northeast, Midwest, South, and West). Using the sample size estimates calculated to enable state-level estimation, the goal is to optimally allocate the sample to allow for as many state-level estimates as possible, while still providing sufficient sample sizes for estimates by Census region, MSA status and community bed size.

If a sample were drawn from each state, the minimum number of RCC completions needed per state is at least 104 in order to estimate a sample proportion of 30% with a relative standard error (RSE) of 15%. This would yield a total of 5,304 completions needed from all 50 states and the District of Columbia.

Various combinations of eligibility and response rates were assessed to determine the optimal sample size in order to have the required 104 completions for each state. Table 1-1 summarizes the overall sample sizes needed under the different scenarios. The scenarios present different response and eligibility rate assumptions.

Table 1-1. Summary of Overall Sample Size Calculations

	Response Rate Assumptions		Eligibility Rate Assumptions	Overall Sample Size
Uniform Response Rate of 65%	Small	65%	63%	9,993
	Medium	65%	77%
	Large	65%	88%
	Very Large	65%	93%
Scenario A (Overall Response Rate of 60%)	Small	63%	63%	10,534
	Medium	59%	77%
	Large	59%	88%
	Very Large	55%	93%
Scenario B (Overall Response Rate of 65%)	Small	68%	63%	9,994
	Medium	64%	77%
	Large	64%	88%
	Very Large	60%	93%

The expected response rate of 65% is based on response rates in other recent mail surveys. For instance, the unweighted response rates of the 2010 and 2011 EMR/EHR mail components of NCHS’s National Ambulatory Medical Care Survey (NAMCS) were 68% (66% weighted) and 64% (61% weighted) (http://www.cdc.gov/nchs/data/databriefs/DB79.pdf). Although NCHS will make a concerted effort to obtain a minimum response rate of 65% using the mail/web/telephone protocol, if the actual response rate is lower this could result in states for which state-level estimates are not possible. The final sample size was determined using bed size-specific eligibility rates based on the 2010 NSRCF, and scaling down from the 2010 NSRCF response rates (an overall weighted NSRCF 81% response rate which was achieved using in-person mode and a substantially longer field period than is possible for NSLTCP) to more attainable values (while retaining the relative ordering of response rates by the size strata), given the different mode and field period of the NSLTCP survey. As a result, applying the 2010 NSRCF strata-specific eligibility rates and an overall response rate of 60% adjusted for the 2010 NSRCF bed size strata-specific response would require a total sample of 10,534 (about 206.5 RCCs per state) for NSLTCP (Scenario A in Table 1-1).

Following the strategy used in the 2010 NSRCF, the overall sample will be allocated to the four bed size strata: (1) to detect a 7 percentage point difference in medium, large, and extra large RCCs for two groups of RCCs of equal size. This difference can be for two groups of equal size within a size stratum (e.g. rural vs. urban large facilities), or two groups of equal size between strata (e.g. comparing %nonprofit between equal numbers of medium and large facilities. (2) to detect a 7 percentage point difference between small RCCs and RCCs of all other sizes, and (3) to detect an 8 percentage point difference between small and medium or large RCCs, all with 80% power.

Column a in Table 1-2 shows the total number of RCCs from the 2010 NSRCF frame, by state. Columns b and c show the estimated maximum numbers of completions available for each state assuming size-specific eligibility rates from the 2010 NSRCF and a response rate of 65% and 60% respectively The next 2 columns (d and e) of Table 1-2 present the sample allocations for scenario A (defined in Table 1-1). The final sampling design for the RCC component of the NSLTCP is a combination of sampling in some states and taking a census in other states. Column c of Table 1-2 shows that there are 17 states that are expected to have fewer than 104 completions. A census will be taken in these 17 states. There are an additional 15 states where the difference between the total number of communities on the frame and the overall sample size from each state based on assumed response and eligibility rates was less than 200 (Column f). A census will be taken in these 15 states as well, bringing the total number of census states to 32. A difference of 200 between the total number of communities on the frame and the total sample size from each state was used as a threshold for deciding between a census and a sample because in states (except Hawaii) where the difference was over 200, the number of communities in the frame was at least twice the required sample . By increasing the number of census states to 32, this approach will increase the likelihood of producing state-level estimates for these states, which may not have been possible with a sample if response rates were lower than expected.

Probability samples will be selected in the 19 states that have at least 200 RCCs in excess of the total sample needed for state-level estimation. Among RCCs, 11,701 cases will be fielded; 5,193 of these are sampled from the 19 sampled-based states and 6,508 of these are all RCCs from each of the 32 states for which the universe will be included.

Table 1-2. Population Counts of RCCs on 2010 NSRCF frame, maximum number of completions, expected completions, overall sample size, and difference between frame count and overall sample size, by state (in Descending Order of RCC Population Size).

Location State	2010 NSRCF Frame Total	Maximum number of completions from 2010 NSRCF frame using size-specific eligibility rates and uniform response rate of 65%	Maximum number of completions from 2010 NSRCF frame using size-specific eligibility rates and uniform response rate of 60%	Total number of completions for scenario A: allocating overall sample of 10,534 using size-specific eligibility and response rates	Overall sample size for scenario A: sample size of 10,534 using size-specific eligibility and response rates	Difference between the number of RCCs on the 2010 NSRCF frame and the overall sample size for state (a-e)
	a	b	c	d	e	f
CA	7,633	3349	3092	215	476	7157
MI	3,216	1419	1313	172	397	2819
WA	3,072	1336	1234	120	284	2788
OR	2,273	993	919	113	270	2003
FL	2,168	1022	946	133	297	1871
AZ	1,905	816	755	111	269	1636
WI	1,696	801	742	131	295	1401
GA	1,516	692	641	117	270	1246
TX	1,437	717	664	127	275	1162
MN	1,262	650	602	120	255	1007
NC	1,207	587	543	111	246	961
PA	1,161	632	584	117	239	922
MD	1,118	501	465	109	257	861
OH	886	450	416	106	228	658
MO	581	311	288	105	216	365
VA	558	300	279	104	214	344
CO	491	240	225	105	232	259
NY	480	273	254	106	210	270
HI	467	195	181	106	263	204
SC	406	207	194	104	222	184
IA	385	212	198	105	211	174
ME	371	178	165	106	237	134
KS	338	174	164	105	221	117
Location State	2010 NSRCF Frame Total	Maximum number of completions from 2010 NSRCF frame using size-specific eligibility rates and uniform response rate of 65%	Maximum number of completions from 2010 NSRCF frame using size-specific eligibility rates and uniform response rate of 60%	Total number of completions for scenario A: allocating overall sample of 10,534 using size-specific eligibility and response rates	Overall sample size for scenario A: sample size of 10,534 using size-specific eligibility and response rates	Difference between the number of RCCs on the 2010 NSRCF frame and the overall sample size for state (a-e)
	a	b	c	d	e	f
NV	326	142	133	105	252	74
NJ	319	181	168	105	210	109
TN	307	168	156	105	213	94
MA	292	162	151	104	208	84
IL	269	146	136	106	215	54
KY	267	143	134	105	216	51
AK	259	110	103	105	255	4
NE	255	136	128	105	216	39
AL	251	135	126	105	216	35
ID	230	118	110	105	223	7
IN	213	122	114	105	206	7
OK	201	111	104	100	201	0
MT	208	100	94	93	208	0
NM	185	87	83	82	185	0
MS	177	92	86	83	177	0
UT	161	84	78	75	161	0
SD	155	78	75	73	155	0
NH	135	69	66	63	135	0
LA	123	64	61	59	123	0
VT	114	57	55	53	114	0
ND	112	59	57	54	112	0
AR	103	58	54	53	103	0
WV	101	51	50	47	101	0
CT	100	52	49	47	100	0
RI	62	34	34	30	62	0
DE	36	20	20	19	36	0
DC	26	13	13	13	26	0
WY	21	12	13	10	21	0
Total	39,635	18,659	17315	4857	10,534

For each primary stratum defined by bed size and state, RCC selection will be done by systematic random sampling from lists of RCCs in which the RCCs will be sorted by MSA status (metropolitan. micropolitan, neither) and randomly ordered within each MSA status.  Census regions are non-overlapping groups of states and by design, the sample will be sufficient to make census region-level estimates if the sample is sufficient to make state-level estimates.

Using an RCC 81% eligibility rate and a 65% response rate goal, we expect to obtain completed questionnaires for 6,162 RCCs out of the 11,701 cases fielded.

ADSCs: The National Adult Day Services Association (NADSA), a professional trade association, is the leading voice of the rapidly growing adult day services industry and the national focal point for ADSCs (www.nadsa.org). According to NADSA, ADSCs provide a coordinated program of services for adults in a community-based group setting. Services are designed to provide social and health services to adults who need supervised care in a safe place outside the home during the day, and to provide respite for caregivers. ADSCs generally operate during normal business hours five days a week. Although each ADSC may differ in terms of features, most ADSCs provide social activities, transportation to/from the ADSC, meals and snacks, assistance with activities of daily living, and therapeutic activities such as exercise and mental interaction.

The frame that NCHS will use for the ADSC component of the NSLTCP survey is the comprehensive listing of ADSC that NCHS is purchasing from NADSA. Purchasing this list from NADSA represents a substantial cost-savings over collecting, cleaning and concatenating licensing lists of ADSCs from each of the 50 states and the District of Columbia. In addition, NADSA’s list includes ADSCs located in states that do not license ADSCs. The NADSA list is a comprehensive list of ADSCs in the United States created in 2011 and updated in April 2012. The approach that NADSA uses in creating and maintaining the frame is inclusive; any program that self-identified as adult day care, adult day services, or adult day health services was included.  ADSCs were included if they offered socialization, nutritional support,   and “hands-on”   assistance with activities of daily living, at a congregate site, which had daytime hours.   Hands on assistance could include offering an arm as support to the bathroom.  Frame construction started with an existing NADSA data base.  NADSA staff contacted all ADCSs to verify they were still providing adult day care, and updated the contact information.  Several methods were used to identify additional ADSCs.  These included contacting state government offices with oversight of ADSCs to identify ADSCs that met state requirements. Other ADSCs were identified through phone books, the internet, and state adult day services associations not affiliated with NADSA. 

The goals of the survey for ADSCs is to enable reliable estimates to be made for (1) states, including the District of Columbia and (2) MSA status (metropolitan, micropolitan, neither). The ADSC eligibility rate is assumed to be virtually 100%, as all ADSCs included in the universe identified by NADSA will be eligible.

The minimum sample size required for each state to enable state-level estimation is 186 ADSCs, yielding 104 completions, with an overall sample of 8,160 and assuming an eligibility rate of 100% and a response rate of 65%. Table 1-3 shows the population counts on the NADSA sampling frame.

Table 1-3. Distribution of ADSCs by State (in Descending Order of ADSC Population Size)

State	Frame Counts of ADSCs
CA	958
TX	584
NY	407
PA	281
FL	241
MA	215
GA	190
OH	179
NJ	170
MN	158
MD	155
IL	138
MO	136
WI	122
KY	116
SC	114
NC	106
CO	73
MT	67
VA	64
MI	59
LA	57
NE	57
MS	54
TN	54
CT	52
AR	46
IN	45
OK	43
HI	42
ME	40
WA	37
IA	35

Table 1-3. Distribution of ADSCs by State (in Descending Order of ADSC Population Size) (continued)

State	Frame Counts of ADSCs
ND	35
NH	35
SD	33
AZ	24
NV	23
NM	22
OR	22
WV	22
AL	21
RI	21
KS	17
VT	17
AK	15
DE	13
WY	8
UT	6
DC	4
ID	4

Table 1-3 shows that there are currently only seven states with 186 or more ADSCs in the NADSA frame and the ADSC universe has a total of 5,437. A full census will be taken of the ADSCs for the NSLTCP survey, because the required ADSC sample size is larger than the universe of ADSCs. Therefore, no sampling specifications are required for the ADSC component of the NSLTCP survey. Using an ADSC 100% eligibility rate and a 65% response rate, we expect to obtain completed questionnaires for 3,534 ADSCs assuming a census of 5,437. However, the total number of ADSCs on the frame is subject to change after the April, 2012 frame update is received.

2. Procedures for the Collection of Information

NSLTCP includes a series of mailings (Attachments G1-G6). All items in the mailings have been reviewed and approved by NCHS’ confidentiality officer. The first step in data collection is mailout of the chain outreach package (Attachment H), which is sent for all RCC and ADSC cases that have been identified as being part of a national or regional chain. A week later, NCHS will mail an advance notification letter on NCHS letterhead to directors of all RCC and ADSC cases (Attachments G1 and G4). The letter will describe and encourage participation in the survey. The letter informs the RCC and ADSC directors that a survey packet, including a hard copy of the questionnaire and a link to the web survey, will be mailed to them in the next few weeks. In accordance with the Confidential Information Protection and Statistical Efficiency Act (CIPSEA), the letter shall inform the administrator of the authorizing legislation, purpose and content of the survey, as well as its voluntary nature and confidentiality provisions.

About two weeks after sending the advance letter, NCHS will send the first questionnaire mailing (Attachments B, C, G2, and G5) that contains:

• a cover letter signed by the Director of NCHS (separate versions for RCCs and ADSCs),

• FAQs on the back of the cover letter (separate versions for RCCs and ADSCs)

• a study brochure (ADSCs)

• an NCHS data brief (RCCs)

• an NCHS brochure about confidentiality on surveys

• a hard copy of the survey questionnaire (separate versions for RCCs and ADSCs)

• a business reply envelope to return the hardcopy survey, and

• a letter of support from national provider associations (separate versions for RCCs and ADSCs).

The cover letter in the first questionnaire mailing will be personalized with the name of the RCC and ADCS directors. The letter will inform the administrator of the purpose and content of the survey. In addition to explaining the confidentiality of the information provided and the voluntary nature of participation, the letter includes a reference to the legislative authority for the survey and an explanation of how the data will be used. This letter will emphasize that data collected about the RCC or ADSC and its residents/participants will never be linked to their names or other identifying features and that the web survey is administered over a safe and secure network. The cover letter will provide a toll-free number that survey participants/respondents can call with any questions. Anticipated topics include problems logging in and questions about the survey. This toll-free number will be directed to the Data Collection Task Leader and her staff.

The cover letter will also include the web survey URL and unique credentials for the director to access the web survey. Respondents can answer some questions, exit the questionnaire, and return later to the point where they stopped. The web questionnaire option will remain available until the end of the data collection period. The web link will also provide a pdf version of the questionnaire, so that respondents to the web survey can see all questions prior to completing the survey by web.

On the back of the cover letter are frequently asked questions (FAQ), designed to address what are expected to be the primary concerns of RCC and ADSC directors and staff.

The first mailing will include a hardcopy of the survey questionnaire with a pre-addressed, postage-paid return envelope. Attachments B and C contain a current list of questionnaire items. We estimate that it will take 30 minutes on average to answer the questionnaire.

The letters of support were obtained from associations that represent RCCs and ADSCs. We have sought and obtained support from the following organizations:

• RCC provider associations

  • LeadingAge

  • American Seniors Housing Association (ASHA)

  • Assisted Living Federation of America (ALFA)

  • Board and Care Quality Forum, and

  • National Center for Assisted Living (NCAL)

• ADSC provider associations

  • National Adult Day Services Association (NADSA).

One week after the first questionnaire mailing, NCHS will send a reminder/thank you letter to RCCs and ADSCs (Attachments G3 and G6). This reminder/thank you letter informs RCC and ADSC directors that a survey packet was sent recently, encouraging them to participate in the survey, and thanking those who have already responded.

After the reminder/thank you letter is sent, there will be two follow-up mailings sent in one week intervals to initial non-respondents that contain the same materials as the first questionnaire mailing but with a slightly modified cover letter.

About two months into the field period, NCHS will contact RCC and ADSC directors to administer the CATI survey when they are non-responders to the mail and web surveys. The CATI survey questionnaire is the same as the mail/web versions.

There will also be data retrieval calls to RCC and ADSC directors when after receiving the questionnaires there is missing information for critical staffing and resident/participant items, such as the number of full-time and part-time/ Full-Time Equivalent Registered Nurses and race/ethnicity distributions of users aggregated to the provider level (Attachment I). In terms of respondent burden, we have assumed that retrieval telephone calls will average 15 minutes and that no more than 10% of cases will require retrieval.

The hardcopy questionnaire will not have check boxes for “don’t know.” If the respondent has left any critical items blank, written “don’t know” for the item, or provided an answer that is inconsistent with another question, NCHS’ contractor will review the case data. If they can logically resolve the issue using decision rules approved by NCHS, they will edit the case data (saving the original version of the data) and retrieval will not be undertaken. However, if the contractor cannot resolve the issue using these decision rules, the case will be set for data retrieval and loaded into the CATI system. CATI interviewers will perform the retrieval for non-complex retrievals. As noted previously, complex retrievals will be handled by contractor project staff. Retrievers (interviewers and other contractor staff) will attempt to collect answers to all missing critical items. If the respondent indicated “refusal” on a critical item, no action will be taken.

For those who choose to complete the survey via web, every question will have a choice of “Do not know this information” and “Will look up this information and answer later.” A screen at the end of the web questionnaire will display all questions to which the code “will look up this information and answer later” has been input. The screen at the end will present respondents with hyperlinks to these questions so they can jump to the ones they wish in the order they wish. They will then have the option to either supply the information or choose a code indicating “will look up this information and answer later.” After a case on which critical items have the answer choices of “will look up this information and answer later” has remained in an uncompleted status for a period of time, CATI prompts will begin. CATI Interviewers will prompt respondents to complete their questionnaires. As a result of the prompting call:

• If the respondent supplies the answer to the CATI interviewer, the CATI interviewer will input it. This applies to both critical and noncritical items.

• If the respondent says that they do not know and cannot obtain this information, the interviewers will change the answer to “do not know this information.”

• If the respondent says they will return and complete the web questionnaire, the CATI interviewer will thank them and the case will remain pending.

Training interviewing staff is an important requirement for implementing the NSLTCP data collection effort. The general training covers standardized contacting and interviewing skills and educates interviewers on the concepts of data confidentiality and data security. This training is available as an interactive web-based self-learning program that interviewers access over the internet. It includes quizzes on each topic covered to assess the interviewers’ understanding of the information.

Prior to project-specific training, all interviewers will be provided an NSLTCP Telephone Interviewer Manual and will be required to complete a home study exercise using their manual. We have found that requiring the completion of the home study exercise before training helps to familiarize trainees with background information, project terminologies, and job expectations.

Project-specific training will be conducted over 2 days immediately before CATI production is scheduled to begin. Training will be conducted in a state-of-the-art training facility located in Raleigh, North Carolina.

Training materials will include a manual for telephone interviewers, a manual for Quality Control Supervisors, a training agenda, a training guide with PowerPoint presentation, FAQs for answering respondent questions, mock scenarios for averting/converting refusals, mock interview scripts for conducting the NSLTCP interview, mock scripts for conducting retrievals, and job aids to assist interviewers in their work.

At the end of training, interviewers must be certified for data collection by successfully completing a certification interview. Certification will be conducted by approved project personnel who will evaluate the interviewers’ mastery of the required skills and knowledge for NSLTCP. The certification process will consist of:

• a brief oral exam covering selected FAQs to ensure that Interviewers are able to answer the most frequently asked questions from survey participants/respondents;

• a full-length mock interview with another trainee under the observation of project staff to demonstrate knowledge of proper interviewing techniques;

• a mock interview with another trainee under the observation of project staff to demonstrate knowledge of retrieval and callback procedures; and

• a short exercise on selecting appropriate event (disposition) codes and working in CATI to demonstrate knowledge of how to work in the data collection systems.

The period of data collection will be determined based on time needed for expected completions and the availability of resources. Field staff will closely monitor RCCs/ADSCs that have not responded to the mail or web surveys. Secondly, there will be a very strong effort during the first wave of contacts, followed by persistent follow-up. During the CATI portion of the survey protocol, a maximum of six contacts (calls) will be made before a case will be considered as a noncontact or refusal. Each sampled case will receive the same field effort needed for contact and response. NCHS will receive weekly production reports from its contractor that will show the contact/response trends at the national and state levels and help to identify problem spots at as early as stage in the data collection process as is feasible.

After the data have been processed, post-data collection edit checks have been completed, disclosure risk assessment has been done, and weights have been developed, NCHS plans to create public-use data files for the RCC and ADSC versions of the survey. All data will be weighted to national estimates—and where feasible state estimates--using the inverses of selection probabilities (for states using a sampling design), and adjusting for non-response. Sampling errors are computed using the linearized Taylor series method of approximation as applied in the SUDAAN software package.

3. Methods to Maximize Response Rates and Deal with Nonresponse

NCHS will make every reasonable attempt to encourage completion of NSLTCP. To maximize response rates, NCHS will use methods similar to those used in previous establishment surveys (e.g., National Survey of Residential Care Facilities, National Home and Hospice Care Survey, National Nursing Home Survey). To this end, we will use the refusal aversion techniques described below.

Robust mailout materials. NSLTP’s questionnaire mailout packet conveys the legitimacy of the study and helps respondents understand the relevance and importance of the survey. The materials and FAQs are based on those used successfully in NSRCF (OMB No. 0920-0780).

Low burden. We estimate that it will take 30 minutes on average to answer the current list of questionnaire items (Attachments B and C).

Multimode approach. The hardcopy and web modes offer RCCs/ADSCs the flexibility to complete the survey at their convenience. Sessions can be stopped and restarted as needed.

Industry outreach. NCHS has contacted and will continue outreach efforts to national long-term care provider associations representing the RCC and the ADSC industries to inform them of the survey and ask for their support. NCHS is working with these organizations to share information about the study with their state member affiliates and publicize the study in newsletters.

Chains outreach. We will send an outreach package with a cover letter, signed by Dr. Sondik, to the corporate office of national and other large chains affiliated with one or more sampled RCCs and the five largest chains for ADSCs (Attachments H1 and H2). In the case of regional chains, we will send the outreach letter to the regional office. The letter briefly describes the study and explains that one or more members of their organizations will be contacted to participate. This letter also requests that they allow and encourage members of their organization to complete the survey if it is sent to them as well as provides a link where they can view a PDF of the questionnaire. This letter will also help CATI Interviewers to convert/avert refusals. The package also includes an NSLTCP folder, a study brochure (ADSC), an NCHS data brief (RCCs), an NCHS brochure about confidentiality on surveys, a hard copy of the survey questionnaire (separate versions for RCCs and ADSCs), and letters of support from national provider associations (separate versions for RCCs and ADSCs).

Directors of chain-affiliated providers often require the approval of the chain leadership to participate. The CATI Interviewer will explain that NCHS has a package of materials explaining the study that is designed for corporate offices / chain organizations such as the one with which they are affiliated. If the chain has been sent the package, the interviewer will supply the name and address of the person in the chain corporate office to whom the letter was sent. Regardless of whether the chain outreach package has previously been sent, the interviewer will then offer to e-mail this information to the person in their organization who needs to grant approval to this sample member. The e-mail will contain a message about how the director is seeking their approval to participate in the survey, and how RTI will call back the director/respondent after two weeks to see if approval was granted. Two weeks later, the interviewer will call back the RCC/ADSC to see if approval was granted. If the chain contact responds and tells the interviewer that they wish to speak with someone in more depth about the study, the interviewer will collect the appropriate contact information and convey this to their supervisor. The supervisor will forward the information to the Data Collection Task Leader; the Project Director of NCHS’ contractor will then call the chain, answer any specific questions they might have, and try to convince the chain to authorize participation.

CATI follow-up. Some sample members will not be persuaded to complete a survey using only mailing or web techniques. Telephone work will be required, and the CATI follow-up is therefore an important part of NSLTCP’s refusal aversion/conversion program. It is important that the CATI Interviewers be extremely professional, efficient, and convey to respondents the legitimacy and importance of the survey for U.S. LTC policy. Interviewers will be trained how to convey the importance of this work. We have developed a set of CATI FAQs (Attachment J) to help with refusal aversion/conversion.

Methods Experiments. In an effort to determine ways to further maximize response rates, NCHS is including three experiments in NSLTCP (Attachments K-M). NCHS aims for the results of the experiments to serve as a starting point for the development of an evidence base for fielding the survey component of NSLTCP in future years. In the attachments, we include only the materials that differ between the base protocol (Attachment G) and the three experiments. In the attached materials, we have highlighted text to show the differences made to the base protocol materials to adapt them for the experiment groups.

In the first experiment, the experimental group mailings for the first several contacts will present the study as a web survey only, and will include information only about how to complete the questionnaire by web (Attachments K1-K2). The option to complete the survey by mail and the hardcopy questionnaire will be omitted from the initial mailing, reminder/thank you letter, and the first follow-up mailing. However, a hardcopy questionnaire will be available upon request and will be included in the second follow-up mailing. For this first experiment, we hypothesize that--and will examine whether--the web treatment group will have a higher return rate and response rate by web compared with the control group. We also hypothesize that the web mode will also result in lower item nonresponse than the mail survey since respondents will be prompted to complete items that are left blank.

In the second experiment, NCHS will offer a provider-specific report to respondents comparing their RCC/ADSC to similar communities/centers in the aggregate (Attachments L1-L2). The letters to this experimental group will explain the content of the report, and when such a report would be available to them. In order to efficiently track the requests for the report, we will add a question to the web and CATI instruments asking whether they would like the report. This question will be posed only to this experimental group. Because we cannot add a question to the hardcopy instrument for just one experimental group, we will insert a loose sheet in the questionnaire packet with the same question added to the web and CATI instruments. Respondents will be instructed to return it with the hardcopy survey. For this second experiment, we hypothesize that--and will examine whether--the treatment group will have a higher return rate and response rate compared with the control group.

For the third and final experiment, we will add text to the reminder/thank you letter that goes out after the first questionnaire mailing, similar to the following: “If you have not completed the survey by mail or web by [insert date], we will call you to complete it by phone” (Attachments M1-M2). Since we speculate that many RCCs and ADSCs will prefer to self-complete by web or mail, it is possible that they will elect to complete the questionnaire via mail or web as soon as possible to avoid the phone call. For this third experiment, we hypothesize that--and will examine whether--the treatment group will have a higher percentage of completions by mail and web compared with the control group. Further, we hypothesize that--and will examine whether--the treatment group will have a higher return and response rate at the point in the schedule just before CATI calling begins compared with the control group

For each of the RCC and ADSC surveys, there are three experimental groups and a control group.  Based on the conservative goals of 80% power, 5% minimum detectable difference between each experimental group and the control group when the hypothesized rates are 50% and 55% (for return rate or response rate), and 5% significance level, the minimum sample size needed is 1,231 per each experimental group and control group.

The expected total universe size of 5,000 ADSCs is adequate to simultaneously run Experiments 1, 2 and 3 and have a single control group.  The 5,000 cases will be divided evenly so that 1,250 cases are randomly assigned to each of the three ADSC experimental treatment groups and the ADSC control group. We will use a serpentine sorting mechanism to sort all ADSC cases by state. Once the sample is sorted, we will assign numbers 1, 2, 3, 4 consecutively to the cases.

Although all ADSC cases are assumed to be eligible for NSLTCP, among RCCs we assume the same eligibility rate of 81% as occurred with the 2010 NSRCF.  Using the same conservative goals as noted above for ADSCs but taking into account the lower eligibility rate for RCCs, a minimum of 1,520 cases is needed to devote to each RCC experimental group and to the RCC control group to evaluate the difference in response rates between each experiment and the control group.  Among the RCCs, the expected total sample size of 11,701 is adequate to simultaneously run Experiments 1, 2 and 3 and have a single control group.  For Experiment 2, to minimize the effort entailed in producing and mailing reports to providers who request one as part of the provider-specific report experiment, NCHS seeks to offer the report to the minimum number of cases needed to allow for reliable results in the experiment.  Therefore, NCHS will not simply divide in four the 11,701 sample evenly amongst the three experimental groups and the control.  Instead, 1,520 cases will be randomly assigned to the provider-specific report experiment group.  Then, the remaining 10,181 (11,701 minus 1,520) cases will be randomly assigned amongst the remaining two experimental groups and the control group, so that each of these groups has 3,394 cases. For the control and experimental groups 1 and 3, we will use a serpentine sorting mechanism to sort all sample RCC cases by facility size strata and state. Serpentine sorting involves sorting first by facility size stratum in ascending order. Within each size stratum, the cases are sorted by state, with the direction of the sort (ascending or descending) changing each time the size stratum changes value. Once the sample is sorted, we will assign numbers 1, 2, 3, 4 consecutively to the cases. For experiment group 2, since it will require fewer cases than the full assigned sample of 1/4th, we will randomly select the desired number of cases from the cases assigned to that group.

Despite efforts to avert refusals, refusals can be expected. CATI staff will be trained so that if they encounter a potential refusal, they will listen to the concerns raised and attempt to address these concerns. When appropriate, CATI staff will provide a few weeks’ cooling off period before they contact RCCs and ADSCs again. CATI staff will provide detailed notes of these exchanges, and discuss the best course of action. In some cases, NCHS staff and/or senior staff at the NCHS contractor organization will be involved.

The overall target response rate for this mail/web/telephone survey is at least 65 percent. After the field period ends, NCHS will assess nonresponse bias by examining how much the respondents and nonrespondents differed on key relevant variables available from the respective RCC and ADSC frames. These include bed size (for RCCs only), Census geographic region (Northeast, Midwest, South, and West), and MSA status (metropolitan or nonmetropolitan). Because CATI is being used as a follow-up for non-respondents to the mail and web surveys, CATI responders may be considered as a late responder and possibly as non-responder had CATI not been included in the protocol. With that in mind, as another way to examine nonresponse bias, NCHS will also examine whether there are differences between mail/web respondents and CATI respondents in their factual survey responses.

4. Tests of Procedures or Methods to be Undertaken

The current lists of survey question items are in Attachments B (for RCCs) and C (for ADSCs). The majority of the NSLTCP survey question items have been (1) drawn from previously fielded NCHS LTC provider surveys (NNHS, NHHCS or NSRCF) and (2) evaluated in two main ways—cognitive interviewing and expert questionnaire appraisal. This section briefly describes these two evaluation approaches.

NCHS’ Questionnaire Design Research Laboratory (QDRL) conducted cognitive interviews with ADSC and RCC Directors in the DC metro area between November 2011 and January 2012 (OMB No. 0920-0222, exp. 03/31/2013). In total, QDRL staff conducted 13 interviews. Five respondents were given the RCC version of the questionnaire and eight were given the ADSC version of the questionnaire. Based on the cognitive interviewing, NCHS revised items on the questionnaires and applied findings to the study protocol.

NCHS’ contractor also conducted an expert review of the questionnaire items to suggest improvements to survey wording, format and design. That review systematically evaluated the survey’s clarity, specificity, question wording, format, response categories, potential sources of bias, and other possible complications. A survey methodologist reviewed the question items using RTI’s Questionnaire Appraisal System (QAS). The QAS is a coding system that evaluates survey questions, response options, instructions for problems that may affect respondent comprehension, recall, judgment, and response generation.

Based on the cognitive interviewing and the QAS review, NCHS revised selected NSRCF survey question items. Question item changes included the following types of revisions:

• Wording changes to address problems respondents had understanding the intent of the question or answering the question correctly, or difficulties encountered for other reasons;

• Re-ordering question items to improve question flow or questions being asked unnecessarily, given previous responses provided;

• Revising response categories to more closely align with expected analytic groupings and to address respondent queries about response options.

NCHS will conduct three experiments as part of the 2012 NSLTCP surveys to further test methods. Detailed explanation of these experiments is in Section B3.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The following government employee is responsible for oversight on the design and implementation of this collection:

NCHS

Lauren Harris-Kojetin, Ph.D.

Chief, Long-Term Care Statistics Branch

National Center for Health Statistics

3311 Toledo Road, Room 3431

Hyattsville, Maryland 20782

Phone: (301) 458-4369

Fax: (301) 458-4350

E-Mail: [email protected]

RTI International is NCHS’ contractor for the sampling design and the implementation of the survey data collection component for NSLTCP. The following RTI person oversees this contract:

Angela M. Greene, M.A.

Project Director

Program on Aging, Disability and Long-Term Care

RTI International

P.O. Box 2194

RTP, NC 27709

(919) 541-6675 (voice)

(919) 990-8454 (fax)

[email protected]

LIST OF ATTACHMENTS

Attachment A: NCHS Legislation – Section 306 of the Public Health Services Act (42 USC 242k)

Attachment B: RCC Questionnaire Items

Attachment C: ADSC Questionnaire Items

Attachment D: Federal Register Notice

Attachment E: Workgroup Participants

Attachment F: Human Subjects Determination

Attachment G: Base Protocol Respondent Contact Materials

G1 RCC NCHS Advance Letter

G2 RCC Questionnaire mailings

G3 RCC Reminder/Thank You Letter

G4 ADSC NCHS Advance Letter

G5 ADSC Questionnaire mailings

G6 ADSC Reminder/Thank You Letter

Attachment H: Chain Outreach Materials

H1 RCC Chain Outreach Package

H2 ADSC Chain Outreach Package

Attachment I: Data Retrieval Call

Attachment J: CATI FAQs

Attachment K: Experiment 1 Respondent Contact Materials

K1 RCC Experiment 1 materials

K2 ADSC Experiment 1 materials

Attachment L: Experiment 2 Respondent Contact Materials

L1 RCC Experiment 2 materials

L2 ADSC Experiment 2 materials

Attachment M: Experiment 3 Respondent Contact Materials

M1 RCC Experiment 3 materials

M2 ADSC Experiment 3 materials

2