20180501 National Survey Supporting Statement B - CLEAN

Supporting Statement B

National Survey of Organ Donation Attitudes and Practices

OMB Control No. 0915-0290 Reinstatement

B. Collection of Information Employing Statistical Methods

1. Respondent Universe and Sampling Methods

The 2019 National Survey of Organ Donation Attitudes and Practices (NSODAP) will use two survey modes with two distinct sampling frames. One is a telephone-based sample using address-based sampling (ABS) that includes both landlines and cell phones (ABS N = 2,000). The other mode is a stratified web-based survey using Survey Sampling International (SSI) Research Now’s demographically balanced web survey panel (Web N = 8,000). For both modes, the respondent universe will consist of U.S. adults ages 18 years or older. The goal of this sample is to allow oversampling of minority groups, including African American, Asians, Hispanics, and Native Americans. This sample will provide sufficient statistical power to support drill down analyses of several target subgroups, representing age, gender, education level, income, and racial/ethnic groups.

The address-based sample will collect complete cases for 1,000 randomly selected residential addresses and 1,000 addresses selected for ZIP codes with a high prevalence of minority residents. Only primary residences are eligible for selection, excluding secondary residences, P.O. Boxes, and business addresses. Each sampled address is associated with a landline and/or cell phone. The sampling frame will be drawn by Survey SSI Research Now. Zip codes selected for oversampling, will include a high prevalence of African American, Asian, Hispanic, or Native American residents.

The web panel will include 8,000 complete cases drawn from the SSI Research Now web panel. This sample will be stratified to include oversamples of racial/ethnic groups (African American, Asian, Hispanic, and Native American), education levels, and those over the age of 65. Because demographic information is already known for panelists, sub-groups of participants can be targeted precisely and directly. The use of a panel to target participant sub-groups is similar to the approach used for the 2012 survey, which used the Gallup Panel to oversample African Americans. In this case, we will use a larger total sample to target several demographically distinct participant groups.

Exhibit 1: Table of sample source cell counts and expected precision of estimates

Sub Group	Population Prevalence	ABS CATI EPSEM*	ABS CATI Oversample	CATI Total	Web Panel	Total Cases	95% CI
Total		1,000	1,000	2,000	8,000	10,000	+/-1.0%
White	61.3%	617	181	798	5,202	6,000	+/-1.3%
Black	13.3%	133	200	333	667	1,000	+/-3.1%
Hispanic	17.8%	179	178	357	643	1,000	+/-3.1%
Asian	5.7%	58	285	343	657	1,000	+/-3.1%
Nat. Amer.	1.3%	13	156	169	831	1,000	+/-3.1%
Age 65+	15.2%	152	152	304	1,216	1,520	+/-2.5%
*Computer Assisted Telephone Interview, Equal Probability Selection Method

Address-Based Sample Design

ABS is a relatively new technique that has been enabled by improved geographic information systems and extensive demographic data resources (AAPOR, 2016). ABS can provide survey samples that are nationally representative or enriched for efficient oversamples by ethnicity and similar stratifiers. We can enhance the ABS lists to provide household composition, telephone numbers, and other auxiliary data variables. We propose using an ABS split-sample frame for our Computer Assisted Telephone Interviewing (CATI). Half of this sample will be an equal probability of selection method (EPSEM), while the other half will be an ethnic oversample.

We will draw the ABS sampling frame from all residential addresses across the nation. This will exclude P.O. Boxes, business or organizational addresses, and vacant/seasonal residencies. The ABS sampling frame will include 10,000 addresses with appended phone numbers procured through SSI Research Now. Within this frame, 5,000 will be representative of the U.S. population (EPSEM). An oversample of an additional 5,000 will specifically target minorities as shown in Exhibit 1. We will target minority respondents by sampling ZIP codes with higher proportions of minority residents. All U.S. residences with an associated phone number, including both landline and cell phone, will be eligible.

Before any further processing of the ABS list, we will update all addresses through the National Change of Address database. After address updates, we will profile the new geographic distribution to ensure that any household shifts are randomized. Each ABS record contains auxiliary data variables including the ZIP code-specific percent of varying ethnic groups. Using expected ethnic prevalence in the population, we will draw on the pool of 10,000 records to organize an initial sample replicate (i.e., Replicate 01) of 5,000 records that is proportional to our target goals for each ethnic group. This initial replicate will receive a postal pre-notification mailing about one week before the initial CATI field start. During the initial days of the Replicate 01 field period, our sampling statisticians will closely monitor progress towards goals and ensure that the staff conducts full calling protocol of ten call attempts on each sample record.

Once we select a household, we will randomly choose one adult from all adults living in the selected household using the “most recent birthday” method (O’Rourke, D. & Blair, J. (1983)). The “most recent birthday” method asks for the eligible person (18 years of age or older) within the sampled household who, at the time of respondent selection, has the most recent (last) birthday. The “most recent birthday” method represents a random selection of eligible household members. This method is considered to be much less intrusive than the purely random selection method or grid selection that requires enumeration of all household members to make a respondent selection. This is the same method used in prior versions of the NSODAP survey.

Good survey practice requires that we execute full calling protocol for each sample record, once released, so replicates should be no larger than necessary to achieve target goals. Our provisional field calendar provides for a total of three replicate releases, although this can be adjusted up or down, as actual field experience indicates. Should Replicate 01 be insufficient to produce the target quotas, we will assemble a Replicate 02 from the residual ABS list, and we will construct it by referring to observed response rates by ethnicity and by the observed shortfall in demographic quotas. In constructing this sample, we will append the basic sampling weight to each record to inform post-field weighting. Before constructing replicates beyond the first, we will observe the actual prevalence and adjust replicate construction based on field experience. Thus, we will converge to our target quotas while optimizing for efficiency and response rate.

The final calculation of weights will require a post-stratification analysis and an Iterative Proportional Fitting process. If necessary, we will repeat this process for Replicates 03 and 04, with continuous monitoring of field status. Each new replicate will have an associated pre-notification mailing so that potential respondents receive the mailing shortly before initial calls. Each replicate will receive approximately a 12-week field period to provide adequate time to execute the entire calling protocol. At the end of all replicate field cycles, we will have a final push to resolve all open calling dispositions and maximize response rate.

Web Panel Sample Design

We will supplement the ABS CATI sample with 8,000 web panel completes from SSI Research Now. SSI Research Now maintains the nation’s largest census-balanced representative web panel. The purpose of the large web panel is to allow sufficiently large numbers of respondents for key demographic groups, including all races/ethnicities, age groups, income levels, and education levels. SSI Research Now will contact panel members and perform the initial screening on their internal systems. Eligible respondents identified from the web panel receive an invitation to complete the NSODAP survey. The Voxco system, a commercially-available web survey software system, will host this survey.

Web data collection will commence toward the end of the telephone field period. We will use the demographic results from the ABS CATI survey to calculate the exact demographic construction needed for the web sample to reach the target counts for each demographic group. During web data collection, we will monitor sample quotas almost in real-time to ensure that we converge to our target number of cases by strata. This monitoring will allow us to reach our target demographic groups at every level, allowing detailed subgroup and crosstab analyses. We will conduct a thorough mode analysis, described below, to identify any differences between the ABS CATI and web panel responses. If we find any significant differences, a mode correction can be applied to the web panel results to make the CATI and web results comparable.

Precision of Estimates

Our sampling plan calls for a target of 2,000 CATI cases and 8,000 web cases. These CATI cases are split between a sample of 1,000 EPSEM and 1,000 with a racial/ethnic oversample. The 8,000 web cases will expand the total sample size and all levels of key demographic variables. We expect 1,000 cases for each minority racial/ethnic group (Black, Hispanic, Asian, Native American), giving each group a 95% CI of +/- 3.1%. For those over 65, we expect 1,520 cases, yielding a 95% CI of +/- 2.5%. Using a 95% confidence interval, we expect precision of estimates based on Exhibit 2 below. Our 95% CIs assume response proportions of 0.5, which is the most conservative estimate. As responses diverge from 0.5, CIs will become more precise.

Exhibit 2: Target Completed Cases and 95% CI Estimates

Grouping	Target Completed Cases	95% CI Estimate
Total (all modes)	10,000	+/-1.0%
CATI Total	2,000	+/-2.2%
CATI Oversample	1,000	+/-3.1%
CATI EPSEM	1,000	+/-3.1%
Web Panel Total	8,000	+/-1.1%
Race/Ethnicity
White	6,000	+/-1.3%
Black	1,000	+/-3.1%
Hispanic	1,000	+/-3.1%
Asian	1,000	+/-3.1%
Native American	1,000	+/-3.1%
Age Group
18-34	2,840	+/-1.8%
35-54	3,290	+/-1.7%
55-64	2,350	+/-2.0%
65 and over	1,520	+/-2.5%
Education*
High School	3,500	+/-1.7%
Some College	3,000	+/-1.8%
College Graduate	3,000	+/-1.8%

* Sum of education does not add to 10,000 because not all educational statuses are represented in the table.

Weighting of Sample Data

The complex sample design represented within this study will require a four-stage weighting design: 1) base weights; 2) propensity score-adjusted non-response weights; 3) weight trimming, smoothing, and adjustment; and 4) final weights equal to the product of the base weights times the inverse of the propensity score, trimmed and redistributed where excessive and problematic weights are encountered.

1. Base weights. Base weights are the initial weights assigned to a given potential respondent in the sample. These weights are calculated as the inverse of the probability of selection for a given individual from within the population, by strata. Base weights essentially represent the number of people that a given person within the sample initially represents. Given a random draw of individuals, the sample population is representative of the population as a whole once we apply the weights with the base weights summing to strata and population totals.

2. Propensity Score adjusted Non-response weights. Although the base sample weight adjusts for varying probabilities of selection, all studies experience differential non-response across strata. To minimize potential bias in results, this differential response requires a post-field non-response weight to be calculated, to bring the final collected sample back to representing the original population. The design and analysis will be based on the generally accepted statistical practice of logistic regression to estimate propensity scores for respondents controlling for known factors among both the respondents and non-respondents. The propensity scores represent the probability of a given person to respond to the survey controlling for known socio-demographic characteristics.

The inverse of the propensity scores will be multiplied by the corresponding base weights to bring the respondents in line and be representative of the national population. The inclusion of propensity score-adjusted weights results in reducing bias within survey results and analyses. We will employ logistic regression such as contained in Stata or SUDAAN’s WTADJUST procedure. A dichotomous dependent variable is created using respondents and non-respondents (1=responded, 0=non-response) and logistic regression is conducted using variable measures known for both respondents and non-respondents to assess which factors influence differential response rates.

3. Weight Trimming and Re-distribution. The application of propensity score-adjusted non-response weights can lead to a misalignment of populations with some potentially excessive weights which skew the respondent population data. To control for this as well as to adjust the weights to ensure they best reflect the populations to which they are to measure, our statisticians will review propensity score-adjusted weights to identify excessive outlier weights due to non-sufficient overlap between respondents and non-respondents, small cell size issues, or other factors. Boundary weight levels will be set. Weights exceeding boundary levels will be reset to boundary level with the difference (amount subtracted from weight) being re-distributed among the given strata or across strata cohorts represented by the observation(s) with excessive weights, as appropriate.

4. Final weights. Final weights for each respondent will be calculated as the product of Base weight * inverse of propensity score-based non-response weight, trimmed and redistributed, effectively integrating each of the preceding three steps associated with weight generation. Once final weights are calculated and applied to the data, survey specific analytical techniques and methods must and will be applied. The survey specific techniques help minimize potential bias, account for within strata correlation, and reduce the likelihood of overstating the significance of results. The survey specific analysis techniques incorporate the complex survey design and weighting scheme contained within the NSODAP survey design.

Non-Response and Mode Analysis

Survey-based estimates for this study will be weighted to minimize any potential bias that may be associated with unit level non-response. For the ABS survey, high-level demographic information is often associated with the address. In all cases, demographic information is known about the ZIP code. We will use this data to compare if certain demographic groups appear to be under or over represented in the sample of respondents. Those demographic variables can be used for post-stratification weighting if they are not already included. Also, the respondents to the ABS survey may be split into two groups: (i) early or ‘easy to reach’ and (ii) late or ‘difficult to reach’ respondents. The total number of calls required to complete an interview will be used to identify these groups. These two groups will be compared based on their responses to selected survey questions. This comparison will also be based on the assumption that the latter group may in some ways resemble the population of non-respondents. The goal of the analysis plan is to assess the nature of the non-response pattern in the survey.

Similar to the non-response analysis, we will conduct a cross-mode analysis to identify potential differences between the ABS phone and web surveys. In these cases, scores for key survey outcomes will be compared across mode by every key demographic variable, including race/ethnicity, gender, age group, income level, and education level. In any cases where a significant difference is found, the difference will be noted. This will allow the calculation of a “corrected” score to make phone and web samples equivalent, preserving comparability with prior versions of the survey.

2. Procedures for the Collection of Information

The mode of data collection will be telephone based on a CATI system and web panel. We will conduct phone interviewing w during weekday evenings and on weekends to increase the likelihood of finding respondents at home. A 5 plus five call design (up to five calls to establish human contact and then a maximum of five calls to complete the interview with the selected respondent) will be employed. We will call back respondents who decline to be interviewed (“soft refusals”) on a different day, and we will ask again for their participation. We will conduct the interviews in English and Spanish.

As mentioned earlier, the telephone sample for this study will include both landline and cell phones. We will interview respondents reached on cell phones (landlines) regardless of whether they also have access to landline (cell phones). For both landline and cell phones, the geographic location of the respondent will be determined based on respondents’ self-reported response to a question on location (like ‘what is your ZIP-code?’).

3. Methods to Maximize Response Rates

Our proposed study uses an improved method of telephone outreach (address-based sampling) compared to the prior survey, which used random digit dialing. This change was made in part to improve response rates and representativeness. However, because of this change, an exact response rate is unknown. However, we are committed to using the best possible survey practices to maximize response rates. To do this, HRSA will utilize a comprehensive plan that focuses on (1) having a call design that will ensure call attempts are made at different times of day and different days of the week to maximize contact rates, (2) conducting an extensive interviewer briefing prior to the field period that educates them about the content of the survey as well as how to handle reluctance and refusals, (3) having strong supervision that will ensure that high quality data are collected throughout the field period, (4) utilizing troubleshooting teams to attack specific data collection problems that may occur during the field period, and (5) customizing refusal aversion and conversion techniques.

To maximize the response rate to the survey, the data collection methodology will include the following:

• Calling up to 10 times to reach a household and complete the interview.

• Calling at alternate times of the day and on weekends to reach all respondents.

• Having a carefully designed introduction and the promise of confidentiality to increase trust and salience.

• Having a questionnaire designed to increase completion and minimize item non-response.

3. Tests of Procedures or Methods to be Undertaken

The research planned for 2019 aims to extend and expand upon prior surveys on organ donation especially the one conducted for HRSA in 2005 and 2012. A majority of the items (more than 80 percent) included in the current survey were included in the 2012 survey. In some cases, we made minor revisions. Also, initial cognitive tests of 7 respondents were conducted using the current questionnaire for flow and clarity. No separate pilot study was therefore considered necessary.

3. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

HRSA will conduct this survey through a contract with American Directions Research Group (ADRG). ADRG and their subcontractor Altarum Institute have expertise in developing sampling designs for telephone surveys and web panels, including the kinds of minority and ethnic group oversamples required by the current study. American Directions and Altarum will perform sample selection, data collection, analysis of the results, and writing of the report for public distribution.

Table 1 below provides the names, telephone numbers, and email addresses of personnel who developed the statistical aspects of the design, finalized the data collection plan, and will be primarily responsible for collecting, analyzing, and reporting on the data. It also includes the name, telephone number and email address of the HRSA Contracting Officer Representative who will provide oversight for this project.

Table 1: Contact Information for Survey Personnel

Name	Agency/Company/Organization	Telephone	Email
Dr. S. Janet Kuramoto-Crawford	HRSA/DoT	301-443-2248	[email protected]
Dr. Chris Duke	Altarum	734-302-4642	[email protected]
Tom Wilkinson	Altarum	734-302-5692	[email protected]
Jerry Karson	ADRG	202-596-7966	[email protected]

References

AAPOR (1016). Address-Based Sampling. [https://www.aapor.org/Education-Resources/Reports/Address-based-Sampling.aspx]

O’Rourke, D. & Blair, J. (1983): Improving Random Respondent Selection in Telephone Surveys. Journal of Marketing Research, 20, 428-432.

9