Supporting Statement B - National Survey 0290

Supporting Statement B

National Survey of Organ Donation Attitudes and Practices

OMB Control No. 0915-0290

B. Collection of Information Employing Statistical Methods

1. Respondent Universe and Sampling Methods

HRSA is maintaining the same approach as in the 2019 National Survey. The National Survey of Organ Donation Attitudes and Practices (NSODAP) uses two survey modes with two distinct sampling frames. One is a telephone-based sample using address-based sampling (ABS) that includes both landlines and cell phones (ABS N = 2,000). The other mode is a stratified demographically balanced web-based survey panel (Web N = 8,000). For both modes, the respondent universe will consist of U.S. adults ages 18 years or older. The goal for this sample is to allow oversampling of minority groups, including African Americans, Asians, Hispanics, and Native Americans. This sample will provide sufficient statistical power to support drill down analyses of several target subgroups, representing age, gender, education level, income, and racial/ethnic groups.

The address-based sample will collect complete cases for 1,000 randomly selected residential addresses and 1,000 addresses selected for ZIP codes with a high prevalence of minority residents. Only primary residences are eligible for selection, excluding secondary residences, P.O. Boxes, and business addresses. Each sampled address is associated with a landline and/or cell phone. The sampling frame will be drawn by Survey SSI Research Now. Zip codes selected for oversampling will include a high prevalence of African American, Asian, Hispanic, or Native American residents.

The web panel will include 8,000 complete cases drawn from the SSI Research Now web panel. This sample will be stratified to include oversamples of racial/ethnic groups (African American, Asian, Hispanic, and Native American), education levels, and those over the age of 65. Because demographic information is already known for panelists, sub-groups of participants can be targeted precisely and directly. The use of a panel to target participant sub-groups is similar to the approach used for the 2012 survey, which used the Gallup Panel to oversample African Americans. In this case, we will use a larger total sample to target several demographically distinct participant groups.

Exhibit 1: Table of sample source cell counts and expected precision of estimates

Sub Group	Population Prevalence	ABS CATI EPSEM*	ABS CATI Oversample	CATI Total	Web Panel	Total Cases	95% CI
Total		1,000	1,000	2,000	8,000	10,000	+/-1.0%
White	61.3%	617	181	798	5,202	6,000	+/-1.3%
Black	13.3%	133	200	333	667	1,000	+/-3.1%
Hispanic	17.8%	179	178	357	643	1,000	+/-3.1%
Asian	5.7%	58	285	343	657	1,000	+/-3.1%
Nat. Amer.	1.3%	13	156	169	831	1,000	+/-3.1%
Age 65+	15.2%	152	152	304	1,216	1,520	+/-2.5%
*Computer Assisted Telephone Interview, Equal Probability Selection Method

Address-Based Sample Design

ABS is a technique that has been enabled by improved geographic information systems and extensive demographic data resources (AAPOR, 2016). ABS can provide survey samples that are nationally representative or enriched for efficient oversamples by ethnicity and similar stratifiers. We can enhance the ABS lists to provide household composition, telephone numbers, and other auxiliary data variables. We propose using an ABS split-sample frame for our Computer Assisted Telephone Interviewing (CATI). Half of this sample will be an equal probability of selection method (EPSEM), while the other half will be an ethnic oversample.

We will draw the ABS sampling frame from all residential addresses across the nation. This will exclude P.O. Boxes, business or organizational addresses, and vacant/seasonal residencies. The ABS sampling frame will include 10,000 addresses with appended phone numbers procured through SSI Research Now. Within this frame, 5,000 will be representative of the U.S. population (EPSEM). An oversample of an additional 5,000 will specifically target minorities as shown in Exhibit 1. We will target minority respondents by sampling ZIP codes with higher proportions of minority residents. All U.S. residences with an associated phone number, including both landline and cell phone, will be eligible.

Before any further processing of the ABS list, we will update all addresses through the National Change of Address database. After address updates, we will profile the new geographic distribution to ensure that any household shifts are randomized. Each ABS record contains auxiliary data variables including the ZIP code-specific percent of varying ethnic groups. Using expected ethnic prevalence in the population, we will draw on the pool of 10,000 records to organize an initial sample replicate (i.e., Replicate 01) of 5,000 records that is proportional to our target goals for each ethnic group. This initial replicate will receive a postal pre-notification mailing about one week before the initial CATI field start. During the initial days of the Replicate 01 field period, our sampling statisticians will closely monitor progress towards goals and ensure that the staff conducts a full calling protocol of ten call attempts on each sample record.

Once we select a household, we will randomly choose one adult from all adults living in the selected household using the “most recent birthday” method (O’Rourke, D. & Blair, J. (1983)). The “most recent birthday” method asks for the eligible person (18 years of age or older) within the sampled household who, at the time of respondent selection, has the most recent (last) birthday. The “most recent birthday” method represents a random selection of eligible household members. This method is considered to be much less intrusive than the purely random selection method or grid selection that requires the enumeration of all household members to make a respondent selection. This is the same method used in prior versions of the NSODAP survey.

Good survey practice requires that we execute a full calling protocol for each sample record, once released, so replicates should be no larger than necessary to achieve target goals. Our provisional field calendar provides for a total of three replicate releases, although this can be adjusted up or down, as actual field experience indicates. Should Replicate 01 be insufficient to produce the target quotas, we will assemble a Replicate 02 from the residual ABS list, and we will construct it with reference to observed response rates by ethnicity and by the observed shortfall in demographic quotas. In constructing this sample, we will append the basic sampling weight to each record to inform post-field weighting. Prior to constructing replicates beyond the first, we will observe the actual prevalence and adjust replicate construction based on field experience. Thus, we will converge to our target quotas while optimizing for efficiency and response rate.

The final calculation of weights will require a post-stratification analysis and an Iterative Proportional Fitting process. If necessary, we will repeat this process for Replicates 03 and 04, with continuous monitoring of field status. Each new replicate will have an associated pre-notification mailing so that potential respondents receive the mailing shortly before initial calls. Each replicate will receive approximately a 12-week field period to provide adequate time to execute the entire calling protocol. At the end of all replicate field cycles, we will have a final push to resolve all open calling dispositions and maximize the response rate.

Web Panel Sample Design

The Web panel survey will include 8,000 samples from SSI Research Now. SSI Research Now maintains the nation’s largest census-balanced representative web panel. The purpose of the large web panel is to allow sufficiently large numbers of respondents for key demographic groups, including all races/ethnicities, age groups, income levels, and education levels. SSI Research Now will contact panel members and perform the initial screening on their internal systems. Eligible respondents identified from the web panel receive an invitation to complete the NSODAP survey. The Voxco system, a commercially available web survey software system, will host this survey.

Precision of Estimates

The sampling plan calls for a target of 2,000 CATI cases and 8,000 web cases. These CATI cases are split between a sample of 1,000 EPSEM and 1,000 with a racial/ethnic oversample. The 8,000 web cases will expand the total sample size and all levels of key demographic variables. We expect 1,000 cases for each minority racial/ethnic group (Black, Hispanic, Asian, Native American), giving each group a 95% CI of +/- 3.1%. For those over 65, we expect 1,520 cases, yielding a 95% CI of +/- 2.5%. Using a 95% confidence interval, we expect the precision of estimates based on Exhibit 2 below. Our 95% CIs assume response proportions of 0.5, which is the most conservative estimate. As responses diverge from 0.5, CIs will become more precise.

Exhibit 2: Target Completed Cases and 95% CI Estimates

Grouping	Target Completed Cases	95% CI Estimate
Total (all modes)	10,000	+/-1.0%
CATI Total	2,000	+/-2.2%
CATI Oversample	1,000	+/-3.1%
CATI EPSEM	1,000	+/-3.1%
Web Panel Total	8,000	+/-1.1%
Race/Ethnicity
White	6,000	+/-1.3%
Black	1,000	+/-3.1%
Hispanic	1,000	+/-3.1%
Asian	1,000	+/-3.1%
Native American	1,000	+/-3.1%
Age Group
18-34	2,840	+/-1.8%
35-54	3,290	+/-1.7%
55-64	2,350	+/-2.0%
65 and over	1,520	+/-2.5%
Education*
High School	3,500	+/-1.7%
Some College	3,000	+/-1.8%
College Graduate	3,000	+/-1.8%

* Sum of education does not add to 10,000 because not all educational statuses are represented in the table.

Weighting of Sample Data

The complex sample design represented within this study will require a four-stage weighting design: 1) base weights; 2) propensity score adjusted non-response weights; 3) weight trimming, smoothing, and adjustment; and 4) final weights equal to the product of the base weights times the inverse of the propensity score, trimmed and redistributed where excessive and problematic weights are encountered.

1. Base weights. Base weights are the initial weights assigned to a given potential respondent in the sample. These weights are calculated as the inverse of the probability of selection for a given individual from within the population, by strata. Base weights essentially represent the number of people that a given person within the sample initially represents. Given a random draw of individuals, the sample population is representative of the population as a whole once we apply the weights with the base weights summing to strata and population totals.

2. Propensity Score adjusted Non-response weights. Although the base sample weight adjusts for varying probabilities of selection, all studies experience differential non-response across strata. To minimize potential bias in results, this differential response requires a post-field non-response weight to be calculated, to bring the final collected sample back to representing the original population. The design and analysis will be based on the generally accepted statistical practice of logistic regression to estimate propensity scores for respondents controlling for known factors among both the respondents and non-respondents. The propensity scores represent the probability of a given person to respond to the survey controlling for known socio-demographic characteristics.

The inverse of the propensity scores will be multiplied by the corresponding base weights to bring the respondents in line and be representative of the national population. The inclusion of propensity score adjusted weights results in reducing bias within survey results and analyses. We will employ logistic regression such as contained in Stata or SUDAAN’s WTADJUST procedure. A dichotomous dependent variable is created using respondents and non-respondents (1=responded, 0=non-response) and logistic regression is conducted using variable measures known for both respondents and non-respondents to assess which factors influence differential response rates.

3. Weight Trimming and Re-distribution. The application of propensity score adjusted non-response weights can lead to a misalignment of populations with some potentially excessive weights which skew the respondent population data. To control for this as well as to adjust the weights to ensure they best reflect the populations to which they are to measure, our statisticians will review propensity score adjusted weights to identify excessive outlier weights due to non-sufficient overlap between respondents and non-respondents, small cell size issues, or other factors. Boundary weight levels will be set. Weights exceeding boundary levels will be reset to boundary level with the difference (amount subtracted from the weight) being redistributed among the given strata or across strata cohorts represented by the observation(s) with excessive weights, as appropriate.

4. Final weights. Final weights for each respondent will be calculated as the product of Base weight * inverse of propensity score based non-response weight, trimmed and redistributed, effectively integrating each of the preceding three steps associated with weight generation. Once final weights are calculated and applied to the data, survey specific analytical techniques and methods must and will be applied. The survey specific techniques help minimize potential bias, account for within strata correlation, and reduce the likelihood of overstating the significance of results. The survey specific analysis techniques incorporate the complex survey design and weighting scheme contained within the NSODAP survey design.

Non-Response and Mode Analysis

HRSA uses survey based estimates for this study to minimize any potential bias that may be associated with a unit level non-response. For the ABS survey, high-level demographic information is often associated with the address. In all cases, demographic information is known about the ZIP code. Those demographic variables can be used for post-stratification weighting if they are not already included. In addition, the respondents to the ABS survey may be split into two groups: (i) early or ‘easy to reach’ and (ii) late or ‘difficult to reach’ respondents. The total number of calls required to complete an interview will be used to identify these groups. These two groups will be compared based on their responses to selected survey questions. This comparison will also be based on the assumption that the latter group may in some ways resemble the population of non-respondents. The goal of the analysis plan is to assess the nature of the non-response pattern in the survey.

Similar to the non-response analysis, a cross-mode analysis will be used to identify potential differences between the ABS phone and web surveys. In these cases, scores for key survey outcomes are compared across modes by every key demographic variable, including race/ethnicity, gender, age group, income level, and education level. In any cases where a significant difference is found, the difference will be noted. This will allow the calculation of a “corrected” score to make phone and web samples equivalent, preserving comparability with prior versions of the survey.

2. Procedures for the Collection of Information

The mode of data collection will be telephone based on a CATI system and web panel. The plan is to conduct phone interviewing during weekday evenings and on weekends to increase the likelihood of finding respondents at home. A 5 plus 5 call design (up to five calls to establish human contact and then a maximum of five calls to complete the interview with the selected respondent) will be employed. The plan is to call back respondents who decline to be interviewed (“soft refusals”) on a different day and ask again for their participation. HRSA will conduct the interviews in English and Spanish.

3. Methods to Maximize Response Rates

To maximize the response rate to the survey, the data collection methodology includes the following:

• Calling up to 10 times to reach a household and complete the interview.

• Calling at alternate times of the day and on weekends to reach all respondents.

• Having a carefully designed introduction and promise of confidentiality to increase trust and salience.

• Having a questionnaire designed to increase completion and minimize item non-response.

3. Tests of Procedures or Methods to be Undertaken

The next iteration of the national survey will be in line with the 2019 survey. All the items that will be included in the next survey were included in the 2019 survey.

3. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

HRSA conducted the previous survey through a contract with the American Directions Research Group (ADRG). ADRG and their subcontractor Altarum Institute have expertise in developing sampling designs for telephone surveys and web panels, including the kinds of minority and ethnic group oversamples required by the current study. American Directions and Altarum performed sample selection, data collection, analysis of the results, and writing of the report for public distribution.

Table 1 below provides the names, telephone numbers, and email addresses of personnel who developed the statistical aspects of the design finalized the data collection. It also includes the name, telephone number, and email address of the HRSA contact for this data collection.

Table 1: Contact Information for Survey Personnel

Name	Agency/Company/Organization	Telephone	Email
Adriana Martinez	HRSA/DoT	301-443-9469	[email protected]
Dr. Chris Duke	Altarum	734-302-4642	[email protected]
Tom Wilkinson	Altarum	734-302-5692	[email protected]
Jerry Karson	ADRG	202-596-7966	[email protected]

References

AAPOR (1016). Address Based Sampling. [https://www.aapor.org/Education-Resources/Reports/Address-based-Sampling.aspx]

O’Rourke, D. & Blair, J. (1983): Improving Random Respondent Selection in Telephone Surveys. Journal of Marketing Research, 20, 428-432.