04222024 Supporting Statement B - 032020245002 - REDLINE

Supporting Statement B

National Survey of Organ Donation Attitudes and Practices (NSODAP)

OMB Control No. 0915-0290

1. Respondent Universe and Sampling Methods

The 2024 National Survey of Organ Donation Attitudes and Practices (NSODAP) will be statistically and methodologically identical to the 2019 OMB-approved survey. The 2024 NSODAP will include an Address Based Sampling (ABS) calling and mailing list, supplemented by a census-balanced web survey. The goal will be to collect 10,000 completed surveys with 2,000 from ABS and 8,000 from the web panel. Eligible participants will be 18 years of age and over and will be geographically representative of the United States. Additionally, the survey will oversample subgroups over 50 years of age, Black or African Americans, American Indian/Alaskan Natives, Hispanic/Latino Americans, and Asian Americans to support detailed analyses. HRSA has been oversampling these subgroups since the 2005 national survey. The oversampling of these subgroups enabled for the continued tracking of their attitudes about organ donation over time and allowed for the assessment of the impact our targeting of donation outreach messages and strategies. ABS can provide survey samples that are nationally representative or enriched for efficient oversamples by ethnicity, income, or other demographics. As shown in previous surveys, attitudes and practices regarding organ donation are heavily influenced by factors such as race/ethnicity and age. The survey will collect race/ethnicity data that is in-line with the SPD-15; however, the oversampling will be for only the subgroups listed as they are required by contract and were done in the prior surveys. Changing the sampling plan now is not feasible given the contractual updates that would be needed. Additionally, the survey could not be fielded in time if changes to the sampling plan were pursued.

The overall sample design for 2024 NSODAP includes collecting sufficient cases, within strata defined primarily by ethnicity, to support a statistically defensible analysis of these population subgroups with comparability to the same sub-groups measured in 2019 NSODAP and prior years. Each ABS record contains auxiliary data variables including the zip code-specific percent of varying ethnic groups, household composition, telephone numbers, and other demographic data (such as age), allowing us to plan and execute a stratified sample to achieve our strata-level targets. The sampling plan for completes by demographics for the 2024 NSODAP is shown below in Exhibit 1.

Exhibit 1. Sampling Plan

Demographic Group	Population Prevalence*	ABS CATI: EPSEM	ABS CATI: Oversample	CATI Total	Web Panel	Total cases
Nominal N		1,000	1,000	2,000	8,000	10,000
Caucasian	59.3	593	250	843	4,500	5,343
Black African or African American	13.6	136	250	386	1,000	1,386
Hispanic/Latino Americans	18.9	189	200	389	1,000	1,386
Asian Americans	6.1	61	150	211	1,000	1,211
American Indian/Alaskan Natives	1.3	13	150	163	500	663
Age 65+	16.8	168	168	336	1,344	1,680
*https://www.census.gov/quickfacts/fact/table/US/PST045216

Address-Based Sample Design

The same as in 2019, the ABS will collect complete cases for 1,000 randomly selected residential addresses and 1,000 addresses selected for ZIP codes with a high prevalence of minority residents. Only primary residences are eligible for selection, excluding secondary residences, P.O. Boxes, and business addresses. Each sampled address is associated with a landline and/or cell phone. The sampling frame will be drawn by Dynata. Zip codes selected for oversampling will include a high prevalence of Black or African American, Asian American, Hispanic/Latino American, or American Indian/Alaskan Native residents.

The sample of phone numbers will be organized into replicates reflecting the appropriate mix of national probability sample and oversample. We will be able to release replicates slowly over the fielding period, thus allowing the team to converge to the targeted goals efficiently while reducing respondent burden. Before constructing replicates beyond the first, we will observe the actual prevalence and adjust replicate construction based on field experience. Thus, we will converge to our target quotas while optimizing for efficiency and response rate. In constructing this sample, we will append the basic sampling weight to each record to inform post-field weighting. The final calculation of weights will require a post-stratification analysis and an Iterative Proportional Fitting process.

Web Panel Sample Design

The same as in 2019, we will supplement the ABS Computer-Assisted Telephone Interviewing (CATI) sample with 8,000 web panel completes from Dynata. Dynata maintains the nation’s largest census-balanced representative web panel that was used for the 2019 NSODAP. The purpose of the large web panel is to allow sufficiently large numbers of respondents for key demographic groups, including all races/ethnicities, age groups, income levels, and education levels. Dynata has established highly curated panels that are census-balanced and, through rigorous calibration to national demographics, very representative of the general population. Each panelist comes with a data record including a unique panelist identification and demographics, such as age, sex, race/ethnicity, household income, and zip code. This ensures incoming panelists are representative of the broader population and ensures no duplicate completions.

Web data collection will commence toward the end of the telephone field period. We will use the demographic results from the ABS CATI survey to calculate the exact demographic construction needed for the web sample to reach the target counts for each demographic group. During web data collection, we will monitor sample quotas almost in real time to ensure that we converge to our target number of cases by strata. This method will allow us to reach our target demographic groups at every level, enabling detailed subgroup and crosstab analyses. We will conduct a thorough mode analysis, described below, to identify any differences between the ABS CATI and web panel responses. If we find any significant differences, a mode correction can be applied to the web panel results to make the CATI and web results comparable.

Precision of Estimates

Our sampling plan calls for a target of 2,000 CATI cases and 8,000 web cases. These CATI cases are split between a sample of 1,000 Equal Probability Selection Method (EPSEM) and 1,000 with a racial/ethnic oversample. The 8,000 web cases will expand the total sample size and all levels of key demographic variables. We expect 1,000 cases for each minority racial/ethnic group, giving each group a 95 percent confidence interval or CI of +/- 3.1 percent. For those over 65, we expect 1,520 cases, yielding a 95 percent CI of +/- 2. percent. Using a 95 percent CI, we expect the precision of estimates based on Exhibit 2 below. Our 95 percent CIs assume response proportions of 0.5, which is the most conservative estimate. As responses diverge from 0.5, CIs will become more precise.

Exhibit 2: Target Completed Cases and 95 Percent CI Estimates (margin of error)

Grouping	Target Completed Cases	95% CI Estimate (margin of error)
Total (all modes)	10,000	+/-1.0%
CATI Total	2,000	+/-2.2%
CATI Oversample	1,000	+/-3.1%
CATI EPSEM	1,000	+/-3.1%
Web Panel Total	8,000	+/-1.1%
Race/Ethnicity
White	6,000	+/-1.3%
Black	1,000	+/-3.1%
Hispanic	1,000	+/-3.1%
Asian	1,000	+/-3.1%
Native American	1,000	+/-3.1%
Age Group
18-34	2,840	+/-1.8%
35-54	3,290	+/-1.7%
55-64	2,350	+/-2.0%
65 and over	1,520	+/-2.5%
Education*
High School	3,500	+/-1.7%
Some College	3,000	+/-1.8%
College Graduate	3,000	+/-1.8%

* Sum of education does not add up to 10,000 because not all educational statuses are represented in the table.

Weighting of Sample Data

The complex sample design represented within this survey will require a four-stage weighting design: 1) base weights; 2) propensity score-adjusted non-response weights; 3) weight trimming, smoothing, and adjustment; and 4) final weights equal to the product of the base weights times the inverse of the propensity score, trimmed and redistributed where excessive and problematic weights are encountered. This same method was used in 2019.

1. Base weights. Base weights are the initial weights assigned to a given potential respondent in the sample. These weights are calculated as the inverse of the probability of selection for a given individual from within the population, by strata. Base weights represent the number of people a given person within the sample initially represents. Given a random draw of individuals, the sample population is representative of the population as a whole once we apply the weights with the base weights summing to strata and population totals.

2. Propensity Score adjusted non-response weights. Although the base sample weight adjusts for varying probabilities of selection, all studies experience differential non-response across strata. To minimize potential bias in results, this differential response requires a post-field non-response weight to be calculated, to bring the final collected sample back to representing the original population. The design and analysis will be based on the generally accepted statistical practice of logistic regression to estimate respondents' propensity scores, controlling for known factors among the respondents and non-respondents. The propensity scores represent the probability of a given person to respond to the survey controlling for known socio-demographic characteristics.

The inverse of the propensity scores will be multiplied by the corresponding base weights to bring the respondents in line and be representative of the national population. The inclusion of propensity score-adjusted weights results in reducing bias within survey results and analyses. We will employ logistic regression such as in Stata or SUDAAN’s WTADJUST procedure, which creates survey weights. A dichotomous dependent variable is created using respondents and non-respondents (1=responded, 0=non-response) and logistic regression is conducted using variable measures known for both respondents and non-respondents to assess which factors influence differential response rates.

3. Weight Trimming and Re-distribution. The application of propensity score-adjusted non-response weights can lead to a misalignment of populations with some potentially excessive weights which skew the respondent population data. To control for this as well as to adjust the weights to ensure they best reflect the populations to which they are to measure, our statisticians will review propensity score-adjusted weights to identify excessive outlier weights due to non-sufficient overlap between respondents and non-respondents, small cell size issues, or other factors. Boundary weight levels will be set. Weights exceeding boundary levels will be reset to boundary level with the difference (amount subtracted from weight) being re-distributed among the given strata or across strata cohorts represented by the observation(s) with excessive weights, as appropriate.

4. Final weights. Final weights for each respondent will be calculated as the product of Base weight * inverse of propensity score-based non-response weight, trimmed and redistributed, effectively integrating each of the preceding three steps associated with weight generation. Once final weights are calculated and applied to the data, survey-specific analytical techniques and methods must and will be applied. The survey-specific techniques help minimize potential bias, account for within strata correlation, and reduce the likelihood of overstating the significance of results. The survey-specific analysis techniques incorporate the complex survey design and weighting scheme contained within the NSODAP survey design.

Non-Response and Mode Analysis

Survey-based estimates for the 2024 NSODAP will be weighted to minimize any potential bias that may be associated with unit-level non-response. For the ABS survey, high-level demographic information is often associated with the address. In all cases, demographic information is known about the ZIP code. We will use this data to compare if certain demographic groups appear to be under or over-represented in the sample of respondents. Those demographic variables can be used for post-stratification weighting if they are not already included. Also, the respondents to the ABS survey may be split into two groups: (i) early or ‘easy to reach’ and (ii) late or ‘difficult to reach’ respondents. The total number of calls required to complete an interview will be used to identify these groups. These two groups will be compared based on their responses to selected survey questions. This comparison will also assume that the latter group may, in some ways, resemble the population of non-respondents. The goal of the analysis plan is to assess the nature of the non-response pattern in the survey.

Similar to the non-response analysis, we will conduct a cross-mode analysis to identify potential differences between the ABS phone and web surveys. In these cases, scores for key survey outcomes will be compared across modes by every key demographic variable, including race/ethnicity, gender, age group, income level, and education level. In any cases where a significant difference is found, the difference will be noted. This will allow the calculation of a “corrected” score to make phone and web samples equivalent, preserving comparability with prior versions of the survey.

2. Procedures for the Collection of Information

The mode of data collection will be a web panel and telephone based on a CATI system. Dynata will contact panel members and perform the initial screening on their internal systems. Eligible respondents identified from the web panel receive an invitation to complete the NSODAP survey. Phone interviewing will be conducted during weekday evenings and on weekends to increase the likelihood of finding respondents at home. A five plus five call design (up to five calls to establish human contact and then a maximum of five calls to complete the interview with the selected respondent) will be employed. We will call back respondents who decline to be interviewed (“soft refusals”) on a different day, and we will ask again for their participation. We will conduct the interviews in English and Spanish.

As mentioned earlier, the telephone sample for this survey will include both landline and cell phones. We will interview respondents reached on cell phones (landlines) regardless of whether they also have access to landlines (cell phones). For both landline and cell phones, the geographic location of the respondent will be determined based on respondents’ self-reported response to a question on location (like ‘What is your ZIP-code?’).

3. Methods to Maximize Response Rates and Deal with Nonresponse

HRSA will utilize a comprehensive plan to maximize telephone survey response rates that focuses on (1) having a call design that will ensure call attempts are made at different times of day and different days of the week to maximize contact rates, (2) conducting an extensive interviewer briefing before the field period that educates them about the content of the survey as well as how to handle reluctance and refusals, (3) having strong supervision that will ensure that high-quality data are collected throughout the field period, (4) utilizing troubleshooting teams to attack specific data collection problems that may occur during the field period, and (5) customizing refusal aversion and conversion techniques.

To maximize the response rate to the survey, the data collection methodology will include the following:

• Calling up to ten times to reach a household and complete the interview.

• Calling at alternate times of the day and on weekends to reach all respondents.

• Having a carefully designed introduction and the promise of confidentiality to increase trust and salience.

• Having a questionnaire designed to increase completion and minimize item non-response.

Web survey response rates will be maximized by using a web panel where panelists have already consented to complete surveys.

4. Tests of Procedures or Methods to be Undertaken

The 2024 NSODAP aims to extend and expand upon prior surveys on organ donation especially the ones funded by HRSA in 2005, 2012, and 2019. All methods used in 2024 will be identical to the 2019 approved survey. Additionally, a majority of the questions (almost 80 percent) included in the current survey were included in the 2019 survey. In some cases, we made minor revisions.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

HRSA will conduct this survey through a contract with Altarum Institute. The Altarum team consists of Altarum as the prime contractor with American Directions Research Group (ADRG) as a subcontractor and a clinical psychologist, survey methodologist, and nationally recognized expert in organ transplantation and donation affiliated with Harvard University as a consultant. The team has expertise in developing sampling designs for telephone surveys and web panels, including the kinds of minority and ethnic group oversamples required by the current survey. Together, they will perform sample selection, data collection, analysis of the results, and writing of the report for public distribution. Altarum and ADRG planned and conducted the 2019 NSODAP on behalf of HRSA.

7