0960-NEW- Supporting Statements Part B for Disability Perception Survey (DPS) (final)

Supporting Statements Part B for Disability Perception Survey (DPS)

OMB No. 0960-NEW

B. Collections of Information Employing Statistical Methods

The purpose of the Disability Perception Survey (DPS) is to is understand the type of information working adults currently have about the SSDI program to improve projections of disability applications and incidence. The Disability Perceptions Survey will capture attitudes and perceptions about SSDI among working-age adults in the general population, and what roles those factors ultimately play in individuals’ decisions to apply to the program. The DPS evaluation will consist of two parts: (1) the DPS administered to working-age adults

(18 to 64 years of age) SSDI program recipients, and those who may qualify for SSDI benefits; and (2) links of the survey data, including the individuals’ social security numbers, to individuals’ administrative records for research purpose. SSA will use the data the DPS collects to learn about the average American SSDI adult recipient’s knowledge and understanding of the SSDI program and about who qualifies for these benefits.

1. Statistical Methodology

We expect to achieve 5,011 completed interviews among adults ages 18-64 years old in the U.S. with a projected survey completion rate of 80.2%. The plan includes a small buffer

(of ~2% of the target 5,000 interviews) in the event that we subsequently deem initially assumed completed surveys ineligible for the study or incomplete.

We will select the sample for the study using a stratified sample design. More specifically, we will stratify the AmeriSpeak Panel using the following variables:

1. Census Region of the panelist (1=Northeast, 2=Midwest, 3=South, 4=West)

2. Race/ethnicity of the panelist (1=non-Hispanic White/non-Hispanic All Other, 2=Hispanic/non-Hispanic Black)

3. Age of the panelist (1=18-49 years, 2=50-64 years)

4. Educational attainment of the panelist (1=Some college or less¹, 2=4-year college or above)

5. Gender of the panelist (1=Male, 2=Female).

We will collect demographic variables such as race, ethnicity, age, education, and gender at the time of panel recruitment and therefore, we will know these variables for all AmeriSpeak panelists. We will know geographic information such as Census Region as a result of the AmeriSpeak Panel design, since we know the addresses of all recruited panelists. To achieve a sufficient number of completed surveys among panelists with an educational attainment of “some high school,” we will follow an adaptive design approach and put more effort into obtaining responses from such panelists, including providing them a higher incentive and/or sending them additional mailings.

As a result, we will assign each AmeriSpeak panelist to one of the 64 (=4*2*2*2*2) strata created by cross-classifying the above five variables.

We will estimate the sample size, , for a given stratum is estimated using:

1. Population proportion of 18-64 year old adults in a stratum relative to all 18-64

year old adults in the U.S. estimated using the 2020 Current Population Survey. The Current Population Survey was part of NORC’S Sampling plan, and was drafted in 2020. Since then, the project encountered some delays, and the Current Population was based on 2020 data.

2. Historical average completion rate ( ) for all panelists in a stratum (estimated using all prior AmeriSpeak surveys).

Table 1 provides the number of sampled panelists (i.e., ) in each stratum; estimated as

Based on historical completion rates, we expect to select 6,246 panelists to achieve 5,011 completed interviews. Table 1 also provides the proportion ( ) of sampled panelists in each stratum relative to the total number of sampled panelists and also the proportion ( ) of 18-64 year old adults in a stratum relative to all 18-64 year old adults in the U.S. Note that and are not identical as a result of varying historical completion rates by stratum; in general, strata associated with 18-49 years of age, some college or less, male, and Hispanic or non-Hispanic Black have lower historical completion rates and, as a result, have a slightly higher proportion of sample allocated to such strata relative to the population proportion .

We will select samples independently for each stratum. Within each stratum, panelists with higher historical completion rates have a higher probability of being selected relative to panelists with lower historical completion rates. Households with multiple panelists will have at most one panelist selected for the survey.

To optimally allocate resources and maximize the overall survey completion rate, we will release the selected sample in four “replicates.” We will randomly assign every sampled panelist to one of four replicates, and we will release a replicate each week for the first four weeks of the survey field period. Releasing the sample in this way will facilitate our use of our best telephone interviewers for this study. It also allows for the possibility of making changes (with regards to mailings, incentives, or other details) for subsequent replicates based on what we observe for the first replicate.

Table 2 provides the expected number of completed surveys for key demographic groups, the proportion of completed surveys in a given demographic group relative to the total number of completed surveys, and the corresponding population proportion (estimated using the 2020 Current Population Survey). In general, the projected distribution of for key demographic groups closely mirrors the population distribution.

Table 2 also provides our recommendations for oversampling the following groups:

1. American Indian / Alaska Native (any ethnicity and includes multi-race where panelist designates one of their races as American Indian / Alaska Native)

2. Rural (defined as non-MSA based on the OMB definition)

3. Low-income² (household income < $30,000³).

We propose oversampling American Indian / Alaska Native by over 100%, rural areas by 35%, and low-income households by 80%.

We anticipate a sampling margin of error of +/- 1.90 percentage points for a 50 percent survey statistic at a 95 percent confidence level. This means, for example, that on a Yes/No question where 50 percent of respondents answer “Yes,” the true population measure will be between 48.10 percent and 51.90 percent (taking into account sampling error) 95 percent of the time. The sampling margin of error decreases for survey statistics below or greater than 50 percent, making the sampling margin of error estimate of +/-1.90 percentage points a conservative estimate of the margin of error. For any subpopulation that is at least 10 percent of the overall target population (i.e., at least 500 completed interviews for the subpopulation), we anticipate a sampling margin of error no larger than +/- 5.96 percentage points for a 50 percent survey statistic at a 95 percent confidence level. Given an overall margin of error of less than 2 percentage points and a margin of error of less than 6 percentage points for any subpopulation of at least 10 percent of the overall population, we anticipate that 5,000 completed interviews will be sufficient for most analyses of interest.

Finally, any comparison in substantive estimates between two subpopulations of 10 percent each (i.e., each group having ~500 completed surveys) will have greater than 70 percent power at the 5 percent significance level; any comparison between two groups where one group has 1,000 completed surveys (i.e., 20% subpopulation) and the other group has 500 completed surveys (i.e., 10% subpopulation) will have greater than 80 percent power at the

5 percent significance level. Given these power calculations, we again expect that 5,000 completed interviews will be sufficient for most analyses of interest.

Table 1: Number of sampled panelists by stratum
Sampling strata					Number of sampled panelists	% sampled panelists	% CPS
Census Region	Race / ethnicity	Age	Education	Gender
1	NH-White / Other	18-49	Some college or less	Male	139	2.2	2.1
1	NH-White / Other	18-49	Some college or less	Female	119	1.9	1.9
1	NH-White / Other	18-49	4-year college or above	Male	114	1.8	1.9
1	NH-White / Other	18-49	4-year college or above	Female	129	2.1	2.2
1	NH-White / Other	50-64	Some college or less	Male	77	1.2	1.3
1	NH-White / Other	50-64	Some college or less	Female	76	1.2	1.3
1	NH-White / Other	50-64	4-year college or above	Male	53	0.8	0.9
1	NH-White / Other	50-64	4-year college or above	Female	54	0.9	0.9
1	Hispanic / NH-Black	18-49	Some college or less	Male	87	1.4	1.3
1	Hispanic / NH-Black	18-49	Some college or less	Female	83	1.3	1.3
1	Hispanic / NH-Black	18-49	4-year college or above	Male	26	0.4	0.4
1	Hispanic / NH-Black	18-49	4-year college or above	Female	28	0.4	0.5
1	Hispanic / NH-Black	50-64	Some college or less	Male	28	0.4	0.4
1	Hispanic / NH-Black	50-64	Some college or less	Female	32	0.5	0.5
1	Hispanic / NH-Black	50-64	4-year college or above	Male	10	0.2	0.2
1	Hispanic / NH-Black	50-64	4-year college or above	Female	10	0.2	0.2
2	NH-White / Other	18-49	Some college or less	Male	236	3.8	3.7
2	NH-White / Other	18-49	Some college or less	Female	194	3.1	3.2
2	NH-White / Other	18-49	4-year college or above	Male	111	1.8	1.9
2	NH-White / Other	18-49	4-year college or above	Female	134	2.1	2.3
2	NH-White / Other	50-64	Some college or less	Male	114	1.8	1.9
2	NH-White / Other	50-64	Some college or less	Female	115	1.8	2.0
2	NH-White / Other	50-64	4-year college or above	Male	55	0.9	0.9
2	NH-White / Other	50-64	4-year college or above	Female	60	1.0	1.1
2	Hispanic / NH-Black	18-49	Some college or less	Male	76	1.2	1.1
2	Hispanic / NH-Black	18-49	Some college or less	Female	71	1.1	1.1
2	Hispanic / NH-Black	18-49	4-year college or above	Male	16	0.3	0.3
2	Hispanic / NH-Black	18-49	4-year college or above	Female	22	0.4	0.4
2	Hispanic / NH-Black	50-64	Some college or less	Male	27	0.4	0.4
2	Hispanic / NH-Black	50-64	Some college or less	Female	23	0.4	0.4
2	Hispanic / NH-Black	50-64	4-year college or above	Male	7	0.1	0.1
2	Hispanic / NH-Black	50-64	4-year college or above	Female	8	0.1	0.1
3	NH-White / Other	18-49	Some college or less	Male	313	5.0	4.8
3	NH-White / Other	18-49	Some college or less	Female	282	4.5	4.5
3	NH-White / Other	18-49	4-year college or above	Male	166	2.7	2.8
3	NH-White / Other	18-49	4-year college or above	Female	194	3.1	3.3
3	NH-White / Other	50-64	Some college or less	Male	155	2.5	2.5
3	NH-White / Other	50-64	Some college or less	Female	152	2.4	2.6
3	NH-White / Other	50-64	4-year college or above	Male	85	1.4	1.5
3	NH-White / Other	50-64	4-year college or above	Female	93	1.5	1.6
3	Hispanic / NH-Black	18-49	Some college or less	Male	273	4.4	4.2
3	Hispanic / NH-Black	18-49	Some college or less	Female	279	4.5	4.2
3	Hispanic / NH-Black	18-49	4-year college or above	Male	69	1.1	1.0
3	Hispanic / NH-Black	18-49	4-year college or above	Female	85	1.4	1.4
3	Hispanic / NH-Black	50-64	Some college or less	Male	98	1.6	1.4
3	Hispanic / NH-Black	50-64	Some college or less	Female	98	1.6	1.5
3	Hispanic / NH-Black	50-64	4-year college or above	Male	25	0.4	0.4
3	Hispanic / NH-Black	50-64	4-year college or above	Female	36	0.6	0.6
4	NH-White / Other	18-49	Some college or less	Male	191	3.1	3.1
4	NH-White / Other	18-49	Some college or less	Female	168	2.7	2.7
4	NH-White / Other	18-49	4-year college or above	Male	130	2.1	2.2
4	NH-White / Other	18-49	4-year college or above	Female	148	2.4	2.5
4	NH-White / Other	50-64	Some college or less	Male	94	1.5	1.5
4	NH-White / Other	50-64	Some college or less	Female	91	1.5	1.6
4	NH-White / Other	50-64	4-year college or above	Male	63	1.0	1.1
4	NH-White / Other	50-64	4-year college or above	Female	64	1.0	1.1
4	Hispanic / NH-Black	18-49	Some college or less	Male	193	3.1	2.8
4	Hispanic / NH-Black	18-49	Some college or less	Female	172	2.8	2.6
4	Hispanic / NH-Black	18-49	4-year college or above	Male	28	0.4	0.5
4	Hispanic / NH-Black	18-49	4-year college or above	Female	39	0.6	0.6
4	Hispanic / NH-Black	50-64	Some college or less	Male	55	0.9	0.8
4	Hispanic / NH-Black	50-64	Some college or less	Female	51	0.8	0.8
4	Hispanic / NH-Black	50-64	4-year college or above	Male	12	0.2	0.2
4	Hispanic / NH-Black	50-64	4-year college or above	Female	10	0.2	0.2
					6,246	100.0	100.0

Table 2: Expected number of completes by key demographic groups
	Expected # of completes	% completes	% CPS
Age
18-39	2,162	43.1	48.2
40-49	1,201	24.0	20.1
50-59	1,026	20.5	21.2
60-64	622	12.4	10.5
Gender
Male	2,300	45.9	49.2
Female	2,711	54.1	50.8
Education
Some college or less	3,171	63.3	65.3
4-year college or higher	1,840	36.7	34.8
Race / Ethnicity
non-Hispanic White	3,089	61.6	59.3
non-Hispanic Black	654	13.1	12.7
Hispanic	843	16.8	18.8
All Other	425	8.5	9.3
Census Region
1	850	17.0	17.1
2	1,047	20.9	20.6
3	1,876	37.4	38.0
4	1,238	24.7	24.2
Oversampling groups
American Indian / Alaska Native*	137	2.7	1.2
Rural (non-MSA)**	809	16.1	11.8
Low income (household income < $30,000)	1,270	25.3	14.1
*Any ethnicity and includes multi-race where panelist designates one of the races as American Indian / Alaska Native
** Rural area is defined as non-MSA based on the OMB definition

2. Procedures for Collecting the Information

NORC will select eligible panelists using the statistical methodology described in section B.1 above. Once NORC selects the full sample of eligible participants, NORC will invite sampled AmeriSpeak panelists to participate through an email and/or SMS text invitation and through the AmeriSpeak member web portal, which alerts panelists when there is a survey available for them. We will invite panel members to participate in the survey via email or phone based on the panel members’ contact mode preference. Online mode panelists will also be able to access the survey via the AmeriSpeak panel web portal. Sampled panel members will complete the survey interview via self-administered web mode, or with an NORC telephone interviewer on the phone. Appendix C-O includes examples of the invitation and prompting scripts.

NORC’s telephone center will conduct the phone interviews. The NORC telephone center has a long history of conducting high-quality phone research. We estimate that we will conduct approximately 85% to 90% of the AmeriSpeak interviews online, with the remainder by telephone. We will also mail an additional USPS reminder to encourage response. The mode of data collection will be either online or by telephone.

3. Methods to Maximize Response Rates

NORC employs several strategies to maximize response rates, as needed, which includes:

• Making respondent communications and the survey instrument available in both Spanish and English

• Allowing respondents to participate by web or phone

• Keeping the field period open for 32 weeks (much longer than a typical AmeriSpeak field period of 3-6 weeks)

• Inviting respondents to participate by email, text message, and USPS mailing

• Expanding the communication protocol to include a postcard, email, and text message reminders, as well as a USPS ‘last chance’ mailing that includes a $2 bill

• Testing the instrument through both cognitive interviews and pretesting to refine and make the questionnaire as accessible, interesting, and user-friendly as possible.

In addition to releasing the sample in replicates, NORC will implement a cooperation protocol that will include incentives and multiple contact points as part of an adaptive design to target respondents with the most efficient refusal conversion process:

• Incentive for completing the survey. We will provide an incentive of 20,000 AmeriPoints ($20) for those who complete the survey.

• Email/SMS reminders. We will use email reminders for panel members who complete their surveys through the web. Participants who do not respond to the initial survey invitation will receive a reminder email emphasizing the importance of their participation.

• Telephone outreach. NORC will contact panel members who indicated a preference for phone communications and administer the survey at a time that is convenient to them. A telephone option improves the coverage of the sample, especially for people who would not or cannot respond online. We will also make reminder calls for web-mode panel members in cases where NORC has a valid telephone number on file.

• Mailed reminders. Since NORC has the mailing address for panel members, we will also use mailed reminders. Following best practices, we have proposed three reminder mailings: an initial postcard, a reminder letter with a non-contingent cash incentive of $2, and a final reminder letter.

We will give participants the toll-free telephone number and email address of the AmeriSpeak Support Team as well as the NORC IRB to answer questions pertaining to the study or their rights as a research volunteer.

4. Tests of Procedure

NORC conducted nine cognitive interviews between September 18, 2020, and

October 2, 2020 to test and refine the DPS. The testing examined respondents’ comprehension of the content, definitions, and terminology used in the survey. In addition, this process also helped examine question ordering and the clarity of supporting text and introductory statements. In Round 1, nine respondents participated in a cognitive interview. We revised the DPS based on the feedback from these interviews and then developed for Round 2 testing. In Round 2, four respondents participated in a cognitive interview. The results of this round of testing indicated that the DPS was working well. Based on the cognitive testing, we finalized the instrument for fielding in the pretest.

SSA submitted a clearance package to OMB to conduct the DPS pretest and received approval on May 1, 2021. The target population for the pretest study is 18-64 year old adults in the United States. The respondents are members of a probability-based national panel. We will conduct approximately 250 interviews through the web-based completion portal. The web survey will be available in English. We will pay respondents who complete the survey the equivalent of $20 in panel reward points.

The pretest will provide data to inform refinements to the survey and methodology prior to the fielding of the main DPS. We will use the data from the pretest to evaluate the functioning of the survey questions, assess respondent burden, confirm that the programmed survey is functioning as intended, and test data-processing procedures. Upon completion of the pretest, we will revise the DPS if needed prior to the main data collection.

5. Statistical Agency Contacts for Statistical Information

Kristina, Lowell, PH.D., Principal Investigator

Vice President, Health Care Evaluation Research
Telephone: (301) 634-9488
Email: [email protected]

Vicki Wilmer, Project Manager
Senior Research Director, Economics, Justice, and Society
Telephone (312) 759-4299
Email: [email protected]

Nada Ganesh, Lead Statistician
Principal Statistician, Statistics and Data Science
Telephone: (301) 634-9455
Email: [email protected]

Kai Filion, Contracting Officer Representative
Social Security Administration
Telephone: (603) 205-4504
Email: [email protected]

1 ‘Some college or less’ refers to panelists with an educational attainment of some high school, high school graduate, GED graduate, associate degree, or college without a degree.

2 Adults 18-64 years of age living in households with incomes less than the 15th percentile based on the 2018 American Community Survey.

3 Nominal household income, not adjusted for inflation.

47