2014_SIPP_Panel_Supporting_Statement_Section_B

SUPPORTING STATEMENT

U.S. Department of Commerce

U.S. Census Bureau

2014 Survey of Income & Program Participation Panel

B. Collection of Information Employing Statistical Methods

1. Universe and Respondent Selection

The SIPP respondent universe is the civilian noninstitutionalized population based on the 2010 decennial census, which contains approximately 304.4 million individuals. The SIPP uses a multistage stratified sample of the U.S. civilian noninstitutionalized population. The first stage involves the definition and division of the United States into groups of counties called the PSUs, which are assembled into homogeneous groups called strata. The second stage involves selection of units within the PSU.

The 2014 SIPP Panel sample is the first sample for the SIPP to be fielded from the 2010 redesign of the sample. There are 820 PSUs in the 2010 redesigned SIPP and in sample for the 2014 SIPP Panel. The selected PSUs in the 2010 SIPP sample design cover both urban and rural areas of the United States and many were previously in the 2010 SIPP sample design. PSU definitions, address lists, and area segments are all based on the 2010 decennial census.

Within each PSU, living quarters (LQ) are systematically selected from lists of addresses prepared for the 2010 Decennial Census. Other sampling techniques are used to represent new construction, group quarters, and areas where incomplete addresses are common. Low income households were over sampled from the lists of addresses prepared for the census and from the lists of incomplete addresses. Basically, we are taking SIPP sample from the 2010 sample redesign. The 2014 SIPP Panel sample consists of approximately 52,000 designated LQs and will yield approximately 41,600 occupied LQs at the time of interview in 2014 of which approximately 35,000 will be interviewed. Each household contains an average of 2.1 eligible respondents (aged 15 years and older); therefore, the 2014 SIPP Panel should contain approximately 73,500 survey respondents in each wave. The expected response rate for this panel is 75-80%.

2014 SIPP Sample Design

The SIPP respondent universe is the civilian noninstitutionalized population based on the 2010 decennial census, which contains approximately 304.4 million individuals. The SIPP uses a multistage stratified sample of the U.S. civilian noninstitutionalized population. The first stage involves the definition and division of the United States into groups of counties called the PSUs, which are assembled into homogeneous groups called strata. The second stage involves selection of units within the PSU.

The 2014 SIPP Panel is the first sample for the SIPP to be fielded from the 2010 redesign. There are 820 selected PSUs in the 2010 redesigned SIPP. The selected PSUs in the 2010 SIPP sample design cover both urban and rural areas of the United States. PSU definitions, address lists, and area segments are all based on the 2010 decennial census. PSUs are formed from one or more contiguous counties. Larger populated PSUs (> 100,000 HUs) are identified as self-representing (SR) PSUs, while the remaining PSUs are identified as non-self representing (NSR). SR PSUs are in the SIPP sample with certainty while the NSR PSUs are stratified and selected with a probability proportionate to their size. During the stratification process, NSR PSUs are grouped together within the same state to form strata. During the PSU selection process, two NSR PSUs are selected from each stratum. There are 344 SR PSUs and 476 NSR PSUs in sample for the 2014 SIPP.

Within each selected PSU, living quarters (LQ) are systematically selected from lists of addresses prepared for the 2010 Decennial Census. The universe of addresses within the sample PSU is divided into two strata, one with a higher concentration of low income households and the other with a lower concentration of low income households. In the sampling strata, low income is defined based on the poverty thresholds at the national level for families of certain size and age compositions. For example, the low income household cutoff for one person with no related individual is $16,709, and the low income household cutoff with three people is $26,061. Cut-offs are adjusted by the Consumer Price Index, and applied to the sample frame by household size and also by age of householder for one and two person households. Addresses are sorted by geographic and demographic variables and a systematic selection of units is taken from each stratum. A 23.8% higher sampling rate is used in the stratum with the higher concentration of low income households, thereby resulting in an oversample of low income households. Oversampling is only done to the extent that the rise in the variance for the estimate of low income for persons 55+ is not increased by more than 5%.

The 2014 SIPP Panel sample consists of approximately 52,000 designated LQs and will yield approximately 41,600 eligible LQs at the time of interview in 2014 of which approximately 35,000 will be interviewed. Each household contains an average of 2.1 eligible respondents (aged 15 years and older); therefore, the 2014 SIPP Panel should contain approximately 73,500 survey respondents in each wave. The expected response rate for this panel is 75-80%.

The frame for the SIPP is the Master Address File (MAF), which is maintained by the U.S. Census Bureau and is the source of addresses for the American Community Survey, other demographic surveys, and the decennial census. The MAF is updated using the U.S. Postal Service’s Delivery Sequence File and various automated, clerical, and field operations.

The addresses selected for the 2014 SIPP sample will not be eligible to be selected for another Census Bureau demographic survey (CPS, SCHIP, NCVS, CE, and AHS) before 5 years after the last SIPP interview.

The base sample for the 2014 SIPP is 38,300 addresses; 13,800 addresses comprise the state expansion sample. Sample was added in 16 states in order to reach a 6% CV on the estimate of low income for 20 states. States requiring the least amount of sample to reach the target CV were chosen. CA, TX, FL, and NY required no additional sample. Sample was added in NC, GA, OH, IL, MI, PA, TN, AZ, IN, MS, KY, LA, AL, SC, AR, and NM.

2014 SIPP Weighting

Each year, weights will be created for every month (person level, household level, family level) and the calendar year (person level). Starting in the second year, panel (person level) weights will be produced along with monthly and calendar year weights. Each weight is calculated as the product of three components: the base weight, the household noninterview adjustment factor, and the second stage adjustment factor.

The final head of the household weight is assigned as a household weight. The noninterview adjustment is calculated for each noninterview cell based on the following formula: Adjustment factor = sum (interviewed weights + noninterview weights)/sum(interviewed weights), for each cell. These factors are applied to the base weights. There are 512 noninterview household cells in Wave 1. The second stage weights are calculated as a ratio adjustment of the sum of noninterview weights to the population controls and applied to each cell in six dimensions. The second stage weighting procedure consist of raking (with 400 maximum iteration and 250 tolerance level), cell collapsing and husband/wife equalization. This procedure is done for each reference month separately.

Each monthly weight for the SIPP 2014 Panel is produced based on the SIPP survey universe corresponding to that month. Therefore, the controls (benchmark population estimates) for second stage raking for each monthly weight are those for the corresponding reference month. Each calendar year weight is produced based on the SIPP survey universe in December of that year. Therefore, the controls for second stage raking for the calendar year weights are those for December of that calendar year. Meanwhile, the panel weight is based on December of the first year, so the controls for second stage raking for the panel weights are those for December 2013.

2. Procedures for Collecting Information

In sample households, all people 15 years old and over will be interviewed using regular proxy-respondent rules as described in Attachment F. Wave 1 of the SIPP 2014 Panel will be conducted from February to May of 2014. Wave 2 is scheduled to be conducted from January to April of 2015. Wave 3 is scheduled to be conducted from January to April of 2016. Wave 4 is scheduled to be conducted from January to April of 2017. Approximately 35,000 households are expected to be interviewed for the 2014 Panel. We estimate that each household contains 2.1 people aged 15 and above, yielding approximately 73,500 person-level interviews per wave in this panel. Interviews take approximately 60 minutes per adult on average, consequently the total annual burden for 2014 SIPP-EHC interviews will be 73,500 hours¹ per year in FY 2014, 2015, 2016, and 2017.

We expect the minimum detectable differences between the 2014 SIPP Panel and the 2008 SIPP Panel monthly participation rates to be approximately 0.7 for TANF and SSI, 1.6 for Food Stamps and WIC, and 1.8 for Medicaid at the 10% level of significance.

3. Methods to Maximize Response

In all SIPP Panels, we make special efforts to minimize non-interviews. In each wave, every household in the active sample receives an advance letter that explains the purpose of the survey and why the household’s cooperation is important. In the 2014 SIPP Panel each household will be given one of three brochures that contains specific information about the SIPP and how it specifically relates to them in one of the following three areas: (1) Health (insurance, expenditures, etc.); (2) Labor force (retirement, employment, unemployment, etc.); and (3) Families with children (child care, child well-being, etc.). Each household will also receive an eye-catching glossy “SIPP In the News” flyer which has highlights of when SIPP has been used in major news sources such as the Washington Post. For Type A refusal households, standard procedures include additional visits to the household by another Field Representative (FR) or if needed, a Supervisory Field Representative to convert the household response.

In addition to the methods above, we have designed a multi-wave incentive experiment to evaluate the efficacy of incentives as a means of increasing respondent cooperation with the SIPP. We will divide the panel into four groups and randomly assign each household to one of the groups. Since our sample in 2014 will consist of approximately 52,000 households, each group will contain approximately 13,000 households. The specific details of the 2014 Panel incentive experiment are explained in Section A.9 of the supporting statement.

We have designed the incentive experiment to increase the survey's overall response rate. We hypothesize that the sample receiving the larger ($40) incentive will have a higher response rate than the group receiving the smaller ($20) one; both groups should have a higher response rate than the half of the sample receiving no incentive at all in 2014 Wave 1 interviewing.  Based on the sample sizes in the treatment groups, we will be able to detect a difference in response rate that is at least 1.74 percentage points. .  The statistical differences, programmatic costs, and data quality benefits will all come into consideration as we evaluate the results of the incentive test from the 2014 SIPP panel.

 

Beyond this straightforward test, though, the incentive treatments in 2014 Wave 1 provide the foundation for a more in-depth evaluation into to the utilization of incentives in a longitudinal survey.   The incentive evaluations will consider whether the increase in response rates is linear.  In other words, does going from no incentive to $20 bring a better increase than going from $20 to $40?  We will also test of the effectiveness of the incentive program by examining differences in impact by sub-groups, and decompose the patterns for different groups of respondents.  For example, how much does the effectiveness of the incentive vary based on the income of the respondent households?  Are incentives more effective for some racial/ethnic groups than for others?  Does the size or composition of the household matter?  Third, we can test the effect of the incentives on item nonresponse.  So far, we have focused the experiment on overall response rates to the survey, but it might also be the case that respondents receiving an incentive are more willing to answer the entire survey, even sensitive areas such as assets.  However, we may also find that respondents who know they will receive an incentive for completing the survey might provide a lot of don't know/refusal responses to individual items in an effort to get through the interview as quickly as possible.  Finally, we have designed the incentive experiment to allow for a detailed evaluation of incentive use in a longitudinal survey context. The experimental groups are flexible enough to allow for additional testing in Waves 2+, supporting treatment groups which can be divided starting in Wave 2 to see if a constant incentive is better or worse than one that varies, or if adding or taking away an incentive changes respondents' willingness to continue participating in the SIPP going forward. Further, starting in Wave 2, we will be able to focus on the evaluation of an adaptive design approach to implementing incentives. This is an approach that could allow the targeted administration of incentives to provide the most improvement in sample representativeness and response for a reduced total incentive expenditure. A sample of respondents in Wave 2 will have their incentive receipt eligibility determined based on a model that considers both their prior wave characteristics and response propensity, as well as their potential to improve the sample representativeness in Waves 2 and beyond.

4. Tests of Procedures

The 2014 SIPP Panel is the culmination of a program of evaluation and development emerging from a comprehensive reassessment of the SIPP. Five field tests of the SIPP-EHC instrument have taken place (in 2008, 2010, 2011, 2012, and 2013). A new test sample was initiated in 2011, following the successful 2010 feasibility test. The 2012 SIPP-EHC field test is a wave 2 interview of the 2011 SIPP-EHC field test sample. The reference year for waves one and two of the 2011 SIPP-EHC field tests were calendar years 2010 and 2011. An initial evaluation of the field test results from the 2011 and 2012 field tests is attached (Attachment P). The 2013 SIPP-EHC tested the full implementation of the re-engineered SIPP, including following movers and feeding back data to respondents from the 2012 interview. These developmental stages were interspersed by a series of smaller, specifically focused content, instrument, and systems tests.

5. Contacts for Statistical Aspects and Data Collection

The Census Bureau will collect and process these data. Within the Census Bureau, please consult the following individuals for further information on their areas of expertise:

Sample Design

Tracy Mattingly Lead Scientist (SIPP)

Demographic Statistical Methods Division

301-763-6445

Mahdi S Sundukchi SIPP Survey Design Lead

Demographic Statistical Methods Division

301-763-4228

Data Content

David Johnson Chief

Social, Economic, and Housing Statistics Division

SIPP Principal Investigator

301-763-6443

Data Collection and Tabulation

Jason Fields SIPP Survey Director

Office of the Associate Director for Demographic Programs

301-763-2465

Attachments

A. 2014 SIPP Panel Instrument Booklet

B. SIPP-105(L1)2014–Director's Letter

C. SIPP-105(L1)(SP)2014–Spanish Director's Letter

D. SIPP-105(L2)2014–Director's Letter with $20 incentive

E. SIPP-105(L3)2014–Director's Letter with $40 incentive

F. Respondent Rules

G. Respondent Rules (Spanish)

H. SIPP Fact Sheet

I. SIPP Brochure: Health Insurance

J. SIPP Brochure: Families with Children

K. SIPP Brochure: Labor Force and Employment

L. SIPP In the News

M. SIPP-INF1-2014 Incentive Flyer for Director’s Letter $40

N. SIPP-INF2-2014 Incentive Flyer for Director’s Letter $20

O. SIPP PIN letter

P. EHC Evaluation

1¹See Part A page 9 for a table on burden hours.