Appendix X Comments from Outside Consultation Peer Advisory Panel and FNS Responses

Appendix X – Comments from Peer Advisory Panel and FNS Responses

The Peer Advisory Panel (PAP) met on January 12, 2018 to discuss the Revised Study Plan for the Survey of SNAP and Work. The following are the PAP comments and FNS responses:

Timing of the data - Drawing the survey sample from the SNAP caseload in a single month will produce a sample with a higher proportion of long-term SNAP recipients, who may be less likely to work. Also, consider the possibility of drawing the sample from new entrants. If the sample is drawn from a single month, it will be important to clarify that the study results are representative of SNAP participants at one point in time. Response. FNS will not sample new entrants because it would address a different set of research objectives. Drawing the sample from a single month is consistent with the research objective of describing employment of a current cross-sectional sample or “snapshot” of SNAP participants. A sample of SNAP participants in a single month will include both short-term and long-term participants.

Analysis of ABAWDs in waived areas – It is important to be able to compare ABAWDs in waived versus non-waived areas. Response. FNS will create a “demographic” ABAWD variable using data elements available in State data or survey questions (e.g., age and disability) to compare those in waived areas and non-waived areas.

Oversampling rate for ABAWDs – Will seeking to have ABAWDs represent 50 percent of the sample result in too high a rate of oversampling, and is the assumption too high that 30 percent of nondisabled SNAP participants ages 18 to 64 will be ABAWDs? Response. FNS noted two recent refinements to the sample design that would likely result in ABAWDs comprising fewer than 30 percent of participants. First, the sample is expanded from participants ages 18-59 to include participants ages 60-69, increasing the denominator for calculating the percentage who are ABAWDs. Second, the focus is restricted to ABAWDs subject to time limits. In order to meet the ±5.0 percent precision requirement, FNS will decrease the oversampling rate for ABAWDs. The consequence of decreasing the oversampling rate is that the precision for the ABAWD subgroup will decrease slightly due to the smaller number of ABAWDs that will be included in the sample. The Final Study Plan will provide further details.

Data used to select PSUs – Has FNS considered using historical State SNAP caseload data instead of the American Community Survey (ACS) to select the primary sampling units (PSUs) given that the ACS data are estimates and will have some degree of sampling error? Response. While FNS recognizes that the ACS data has some degree of sampling error, it proposes to use the ACS for several reasons. First, asking States to provide historical data would be a deterrent to many States supporting the survey and providing the data for the sample. Also, some States may be unable to provide historical data, leading to inconsistencies in how PSUs are selected across States. Second, given the need for OMB approval before requesting data from States, it would be impossible to select the PSUs using historical State data and field the survey within the contract timeline. Finally, it seems reasonable to assume that most of the sampling error in ACS SNAP estimates is between States rather than within States.

Use of urbanicity to capture local labor markets –Urbanicity is an important variable to stratify PSUs but other variables may be of interest. While the Census Bureau treats urban clusters (which might be a small town) as being urban, there may be very different labor markets and opportunities in small towns compared to large cities. Will the study will be able to include a deeper analysis of the labor market realities for respondents? Response. FNS agrees that analyzing the data by urbanicity categories alone may not be sufficient to capture differences in local labor market conditions. It will obtain BLS Local Area Unemployment Statistics (LAUS) data on unemployment by county and large cities to understand how local labor market conditions affect SNAP participants’ ability to find work. 

Consideration of place-based characteristics – The study seems to focus on personal characteristics of SNAP recipients that might be relevant for employment. However, place-based characteristics may be worth further consideration, especially accessibility of public transit. Response. FNS will explore the availability of other data to link to the survey to examine placed-based issues. In addition, we will consider adding questions to the survey that capture placed-based issues, such as respondents’ perception of availability of transportation in the local area.

Adding survey items on informal sources of income or food support – Consider how households are “getting by,” and find ways to elicit information on informal sources of income and employment. Response. FNS will include survey questions about how respondents are obtaining food when they are no longer on SNAP or to supplement their benefits while they are receiving SNAP.  

Using paper as a survey mode – Have you considered use of a paper-and-pencil (PAPI) survey because some SNAP participants may be more likely to respond to PAPI than either via the web or telephone. Response. Given the complexity of the skip patterns in the survey and need to collected detailed work history, FNS determined that a PAPI survey should not be used.

Rules for locating participants outside of PSUs – Some members of the sample may move out of the PSU from which they were drawn. You need to develop a rule for how far outside of a sample PSU they will go in tracking someone down. Response. If we discover that a sample member moved out of the PSU and into a PSU that is not in the sample, we will attempt to interview the sample member by telephone. For those who moved out of the PSU and cannot be reached by telephone, we will set a maximum distance. The SIPP has often used a maximum distance of 100 miles, which we will initially consider for this survey. The final decision will be determined after PSUs are selected and data collector staffing is determined. The decision involves a potential tradeoff between nonresponse bias and cost.

Analyzing movement into and out of work and SNAP – Many SNAP participants move into and out of work and into and out of SNAP. The survey should ask about the reasons for these transitions. Response. We will make an effort to reconstruct a history of work and SNAP participation, and reasons for transitions. For example, the survey will include questions to all participants on barriers, regardless of whether or not they are currently working or on SNAP. We will also try to get State administrative data about participation prior to the sample month.

Factors that help participants return to work – Consider looking at factors that help participants return to work, not just the barriers faced. Response. The survey sample will likely include some participants that return to work and the questions on work history should help identify these individuals. We will consider including questions in the survey to identify factors that helped a participant return to work.

1