Memorandum on the requests for nonsubstantive changes and respective justifications

M EMORANDUM

955 Massachusetts Avenue, Suite 801

Cambridge, MA 02139

Telephone (617) 491-7900

Fax (617) 491-8044

www.mathematica-mpr.com

TO: Jennifer Park, Office of Management and Budget

FROM: Nancy Cole DATE: 7/27/2012

SUBJECT: OMB CONTROL NUMBER: 0536-0068

Non-substantive changes resulting from Revised Screener and $5 Screener Incentive, and Postcard to Sample Addresses in Locked Buildings and Gated Communities

This memo describes two changes to FoodAPS instruments/protocols: (1) revised screener and screener incentive; (2) a new postcard designed to reach addresses within locked buildings and gated communities.

FoodAPS Screener

FoodAPS conducts a 5-minute screener with households to determine membership within groups targeted for study completes. As of July 20, the screener cooperation rate for the first three sample releases were 67.5%, 69.9%, 69.3%, compared with a field test rate of 72.2% and a target of 87%.

Mathematica identified two strategies for improving screener response: (1) revise the screener introduction and (2) change the use of the screener incentive. We also understand that we may be limited in our ability to increase response and have added measures of nonrespondent characteristics for use in nonresponse bias analysis.

The screener introduction has been revised to: (a) shorten the introduction and get to the first question faster, (b) put less emphasis on the full-study, (c) describe the purpose of the study in a way that provides salience for all income groups, and (d) emphasize that the screener takes 5 minutes. The screener introduction is also revised to drop the unconditional offer of $5 prior to screener completion. For respondents who refuse to complete the screener, we explain (question 2b) the importance of completing the screener even if they don’t wish to participate in the study, and if they further refuse, we offer the $5 incentive. These changes were made in response to observations from field interviewers that respondents who refuse screening often focus on refusal to participate in the full study, and many respondents were “put off” by the $5 incentive that was offered unconditionally to all households prior to screener completion. The introduction is changed on page 1 through question 2c, and on page 7 (New Respondent Intro).

Mathematica proposes a change in the use of the screener incentive. To date, 82% of screener respondents have accepted the incentive while 18% completed the screener but did not accept the incentive; 14% of screener refusals accepted the incentive but did not complete the screener. Among respondents, the rate of acceptance is lower for higher income respondents than lower income respondents (78% versus 88%).

We are concerned that higher income households are “put off” by a $5 incentive that holds little value for them and is possibly perceived as an unwelcome commitment or a waste of taxpayer money. At the same time, the $5 incentive may add little value for low-income respondents who are sufficiently motivated by the possibility of the study incentive ($100). However, we do not want to drop the incentive entirely because we cannot know how effective the $5 incentive has been in converting reluctant respondents. Certainly, we know that we have screened reluctant respondents. To avoid the disadvantages of the incentive while maintaining the advantages, we suggest use of the incentive conditionally only after a potential respondent refuses to complete the screener.

It may not be possible to raise screener cooperation rates to our target level. Therefore, we have expanded the screener to collect information about nonrespondents. These questions are on the back page of the screener in three sections:

• Four questions for persons who refuse to complete the screener,

• Three questions for respondents who complete the screener and are eligible but refuse to participate in the study, and

• Interviewer observations of all persons who complete or refuse the screener.

Persons who refuse to complete the screener are asked two questions which help determine eligibility: (R1) household size, and (R2) “Is your total household income greater than $30,000 a year?” Nearly half of all households are one- or two-person households and income above $30,000 puts these households in quota group C, which is most often screened out as income ineligible (due to its high prevalence). We do not ask interviewers to use this information to code eligibility. Households responding to questions R1 to R4 are initially statused as screener refusals, however, these households may be re-statused during data processing as ineligible if they are in a replicate that is closed for quota group C.

The questions for nonrespondents and study refusals include indirect measures of nonresponse (household characteristics that may be correlated with food acquisition outcomes) and direct measures of food acquisitions. The latter include two questions about household spending on groceries and eating out last week. These measures are not directly comparable to measures of food acquisitions obtained from detailed tracking and receipts provided by study respondents, but they provide a direct measure of food acquisitions that may be used to inform the direction and potential magnitude of nonresponse bias.

For respondents who complete the screener and are eligible but refuse the study, the new questions about spending on food replace three prior questions (one contained 11 parts) about the types of food stores where the respondent shops. Two questions about spending provide a more direct measure of food acquisitions and reduce burden on these respondents. We also eliminated the question about the number of employed persons in the household to reduce burden and because these respondents answer the screener question about receipt of earned income.

Interviewer observations will be collected for all persons who either complete or refuse the screener. These provide gender, language, approximate age, race/ethnicity, and an indication of whether multiple persons were encountered at the sample address. We ask for observations of multiple contacts because response may be influenced by the presence of “gatekeepers” or the inability to get past gatekeepers (we may find that response is more likely when the initial contact refers us to someone else in the household). Observations of study respondents will be compared with survey data to gauge the accuracy of interviewer observations overall. Interviewer observations provide an important input to the nonresponse bias analysis because they are not dependent on response.

Overall, the impact on burden is minimal and difficult to estimate because it requires assumptions about the change in screener response rate, which determines the percentage of respondents who answer the full screener versus the four nonrespondent questions. We estimated burden for screener respondents to be 10 minutes on average (including the introduction, completion of the screener, and collection of contact information and introduction of the study for those who are eligible). We estimated burden to be no more than 5 minutes on average for nonrespondents and this is not changed by the additional four brief questions which partially are offset by a shorter, streamlined introduction.

Postcard for Residents of Restricted Communities

Mathematica is currently sending a letter to management companies for locked buildings and gated communities when field interviewers are unable to access sample addresses. To date only 20 letters have been sent, which is well below our estimate of 444.

Some field interviewers have reported that managers are unable to provide access and that we can only obtain access directly from residents. To obtain access, we propose to send a second postcard to these addresses, which closely resembles the advance postcard, but says “We’ve been trying to reach you. Please call us at:” (see attached). Actual postcards will contain the phone number of the field interviewer assigned to the case. We propose to use these postcards when we are unable to gain access directly or through building management. The current list of addresses for this new mailing is under 20 and we don’t expect it to reach 100. Thus, the additional burden is offset by the overestimate of the burden on building managers.

Training to Introduce Screener Changes

Mathematica briefed field managers about the proposed changes to the screener and screener incentive on Thursday July 26. Pending OMB approval, Mathematica plans to ship new screeners to field interviewers for receipt on Tuesday, July 31. The Deputy Project Director will present the screener changes to field staff during telephone conference calls on July 31 and August 1. Field managers are required to take a quiz to test their understanding of the revised screener because they are primarily responsible for working with field staff and answering questions on a one-to-one basis.

cc: Mark Denbaly, ERS

An Affirmative Action/Equal Opportunity Employer