Response to OMB Comments

November 9, 2011

MEMORANDUM

To: Shelly Martinez, OMB

From: Andy Zukerberg, NCES

Through: Kashka Kubzdela, NCES

Re: Response to OMB passback on NHES 2012 full scale clearance package (OMB# 1850-0768 v.9)

• When is the Next NHES Scheduled?

Due to the uncertain budget picture, we have not scheduled the next NHES. Ideally, it will occur in 2014 or 2015.

• Will the results of the experiments on incentives and co-branding with Census be included in that next fielding?

The results, budget, and internal contracting factors will drive these decisions. If the Census brand boosts response rates this will play a role in our decision about where future rounds of the survey are conducted. Agency level initiatives regarding contracting could also play a role. In terms of the incentive experiment, we will evaluate the effect of using $5 and $15 incentives with the first topical mailing to late screener respondents (those that responded to the 3^rd or 4^th screener mailing). If the effect that we saw in the 2011 field test holds (late screener respondents were more likely to respond to the topical with a higher incentive) then we would consider proposing the use of the larger incentive ($15) at the topical level in future collections depending on budget situations.

• When will NHES test the adult education module? We had thought an experiment for it might be included in this package.

We are currently coordinating with the Postsecondary, Adult, and Career Education team at NCES and with the Census Bureau to conduct a test of an adult education module that will serve to evaluate the response rates of adults when asked about their own education and training using the NHES methodology. A number of steps need to be taken to move forward with this test, including adapting the adult education items for the self-administered mode, developing a screener that rosters adults (as well as children), developing within household sampling procedures, etc. We anticipate the instruments will require some qualitative testing prior to the field test. Our goal is to conduct the field test sometime toward the end of the NHES:2012 data collection so that we can take advantage of the trained staff and operations infrastructure that Census developed for the NHES. A separate OMB package will be submitted for the field test when the design is formalized.

• Please provide a power analysis for the incentive experiment.

The expected overall sample size for the experiment is 5,959 cases and the expected response rate for the $15 treatment group based on the field test data is 69 percent. If we allocate 80 percent of the sample to the $15 treatment group (n=4,768) and 20 percent of the sample to the $5 treatment group (n=1,192) we expect to be able to detect a 4 percentage point decrease (we would expect to see a decrease in response rate for the $5 treatment group) at an alpha of .05 (t critical=1.96) assuming a design effect of 1.7. The design effect is somewhat conservative given the current sample design and past NHES experience so we might be able to detect slightly smaller differences.

• Sample Size

The final NHES:2012 data collection design included a number of changes based on the NHES:2011 field test that had not been anticipated when the original IGCE and Census Bureau cost estimates were prepared. These changes include a higher screener incentive ($5 rather than $2), additional non response mailings of the screener and topical questionnaires and revised printing and postage estimates based on the finalized forms that were included in the OMB package. As a result of these changes, it was necessary to reduce our initial sample size from 198,000 cases to 160,000 cases. Our analysis indicates that we will still be able to detect differences on key estimates at this reduced sample size. The relevant sections of Part A and B have been updated to reflect the reduced sample size.