Response to OMB questions

To: Shelly Martinez

From: Sharon Boivin

CC: Kashka Kubzdela

Re: Response to Comments on NATES Draft Questionnaire for Cognitive Interviews

Question 8: Is this a “mark one” or a “mark all that apply?” Seems we should specify.

Response: Following NHES precedent, we added to this question (and all similar questions) the instruction “Mark one for each item”. We also added “Mark one” to questions (other than yes/no questions) that did not have this instruction. (“Mark all that apply” instructions were already included for all relevant questions.)

Question 10: (re: specify) Is this asking someone to specify “where” they took the course or to specify “what” courses?

Response: We just want to see if we are missing any important types of instruction—whether it is a type of course or a specific provider. We are including many “other-specify” options in the pilot test, with the hope that this will help us best capture the majority of responses in the (future) final instrument.

Question 20: (re: No skip pattern) Is there any concern that this is too easy to skip out of and be “lost” from the certificate pool for the rest of the survey?

Response: This question (from ATES) is the best we have at the moment. We will probe specifically for “false negatives” on this question during the cognitive interviews (which are being conducted with populations that are likely to have certificates) and if those interviews suggest this is an issue, we will revise the question for the pilot test. If the interviews do not suggest a problem, we will use the question as-is for the pilot. But the IWG is undertaking other developmental work on “certificates”, which will investigate this issue further and help inform how we ask about certificates in a (future) final instrument.

Question 21: (re: were for people) This is awkward and could be read that I got the certificate for the benefit of others. Please probe on this working during the cog lab.

Response: Agreed. We had in fact flagged this question for probing during the cognitive labs.

Additional changes that we made to this version:

1. To maintain the questionnaire flow for the maximum number of respondents, we changed the skip pattern in the “Employment” section (questions 53-74) so that the employed continue from the first question in this section while the unemployed skip after the first section question. (The original version had the reverse—the unemployed continued and the employed skipped.) This changes only the order of the questions (and skip patterns).

2. We had borrowed Question 61 and 68 (it appears twice due to skip patterns) verbatim from the CPS and ACS clarifying the respondent’s industry of work:

Is this business or industry mainly – (Mark one.) [CPS Mar Q47b1, ACS Q44]

• Manufacturing?

• Wholesale trade?

• Retail trade?

• Something else (agriculture, construction, service, government, etc.)?

This question format was designed for CAPI administration, and does not fit well with the paper-and-pencil NATES, so we modified it as follows:

Is this business or industry mainly manufacturing, wholesale trade, retail trade, or something else (agriculture, construction, service, government, etc.)? (Mark one.) [CPS Q47b1, ACS Q44]

• Manufacturing

• Wholesale trade

• Retail trade

• Something else