NPSAS cog lab passback (Qs and As)

Memorandum United States Department of Education

Institute of Education Sciences

National Center for Education Statistics

DATE: September 17, 2010

TO: Shelly Wilkie Martinez, OMB

FROM: Tracy Hunt-White, NCES

THROUGH: Kashka Kubzdela, NCES

RE: Response to 09/16/2010 through 09/28/2010 OMB Passback for NPSAS:12 Cognitive Interview (OMB# 1850-0803 v.36)

Below we have addressed the initial OMB passback questions on the NPSAS cognitive lab package. No revisions were made to the package.

Q1. Are all of the questionnaire items included in the cognitive interview new or revised?  If not, can NCES differentiate these for us?  Can NCES also explain why any “old” ones need to be included in the cognitive testing?

Response: Cognitive testing will be based on the full interview, including new and previously used items, to ensure the appropriate contextual background for the new questions. The new items are not contained in a stand-alone section, but rather are integrated into sections that contain many previously used items. Additionally, cognitive testing the full interview will provide an opportunity to obtain qualitative feedback on portions of the interview that have not been tested in a cognitive lab before.

The attached flow chart presents the flow of interview sections, and uses the following color scheme:

• new and revised items are shown in dark blue,

• experimental versions of new items in red,

• light blue boxes delineate interview section headings,

• yellow boxes represent internally created variables to be used for routing and fills,

• green boxes represent key topic areas that are mainly relevant for first-time beginning students (FTBs).

Q2. Please reassure us that a 45 minute long questionnaire in this context doesn’t mean that the production questionnaire will be that long.

Response: The cognitive lab is only being tested with first-time beginning students. It is anticipated that the field test production questionnaire may be as long as 45 minutes for first-time beginning students due to the addition of several new items intended specifically for that sub-group of students. Extensive review and analysis of field test data will indicate which items worked well and which did not, so that the interview can be revised prior to full-scale implementation.

The burden estimates for student interviews for the field test and full-scale studies are presented below (excerpted from the forthcoming request for approval of student data collection.) Overall, burden for the field test interview is estimated to be about 38 minutes. Based on field test results, the interview will be shortened and streamlined, resulting in a lower estimate of 25 minutes for completion of the full-scale interview.

Maximum estimated costs to respondents for the NPSAS:12 field test and full-scale implementations

Data collection activity	Sample	Expected eligibles	Expected number of respondents	Percent expected response rate (percent)	Average time burden per response	Total time burden (hours)	Rate per hour	Total cost
Student interview (NPSAS:12)
Field test	4,530	4,302	3,000	70	38.1 min.	1,905 hrs.	10	19,050
Full scale study	117,256	111,427	83,876	75	25.1 min.	35,088 hrs.	10	350,880

Q3. Where are the screening numbers and burden estimates in the supporting statement?

Response: Burden estimates for screening procedures were included in the previously approved recruiting request, and are included here for reference (see paragraph and table below describing burden estimates for recruiting/screening). We have revised the label on the third column to clarify the location of the screening burden estimate. The screening process is expected to take 4 minutes per recruit with 25 students recruited for each of the three institution levels. The total burden for recruiting and screening is expected to be about 5 hours for 75 recruits.

Estimate of Hour Burden

Forty-eight individual cognitive interviews are planned. Each interview session is expected to last approximately, but no longer than 2 hours, to allow cognitive interview participants to review the entire survey, while providing think aloud responses and responding to probes. This package, however, only seeks approval for the burden associated with recruitment, approximately 5 hours.

Estimated respondent burden

Institution Level	Number of Recruits	Hours per recruit {Screening}	Total Hours for recruiting	Number selected	Hours per respondent	Hours for Cognitive Testing	Total Burden Hours
4‑year	25	.066 hr (4 minutes)	1.67	16 (8 Web/8 CATI)	2.0	32.0	33.67
2-Year	25	.066 hr (4 minutes)	1.67	16 (8 Web/8 CATI)	2.0	32.0	33.67
Less that 2-year	25	.066 hr (4 minutes)	1.67	16 (8 Web/8 CATI)	2.0	32.0	33.67
Totals	75		5	Up to 48* (24 Web/24 CATI)		94	102

* A maximum of 48 interview participants will be included.

Q4. If NCES wants to add health questions, we really want NCES to use established measures of health. The following are standard questions that should be replaced with what was proposed. We don't have any other changes to propose on the content.

11. How would you rate your physical health? (modified BRFSS)

1 Excellent

2 Very good

3 Good

4 Fair

5 Poor

Do not read:

7 Don't know / Not sure

9 Refused

12. How would you rate your mental health? (modified BRFSS)

1 Excellent

2 Very good

3 Good

4 Fair

5 Poor

Do not read:

7 Don't know / Not sure

9 Refused

Response: We have implemented the requested change.