1850-0803 v29 Part B System Clearance Cognitive Pilot and Field Studies 2010-13

B. Collections of Information Employing Statistical Methods

1. Universe and Respondent Selection

The data collected will be used for questionnaire development activities rather than to produce estimates about populations. For the most part, the small scale testing activities undertaken as part of this clearance will involve purposive or quota samples, with respondents selected either to cover a broad range of demographic subgroups or to include specific characteristics related to the topic of the survey. In some instances, a probability sample may be drawn, for example, for mail surveys or to permit statistical comparisons about the effectiveness of alternative procedural treatments. A description of the plans for selecting respondents for each individual test will be provided to OMB at the time the questionnaires are submitted.

2. Procedures for Collecting Information

Data collection procedures for the testing conducted under this clearance will be varied, and will most likely include group administration, personal visit, telephone, CATI/CAPI, mail, in-person interviews, and Internet surveys. Statistical results will generally be limited to response rates (for mail surveys), item nonresponse rates, frequency distributions of data items, reliability estimates, and analysis of behavior coding and respondent debriefing data. More specific information about data collection procedures will be contained in the description provided to OMB at the time the questionnaires are submitted.

3. Methods to Maximize Response

In general, callbacks will be used to maximize response rates in personal visit or telephone field tests; reminder phone calls, letters, or second questionnaires will be used to maximize response rates in group administration, mail, and Internet surveys. For cognitive interviews and focus groups, participants will be provided with a token of appreciation for their time and effort. Reminder phone calls and/or letters to participants will be used to encourage them to keep their appointments. Tallies will be kept of the number of nonrespondents to all testing activities. More specific information will be contained in the description provided to OMB at the time the questionnaires are submitted.

4. Testing of Procedures

This entire submission consists of tests of data collection instruments and survey/assessment procedures. We expect that all the tests conducted under this clearance will result in more refined and clearer questionnaires and/or procedures and thus reduced respondent burden.

5. Contacts for Statistical Aspects and Data Collection

Advice on statistical aspects of each individual survey will be sought as the testing program proceeds. Depending on the nature of the research, staff from the four NCES divisions, the Office of the Commissioner, the Office of the Deputy Commissioner, or the Chief Statistician will have primary responsibility for data collection and analysis. The specific research project will also determine whether the data will be collected by NCES or through a contractor. Kashka Kubzdela is the contact person for general questions about data collection and analysis. Other contact persons for questions regarding data collection and statistical aspects of the design will be provided to OMB at the time the questionnaires are submitted.