Supporting Statement Part B

NCEE System Clearance for DESIGN and Field Studies 2020-2023

OMB# 1850-xxxx v.X

Supporting Statement Part B

Prepared by:

National Center for Education Evaluation

U.S. Department of Education

Washington, DC

April 2020

B. Collections of Information Employing Statistical Methods

1. Universe and Respondent Selection

The data collected will be used for study design activities, including instrument development, rather than producing estimates about populations. For the most part, the exploratory and testing activities undertaken as part of this clearance will involve purposive or quota samples, with respondents selected either to cover a broad range of demographic subgroups or to include specific characteristics related to the focus of an intervention to be studied (e.g., a curriculum for English Learners) or the topic of the data collection. In some instances, a probability sample may be drawn, for example, for mail surveys or to permit statistical comparisons about the effectiveness of alternative procedural treatments. A description of the plans for selecting respondents for each individual test will be provided to OMB at the time each instrument is submitted.

2. Procedures for Collecting Information

Data collection procedures conducted under this clearance will be varied, and will most likely include group administration, personal visit, telephone, CATI/CAPI, mail, in-person or virtual interviews and focus groups, and online surveys. Statistical results will generally be limited to response rates, item nonresponse rates, frequency distributions of data items, reliability estimates, and analysis of behavior coding and respondent debriefing data. More specific information about data collection procedures will be contained in the description provided to OMB at the time each instrument is submitted.

3. Methods to Maximize Response

In general, callbacks will be used to maximize response rates in personal visits, telephone field tests, or interviews and focus groups; reminder phone calls, letters, or second questionnaires will be used to maximize response rates in group administration, mail, and online surveys. For cognitive interviews and focus groups, participants will be provided with a token of appreciation for their time and effort. Reminder phone calls and/or letters to participants will be used to encourage them to keep their appointments. Tallies will be kept of the number of nonrespondents to all exploratory design and testing activities. More specific information will be contained in the description provided to OMB at the time each instrument is submitted.

4. Testing of Procedures

This entire submission consists of exploratory investigations of evaluation feasibility and design, including tests of data collection instruments and survey/assessment procedures. We expect that all investigation and testing conducted under this clearance will result in more refined and clearer evaluation designs, instruments, and/or procedures and thus reduced respondent burden.

5. Contacts for Statistical Aspects and Data Collection

Advice on statistical aspects of each individual instrument will be sought as the testing and feasibility program proceeds. Depending on the nature of the research, NCEE staff from different program areas will have the primary responsibility for data collection and analysis. Each specific research project will also determine whether the data will be collected by NCEE or through a contractor. Michael Fong is the contact person for general questions about data collection and analysis. Other contact persons for questions regarding data collection and statistical aspects of the design will be provided to OMB at the time each instrument is submitted.