2019 Part_B_Generic_Clearance_Cognitive_Pilot_and_Field_Studies_final 2_21_19

B. Collections of Information Employing Statistical Methods

1. Universe and Respondent Selection

Under this generic clearance, BJS plans to continue its efforts to develop, test, and improve its statistical methods and survey design and data collection instruments (e.g., questionnaires and paper and online surveys) for new and ongoing projects. BJS use different procedures to complete various statistical design and developmental activities, including (but not limited to): pretesting of different types of survey and data collection methodologies; focus groups; cognitive laboratory activities; pilot testing; field testing; exploratory interviews; experiments with questionnaire design; and usability testing of electronic data collection instruments. The exact nature of the surveys and samples is undetermined at present. BJS will use the information it collects for projects covered under this clearance for methodological and survey design work rather than to produce estimates about populations.

For the most part, the small scale testing activities undertaken as part of this clearance will involve purposive or quota samples, with respondents selected either to cover a range of demographic subgroups or to include specific characteristics related to the topic of the survey. In some instances, a probability sample may be drawn, such as for mail surveys or to permit statistical comparisons about the effectiveness of alternative procedural treatments.

Due to the small nature of the tests, BJS expects that some tests will not involve representative samples. In these cases, samples will be selected more as convenience samples and might, for example: be limited to specific geographic locations; involve expired rotation groups of a current survey; or be known to have specific aggregate demographic characteristics. The needs of the particular sample will vary based on the content of the survey being tested, but the selection of sample cases will not be completely arbitrary in any instance. Where applicable, BJS will include the collection of sample administrative data as part of its pilot testing to allow for the testing of procedures regarding data procurement, as well as the comparability of data across sites.

BJS will describe its specific plans for selecting respondents in the individual project clearance submissions that will be provided to OMB for review and approval.

2. Procedures for Collecting Information

Data collection procedures for the testing conducted under this clearance will vary, and will most likely include: group administration; personal visits; telephone interviews; Audio Computer-Assisted Self-Interview Software (ACASI); Computer-Assisted Telephone Interviewing (CATI); Computer-Assisted Personal Interviewing (CAPI); mail surveys, in-person interviews; web-based surveys; and in-person focus groups. Statistical results will generally be limited to response rates, item nonresponse rates, frequency distributions of data items, reliability estimates, and analysis of behavior coding and respondent debriefing data. BJS will provide more specific information about the statistical methods and data collection procedures that will be used in the individual project clearance submissions.

3. Methods to Maximize Response

BJS will use various measures to maximize response rates, including (but not limited to): callbacks for personal visit or telephone field tests and reminder calls; letters; and follow-up questionnaires for group administration, mail, and web-based surveys. BJS (or its data collection agent for the project) will make reminder phone calls and/or send letters to encourage participants to keep their appointment(s) and will track the number of nonrespondents for each testing activity. BJS does not currently have plans to offer incentives to participants of cognitive interviews or focus groups, but may opt to do so as described in Part A. BJS will provide more information about the specific methods that will be used to maximize and encourage response rates in the individual project clearance submissions.

4. Testing of Procedures

This generic clearance covers statistical activities that will be completed to develop, inform, and improve BJS’s data collection instruments, survey design, and other methodological procedures. BJS expects that all the development activities conducted under this clearance will improve data quality, result in more refined and clearer data collection instruments, and improve data collection procedures. BJS also anticipates that these improvements will decrease the respondent burden associated with the projects, reduce costs, and maximize staff resources.

5. Contacts for Statistical Aspects and Data Collection

BJS will seek input on statistical aspects of each individual survey as the testing program proceeds. The nature and topical area of the project will determine who in BJS (e.g., a statistician from the appropriate statistical unit) will have primary responsibility for the data collection, analysis, and oversight of each project. The specific scope of work associated with the project will also determine whether the data will be collected by BJS or by an award recipient that serves as a BJS data collection agent.

Dr. Allen Beck and Ms. Allina Lee are the contacts for general questions about BJS data collection activities submitted to OMB for approval. The appropriate BJS contact(s) for questions regarding the data collection and methodological aspects for specific projects will be included in the individual clearance submissions that are provided to OMB for review and approval.

BJS contacts for the generic clearance include:

Allen J. Beck

Senior Statistical Advisor

810 Seventh Street, N.W.

Washington, DC 20531

202-616-3277

[email protected]

Allina D. Lee

Statistical Policy Advisor

Criminal Justice Data Improvement Program Unit

810 Seventh Street, NW

Washington, DC 20531

202-305-2696

[email protected]

2