Supporting Statement B - Umbrella Generic for Pretesting_Extension 2021_final Aug 2021

Alternative Supporting Statement for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

Pre-testing of Evaluation Data Collection Activities

OMB Information Collection Request

0970 - 0355

Supporting Statement

Part B

March 2021

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Part B

B1. Objectives

Study Objectives

The objectives of the generic information collections (GenICs) submitted under this umbrella generic are to develop and test information collection instruments and procedures. The activities will help the Administration for Children and Families (ACF) evaluate and improve the quality of the data gathered through ACF’s research and evaluation studies.

Generalizability of Results

The GenICs under this generic umbrella are intended to pre-test data collection efforts. The results are not intended to promote statistical generalizations, but ACF will work to select respondents with characteristics typical of each study’s target population. For example, for instruments that are likely to be used with respondents who speak different languages, ACF may perform cognitive testing to ensure different populations understand questions as translated. As additional examples, a study focusing on low-income married couples would pre-test instruments with low-income married couples; similarly, a study focusing on youth in foster care would pre-test instruments with youth in foster care.

Each individual GenIC will address expected generalizability of results.

Appropriateness of Study Design and Methods for Planned Uses

The GenICs submitted under this generic will employ methods, as described in Supporting Statement A (see A2, Study Design). These methods are appropriate for the pre-testing of data collection instruments and procedures. Each individual GenIC submission will address appropriateness of study design and methods for the specific project.

As noted in Supporting Statement A, this information is not intended to be used as the principal basis for public policy decisions and is not expected to meet the threshold of influential or highly influential scientific information.

B2. Methods and Design

ACF will use the data collected for data collection development activities. Generally, the testing activities undertaken as part of this clearance have and will involve purposive samples with respondents selected either to cover a broad range of demographic subgroups or to include specific characteristics related to the topic of the survey. In some instances, a probability sample may be drawn, for example, for mail surveys or to permit statistical comparisons about the effectiveness of alternative procedural treatments. A description of the plans for selecting respondents will be provided to OMB as part of each Gen IC request.

B3. Design of Data Collection Instruments

Development of Data Collection Instruments

The purpose of this information collection request is to test data collection instruments and survey procedures. ACF expects that all the tests conducted under this clearance will result in improved instruments and/or procedures and thus reduced respondent burden.

Advice on statistical aspects of each individual survey will be sought, as appropriate, as the testing program proceeds. Additional information about consultation and contact information will be provided to OMB within each GenIC request.

B4. Collection of Data and Quality Control

Data collection procedures for the testing conducted under this clearance will vary but are likely to include in-person or telephone interviews, mail surveys and discussion guides. Efforts will collect data using well established methodologies, including: (a) cognitive and usability laboratory and field techniques, (b) behavior coding (c) exploratory interviews (d) respondent debriefing questionnaires, (e) split sample experiments,(f) focus groups, and (g) pilot studies/pretests. Statistical results will address a variety of issues including response rates, item non-response rates, frequency distributions of data items, and analysis of behavior coding and respondent debriefing data. Depending on the nature of the research, ACF/OPRE staff and research and evaluation contractors will have responsibility for data collection and analysis. Each GenIC request will provide specific information about data collection procedures for each individual information collection.

B5. Response Rates and Potential Nonresponse Bias

Response Rates

Expected response rates will vary for individual GenIC requests. Information about expected response rates will be provided with individual GenIC requests.

NonResponse

In general, callbacks will be used to maximize response rates in telephone surveys; reminder phone calls, letters, or second questionnaires will be used to maximize response rates in mail surveys. Reminder phone calls and/or letters to participants will be used to encourage them to keep their appointments. Tallies will be kept of the number of non-respondents to all testing activities. More specific information will be provided to OMB at the time the GenICs are submitted. Each GenIC request will provide specific information about methods to maximize response rates and deal with nonresponse, as appropriate.

B6. Production of Estimates and Projections

The data collected under this umbrella generic will not be used to generate population estimates, either for internal use or dissemination.

B7. Data Handling and Analysis

Data Handling

Individual GenICs submitted under this umbrella generic will describe, as appropriate for the proposed activities, the following:

• Procedures for editing to mitigate or correct detectable errors, including checks built into computerized instruments.

• Procedures to minimize errors due to data entry, coding, and data processing.

Data Analysis

Individual GenICs submitted under this umbrella generic will describe data analysis plans for the proposed activities. This will include, as appropriate, methods for any statistical tests, planned analytical techniques to be used, and how the information collected will be used or interpreted in conjunction with any other sources of information. ACF anticipates that data tabulations will be used to evaluate the results of instrument testing.

Data Use

ACF will use the results internally to inform subsequent information collection requests.

Results of these methodological studies may be made public through methodological appendices or footnotes, reports on instrument development, instrument user guides, descriptions of respondent behavior, and other publications or presentations describing findings of methodological interest. The results of these pre-testing activities may be prepared for presentation at professional meetings or publication in professional journals. When necessary, results will be labeled as exploratory in nature and any limitations will be described.

Individual GenICs submitted under this umbrella generic will describe all planned uses of data, including a description of plans to share of the information and any documentation that will be included to improve understanding of how to interpret, analyze, and evaluate information from the collection.

B8. Contact Persons

Individual GenICs will include the name and contact information for the lead individual(s) who can answer questions about the data collection activities.

Attachments

Attachment A: Use of Pretesting Generic Clearance (0970-0355) – 2018-2021

5