GenericClearanceforInternetPanelPretestingandQualitativeSurveyMethodsTesting_SupportingStatementB_072320

SUPPORTING STATEMENT B

U.S. Department of Commerce

U.S. Census Bureau

Generic Clearance for Internet Panel Pretesting and Qualitative Survey Methods Testing

OMB Control No. 0607-0978

B. Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

The data collected will be used for questionnaire development and pretesting activities rather than to produce estimates about populations thus the anticipated response rates to these surveys would be less than 80 percent. Some of the small-scale testing activities undertaken as part of this clearance will involve nonprobability or opt-in samples, with respondents who self-select to participate in the survey or usability test. In other instances, a probability sample may be drawn, for example, for an opinion survey or online surveys, that would permit statistical inferences about the effectiveness of alternative advertising treatments. For usability testing, we might also send emails to a specified group, such as the Census Data Center staff. A description of the plans for selecting respondents for each individual test will be provided to OMB at the time the individual clearance requests are submitted.

2. Describe the procedures for the collection of information including:

   • Statistical methodology for stratification and sample selection,

   • Estimation procedure,

   • Degree of accuracy needed for the purpose described in the justification,

   • Unusual problems requiring specialized sampling procedures, and

   • Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

There are no substantive changes to methodology from the last renewal. Data will be collected via the Internet, telephone, mail and in-person studies. Remote usability testing will be through the Internet. Statistical results will include response rates, click rates, item nonresponse rates, usability paradata, frequency distributions of data items, and analysis of opinion data. More specific information about data collection procedures will be contained in the description provided to OMB at the time the individual clearance requests are submitted.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

In general, reminder emails, calls, mailing or text messages may be used to maximize response rates in surveys. These may be the topic of a given experiment, or may be used to maximize overall response. Tallies will be kept of the number of nonrespondents to all testing activities that involve an invitation to the individual survey (if respondents are invited via a link on our website, we will not have a measure of nonresponse). More specific information will be contained in the description provided to OMB at the time the individual clearance requests are submitted.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

This entire submission consists of tests of data dissemination websites, advertising, data collection instruments and survey/census procedures. We expect that all the tests conducted under this clearance will result in more usable websites, simpler questionnaires, easy-to-understand advertisements and/or procedures and thus reduced user or respondent burden.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Nancy Bates, Dr. Paul Beatty and Dr. Peter Miller have served as general methodological consultants for this project. Reg Baker, Consultant at Market Strategies International, has also provided external guidance on this topic previously. Additional advice on statistical aspects of each individual survey will be sought as the testing program proceeds. Depending on the nature of the research, staff from subject-matter divisions, operational divisions, and the Center for Behavioral Science Methods will have primary responsibility for data collection and analysis. The specific research project will also determine whether the data will be collected by the Census Bureau or through a contractor. Jennifer Hunter Childs (202-603-4827) is the contact person for general questions about data collection and analysis. Other contact persons for questions regarding data collection and statistical aspects of the design will be provided to OMB at the time the questionnaires are submitted.

Page | 2