SUPPORTING STATEMENT A
U.S. Department of Commerce
U.S. Census Bureau
Generic Clearance for Questionnaire Pretesting Research
OMB Control No. 0607-0725
The Census Bureau requests an extension of the current OMB approval to conduct a variety of small-scale questionnaire pretesting activities under this generic clearance. A block of hours will be dedicated to these activities for each of the next three years. OMB will be informed in writing of the purpose and scope of each of these activities, as well as the time frame and the number of burden hours used. The number of hours used will not exceed the number set aside for this purpose.
This research program will be used by the Census Bureau and survey sponsors to improve questionnaires and procedures, reduce respondent burden, and ultimately increase the quality of data collected in the Census Bureau censuses and surveys. The clearance will be used to conduct pretesting of decennial, demographic, and economic census and survey questionnaires prior to fielding them. Pretesting activities will involve one of the following methods for identifying measurement problems with the questionnaire or survey procedure: cognitive interviews, focus groups, respondent debriefing, behavior coding of respondent/interviewer interaction, and split panel tests.
Explain the circumstances that make the collection of information necessary. Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.
In September 1991, the Census Bureau requested and received a generic clearance (Number 0607-0725) on an experimental basis, which relaxed some of the time constraints and enabled the Census Bureau to begin conducting extended cognitive and questionnaire design research as part of testing for its censuses and surveys. The clearance covered data collections in the demographic, economic, and decennial areas of the Bureau, and specifically applied to research that is focused on questionnaire design and procedures aimed at reducing measurement errors in surveys. As part of the experimental clearance, the Census Bureau submitted to the Office of Management and Budget (OMB) a report that contained indicators of the work that was conducted under the clearance. At the end of the experimental period, the Census Bureau requested and received a three-year renewal of the clearance (through December 1995), covering the same kinds of research activities. Later it was enlarged to include research about incentives, expanded field tests conducted to include split sample questionnaire experiments in multiple panels, and usability testing of electronic instruments. Subsequently, the Census Bureau has received eight more three-year renewals of the generic clearance for pretesting (through December 2022).
At this time, the Census Bureau is seeking another three-year renewal of the generic clearance for pretesting, with the same conditions as the previous clearance. This will enable the Census Bureau to continue providing support for pretesting activities, which is important given the length of time required to plan the activities.
The specific methods proposed for coverage by this clearance are described below. Also outlined are the procedures in place for keeping OMB informed about the nature of the research activities being conducted.
The methods proposed for use in questionnaire development are as follows:
Respondent debriefing questionnaire. In this method, standardized debriefing questionnaires are administered to respondents who have participated in an actual survey or a field test. The debriefing form is administered at the end of the questionnaire being tested, and contains questions that probe to determine how respondents interpret the questions and whether they have problems in completing the survey/questionnaire. This structured approach to debriefing enables quantitative analysis of data from a representative sample of respondents, to learn whether respondents can answer the questions, and whether they interpret them in the manner intended by the questionnaire designers.
Cognitive interviews. This method involves one-on-one interviews in which the respondent is typically asked to "think aloud" as he or she answers survey questions. A number of different techniques may be involved, including asking respondents to paraphrase questions, probing questions asked to determine how respondents came up with their answers, and so on. The objective is to identify problems of ambiguity or misunderstanding, or other difficulties respondents may have answering questions. This is frequently the first stage of revising a questionnaire.
Usability Interviews. This method involves getting respondent input to aid in the development of automated questionnaires and websites and associated materials. A number of different techniques may be involved, such as one-on-one usability interviews with think aloud, probing, and paraphrasing tasks, card-sorting techniques, eye-tracking, and disability accommodation testing. The objective is to identify problems that keep respondents from completing automated questionnaires accurately and efficiently, with minimal burden or that prevent respondents from successfully navigating websites and finding the information they seek.
Focus groups. This method involves group sessions guided by a moderator, who follows a topical outline containing questions or topics focused on a particular issue, rather than adhering to a standardized questionnaire. Focus groups are useful for surfacing and exploring issues (e.g., confidentiality concerns) which people may feel some hesitation about discussing.
Procedures for Clearance
We will provide OMB with a copy of questionnaires, protocols, and debriefing materials in advance of any testing activity. Depending on the stage of questionnaire development, this may be the printed questionnaire from the last round of a survey or a revised draft based on analysis of other evaluation data. OMB will endeavor to provide comments on substantive issues within 10 working days of receipt.
Attachment B contains a generic screener that will be used to recruit members of the general population for cognitive and usability testing. When additional questions are needed for screening, they will be included in the individual submissions.
Any large field tests or dress rehearsals that follow from the initial questionnaire development activity included here are not covered by this generic clearance. Separate submissions for any such data collection efforts will be made.
Data collection for this project is authorized under the authorizing legislation for the questionnaire being tested. This may be Title 13, Sections 131, 141, 161, 181, 182, 193, and 301 for Census Bureau-sponsored surveys, and Title 13, Section 8 and Title 15, Section 1525 for surveys sponsored by other Federal agencies. We do not now know what other titles will be referenced, since we do not know what survey questionnaires will be pretested during the course of the clearance.
The information collected in this program of developing and testing questionnaires will be used by staff from the Census Bureau and sponsoring agencies to evaluate and improve the quality of the data in the surveys and censuses that are ultimately conducted. None of the data collected under this clearance will be published for its own sake.
Because the questionnaires being tested under this clearance are still in the process of development, the data that result from these collections are not considered official statistics of the Census Bureau or other Federal agencies. Data will be included in research reports prepared for sponsors inside and outside of the Census Bureau. The results may also be prepared for presentations related to survey methodology at professional meetings or publications in professional journals.
Information quality is an integral part of the pre-dissemination review of the information disseminated by the Census Bureau (fully described in the Census Bureau's Information Quality Guidelines). Information quality is also integral to the information collections conducted by the Census Bureau and is incorporated into the clearance process required by the Paperwork Reduction Act.
Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g. permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also, describe any consideration of using information technology to reduce burden.
When the census or survey being pretested employs automated methods for its data collection, the questionnaire research conducted under this submission will also utilize automated data collection techniques.
Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purposes described in Question 2.
This research does not duplicate any other questionnaire design work being done by the Census Bureau or other Federal agencies. The purpose of this clearance is to stimulate additional research, which would not be done under other circumstances due to time constraints. This research will involve collaboration with staff from other agencies who are sponsoring the surveys conducted by the Census Bureau. The research may also involve joint efforts with staff from other Federal laboratory facilities. All efforts would be collaborative in nature, and no duplication in this area is anticipated.
To the maximum extent possible, we will make use of previous information, reviewing results of previous evaluations of survey data before we attempt to revise questionnaires. However, this information is not sufficient to refine our census and survey questionnaires without conducting additional research.
This research will be designed as relatively small-scale data collection efforts. This will minimize the amount of burden required to improve questionnaires and procedures, test new ideas, and refine or improve upon positive or unclear results from other tests.
This clearance involves one-time questionnaire development activities for each survey that is connected with the clearance. If this project were not carried out, the quality of the data collected in the surveys would suffer.
requiring respondents to report information to the agency more often than quarterly;
requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;
requiring respondents to submit more than an original and two copies of any document;
requiring respondents to retain records, other than health, medical, government contract, grant-in- aid, or tax records for more than three years;
in connection with a statistical survey, that is not designed to produce valid and reliable results that can be generalized to the universe of study;
requiring the use of a statistical data classification that has not been reviewed and approved by OMB;
that includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use; or
requiring respondents to submit proprietary trade secret, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information's confidentiality to the extent permitted by law.
All the guidelines listed in the OMB guidelines are planned to be met. There are no special circumstances. If a particular project needed a special circumstance, we would address that at the project level with OMB.
Consultation with representatives of those from whom information is to be obtained or those who must compile records should occur at least once every 3 years - even if the collection of information activity is the same as in prior periods. There may be circumstances that may preclude consultation in a specific situation. These circumstances should be explained.
Consultation with staff from other Federal agencies that sponsor surveys conducted by the Census Bureau will occur in conjunction with the testing program for the individual survey. Consultation with staff from other Federal laboratory facilities may also occur as part of joint research efforts. These consultations will include discussions concerning potential response problems, clarity of questions and instructions, and other aspects of respondent burden. Additional efforts to consult with potential respondents to obtain their views on the availability of data, clarity of instructions, etc., may be undertaken as part of the testing that is conducted under this clearance.
A notice was published in the Federal Register on June 29, 2022, Vol. 87, Document Number 2022-13853, Pages 38703-38704, inviting public comment on our plans to submit this request. We received one non-substantive response.
Respondents for activities conducted in the laboratory (that is, cognitive interviews, usability interviews and focus groups) under this clearance will receive a small stipend to offset costs of participation. This practice has proven necessary and effective in recruiting subjects to participate in this small-scale research, and is also employed by the other Federal cognitive laboratories. The incentive for participation in a cognitive or usability interview is $40 for an interview under 90 minutes conducted remotely, $50 for an interview under 90 minutes conducted in person, and $60 for an interview lasting more than 90 minutes, and for participation in a focus group it is $75 unless approval for a higher amount is granted by OMB on a case-by-case basis. Census may provide smaller incentives than these amounts at its discretion. All requests for incentives will be justified in writing to OMB.
All respondents who participate in research under this clearance will be informed that the information they provide is confidential and that their participation is voluntary. For personal visit and telephone interviews, this information will be conveyed verbally by the interviewer. For personal visit interviews, respondents will also be notified in writing so they will have something they can keep and read. For self-administered questionnaires, the information will be included in the mailing package, on either the questionnaire or the instructions. All participants in cognitive research will be required to sign written notification concerning the voluntary and confidential nature of their participation. We will also inform respondents in writing of the need to have an OMB number. Copies of the generic clearance consent form templates are contained in Attachment A. They include a consent form for conducting demographic pretesting with adults, parental consent form for conducting demographic pretesting with minors, consent form for conducting usability interviews with adults, a parental consent form for conducting usability interviews with minors, and a consent form for economic survey pretesting.
The confidentiality of information is assured by Title 13, United States Code, or the confidentiality provisions of other agencies’ applicable Titles that authorize the collection of information.
Most of the questions that are included on Census Bureau questionnaires are not of a sensitive nature and should not pose a problem to respondents. However, it is possible that some potentially sensitive questions may be included in questionnaires that are tested under this clearance. One of the purposes of the testing is to identify such questions, determine sources of sensitivity, and alleviate them insofar as possible before the actual survey is administered.
Indicate the number of respondents, frequency of response, annual hour burden, and an explanation of how the burden was estimated. Unless directed to do so, agencies should not conduct special surveys to obtain information on which to base hour burden estimates. Consultation with a sample (fewer than 10) of potential respondents is desirable. If the hour burden on respondents is expected to vary widely because of differences in activity, size, or complexity, show the range of estimated hour burden, and explain the reasons for the variance. Generally, estimates should not include burden hours for customary and usual business practices.
If this request for approval covers more than one form, provide separate hour burden estimates for each form and aggregate the hour burdens.
Provide estimates of annualized cost to respondents for the hour burdens for collections of information, identifying and using appropriate wage rate categories. The cost of contracting out or paying outside parties for information collection activities should not be included here. Instead, this cost should be included under ‘Annual Cost to Federal Government’ (Item #14).
The total estimated respondent burden is 16,500 hours for the period from December 2022 through November 2025. These hours will be distributed as follows:
Timeline Respondent burden hours
December 2022 - November 2023 5,500 hours
December 2023 - November 2024 5,500 hours
December 2024 - November 2025 5,500 hours
Total 16,500 hours
This estimate is based on our experience with this clearance over history of this clearance, and our anticipated use of the clearance over the next three years, including burden hours expended for multi-panel questionnaire experiments.
A variety of forms will be used in conducting the research under this clearance, and the exact number of different forms, length of each form, and number of subjects/respondents per form are unknown at this time.
Provide an estimate for the total annual cost burden to respondents or record keepers resulting from the collection of information. (Do not include the cost of any hour burden already reflected on the burden worksheet).
There is no cost to respondents for participating in the research being conducted under this clearance, except for their time to complete the questionnaire.
There is no way to anticipate the actual number of participants, length of interview, and/or mode of data collection for the surveys to be conducted under this clearance. Thus, it is impossible to estimate in advance the cost to the Federal Government. Costs will be covered by divisions conducting the research from their data collection budgets.
Explain the reasons for any program changes or adjustments reported in ROCIS.
There are no changes to the information collection since the last OMB approval.
For collections of information whose results will be published, outline plans for tabulation and publication. Address any complex analytical techniques that will be used. Provide the time schedule for the entire project, including beginning and ending dates of the collection of information, completion of report, publication dates, and other actions.
This research program is for questionnaire and procedure development purposes. Data tabulations will be used to evaluate the results of questionnaire testing. The information collected in this effort will not be the subject of any printed Census Bureau reports; however, it might be included as a methodological appendix or footnote in a report containing data from a larger data collection effort. The results of this research may be prepared for presentation at professional meetings or publication in professional journals.
Due to the nature of this clearance, there is no definite or tentative time schedule at this point. We expect work to continue more or less continuously throughout the duration of the clearance.
The agency plans to display the expiration date for OMB approval of the information collection on all instruments. If any particular study is unable to accommodate this, it will be addressed at the IC level with OMB.
The agency certifies compliance with 5 CFR 1320.9 and the related provisions of 5 CFR 1320.8(b)(3).
Page
|
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | DOC PRA TOOLS 2020 |
| Subject | 2020 |
| Author | Dumas, Sheleen (Federal) |
| File Modified | 0000-00-00 |
| File Created | 2022-10-17 |