Supporting Statement A for Request for Clearance:
COLLABORATING CENTER FOR QUESTIONNAIRE DESIGN AND EVALUATION RESEARCH
Research and Development Survey 8
OMB No. 0920-0222
Expiration Date: 01/31/2026
Contact Information:
Amanda Titus
Behavioral Scientist, Collaborating Center for Questionnaire Design and Evaluation Research
Division of Research and Methodology
National Center for Health Statistics/CDC
3311 Toledo Road, Room 5451
Hyattsville, MD 20782
301-458-4579
April 6, 2023
Table of Contents
A. Justification
A.1. Circumstance Making the Collection of Information Necessary 3
A.2. Purpose and Use of Information Collection 4
A.3. Use of Improved Information Technology and Burden Reduction 7
A.4. Efforts to Identify Duplication and Use of Similar Information 7
A.5. Impact on Small Businesses or Other Small Entities 7
A.6. Consequences of Collecting the Information Less Frequently 7
A.7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5 7
A.8. Comments in Response to the Federal Register Notice and
Efforts to Consult Outside the Agency 7
A.9. Explanation of Any Payment or Gift to Respondents 8
A.10. Protection of the Privacy and Confidentiality of Information Provided
by Respondents……………………………………………. 8
A.11. Institutional Review Board (IRB) and Justification for Sensitive Questions 10
A.12. Estimates of Annualized Burden Hours and Costs 10
A.13. Estimates of Other Total Annual Cost Burden to Respondents
or Record Keepers 11
A.14. Annualized Cost to the Federal Government 11
A.15. Explanation for Program Changes or Adjustments 11
A.16. Plans for Tabulation and Publication and Project Time Schedule 11
A.17. Reason(s) Display of OMB Expiration Date in Inappropriate 12
A.18. Exceptions to Certification for Paperwork Reduction Act Submissions 12
LIST OF ATTACHMENTS
Attachment 1: RANDS 8 Questionnaire
Attachment 2: Advertisement
Attachment 3: Screening Script
Attachment 4: Informed Consent
Attachment 5: Thank You Letter
Attachment 6: Data Retention Policy
Attachment 7: NCHS Affidavit of Nondisclosure
Attachment 8: NORC CIPSEA Protection Plan
Supporting Statement A
Collaborating Center for Questionnaire Design and Evaluation Research
The staff of the National Center for Health Statistics’ (NCHS) Collaborating Center for Questionnaire Design and Evaluation Research (CCQDER) (OMB No. 0920-0222, exp. 01/31/2026), in collaboration with the Division of Research and Methodology (DRM) and the Division of Health Interview Statistics (DHIS), plans to conduct a methodological survey as part of its Research and Development Survey (RANDS) program. Similar to previous rounds of RANDS that have been approved in the past, the survey content for this round will be related to health and will generally mirror content found on the National Health Interview Survey (NHIS), the Census Bureau’s Household Pulse Survey, and the Behavioral Risk Factor Surveillance System (BRFSS). RANDS will be to continue to develop NCHS’ statistical and weighting approaches to using commercial survey panel data and to evaluate potential questions for NCHS and CDC surveys.
Specifically, this request for approval is to conduct one round of RANDS (RANDS 8); which will follow the same protocols approved previously under this generic clearance. It will continue to use NORC’s AmeriSpeak Panel as the primary sample source, though it will also include a supplemental opt-in sample that will allow NCHS to broaden its methodological research. Note that unlike the special RANDS during COVID-19 series (OMB No. 0920-1298, expiration: 11/30/2020 and OMB No. 0920-1323, expiration: 8/31/2021), NCHS is not requesting clearance to release prevalence estimates of any variables in this survey, which will be conducted for methodological purposes related to the underlying generic clearance. As is typical for RANDS projects, these data will be used by NCHS for methodological purposes only, and public data files will be released along with extensive documentation about the limitations of the data1.
We propose to start programming the survey instrument for RANDS 8 as soon as we receive OMB approval. This round will collect complete responses from 6,600 AmeriSpeak (6,000 via web, with an additional 600 via phone interview) panelists and 10,000 opt-in (all collected via web) panelists.
A. JUSTIFICATION
1. Circumstances Making the Collection of Information Necessary
As the nation’s principal health statistics agency, the National Center for Health Statistics (NCHS) is responsible for not only producing high-quality statistics on the health and wellbeing of the American public and the state of the country’s health care system, but also for contributing to the development of survey methodologies that will allow the agency to continue producing these health statistics in the future. OMB previously approved (2017-2018 RANDS, Cognitive Interview Sub-Study, approved 2/19/18; 2017-2018 RANDS, Web Panel Sub-Study, approved 11/19/18; RANDS 4 Telephone Sample, approved 2/5/20, RANDS-NISVS, approved 7/30/21; and the three rounds RANDS during COVID-19 series OMB Nos. 0920-1298 and 0920-1323). NCHS uses commercially available, pre-existing survey panels to supplement its ongoing examinations of measurement error and estimation techniques. This methodological survey system—NCHS’ RANDS—has allowed the Center to not only conduct meaningful subgroup analyses of survey response patterns, explore patterns of survey item interpretation and potential reporting errors at a population level, but also to explore how estimates from commercial survey panels might be used in the future to supplement NCHS’ official statistics information collections and survey systems. To date, seven separate cycles of RANDS have been conducted2 with an additional 3 occurring as part of RANDS during COVID-193; totaling 10 rounds.
As with previous rounds, DRM proposes the round of RANDS in this ICR to function a methodological study that will provide a testbed for NCHS’ estimation and question evaluation research. Specifically, it will focus on:
Evaluating potential calibration variables that will allow NCHS to re-calibrate RANDS’ weights to surveys other than just the NHIS
The response processes behind those calibration variables
The intersection of measurement and sample error, particularly in small population subgroups and intersectional groups.
Refining processes for integrating web probes with post hoc cognitive interviewing findings for question and questionnaire evaluation purposes.
Additionally, NCHS will continue to use the proposed round of RANDS to supplement its ongoing question evaluation work, including evaluations of items collecting information on gender identity, vaccination practices and beliefs, functional disability, emotional and social well-being, discrimination, and COVID-related topics.
2. Purpose and Use of Information Collection
Questionnaire Content:
The questions we plan on administering in RANDS 8 are included as Attachment 1. In addition to the inclusion of the standard RANDS variables that allow NCHS to calibrate and benchmark RANDS data to the National Health Interview Survey (NHIS), this questionnaire covers other topics that will help DRM and NCHS advance its methodological research, including:
Gender Identity: NCHS is continuing its work on developing novel approaches and evaluating existing gender identity items. In light of CCQDER’s recent work testing a single non-binary gender item on the U.S. State Department’s passport form4, the State Department’s employee management system5, and a series of cognitive interviews that took place in September of 20226, CCQDER plans to explore how response to a single item in a household survey context differs from that of a two-question approach (such as that recently suggested by a National Academies panel7) where respondents are asked to provide their sex at birth/sex on birth certificate. This also builds on work previously approved and carried out on RANDS 58 and 79. In addition to comparing the response and outcomes of the two approaches, RANDS 8 will also be used to examine the comprehension of, and response problems stemming from the inclusion of, the “sex assigned at birth” item (which is typically the first question in a two-question approach). The cognitive interviews noted above have indicated that this question is problematic among certain gender minority respondents and may lead to survey drop-offs or disengagement. RANDS 8 will provide an opportunity to quantitatively examine this item and its potential sources of error.
Novel Calibration Variables: To this point, DRM has found success calibrating RANDS data to the NHIS using a combination of demographic and chronic condition variables. DRM and NCHS’ Division of Health Interview Statistics (DHIS) have developed an expanded list of potential calibration variables. Specific items include questions about volunteerism, health care access, internet use, and functional disability. NCHS plans on using the proposed RANDS to further evaluate these variables’ response patterns and how the inclusion of these variables in its calibration approach affects estimation. The findings of these analyses will inform the design of NCHS’ new Rapid Survey System.
Emotional Well Being: As part of its research for Healthy People 2030 and for potential future collaborations with the National Center for Chronic Disease Prevention and Health Promotion (NCCDPHP), NCHS plans to explore the construct of emotional well-being (EWB) and how it could be best measured. Initial cognitive interviews were conducted in the Fall of 202210. DRM will use RANDS 8 to both expand upon this earlier qualitative work and to explore the construct validity of other existing measures of EWB, allowing it to eventually propose best practices and potentially develop a short set of EWB items appropriate for large household surveys including the NHIS and the Behavioral Risk Factor Surveillance System (BRFSS).
COVID-19 Vaccination and Testing: DRM plans on using RANDS to continue evaluating how vaccination and vaccine hesitance questions (such as the set that CCQDER developed11 alongside the National Center for Immunization and Respiratory Diseases) can be best adapted to measure hesitance towards specific vaccines—in this case the COVID-19 vaccines. Additionally, DRM will examine a set of screening items commonly used by commercial pharmacies that participate in HHS’ efforts to provide free or low-cost vaccines and COVID treatment. The analysis of a series of cognitive interviews conducted on these items have suggested that some variation exists surrounding the concepts of vaccine boosters and pre-travel COVID testing12; as a result, RANDS 8 will include embedded probes specifically examining the response to these items.
It is important to note that many COVID-19-related survey questions, including those about coronavirus vaccines and COVID testing, were first developed during a public health emergency and were subject to very limited amounts of question evaluation before being fielded. As the pandemic has progressed, public health guidance has evolved, and as some data collection modes (i.e., phone interviews and self-response web surveys) have replaced in-person or paper modes, it is important to evaluate these items now. Doing so will allow survey programs that are currently (or plan on) collecting information on these constructs—including NHIS, Census’ Pulse, and BRFSS—data about the interpretation of these items and how they should be analyzed.
Discrimination: NCHS is considering how it can better incorporate measures of discrimination in its household surveys to better support the Healthy People 2030 goal of better understanding the Social Determinants of Health (SOH). NCHS recently received guidance from its Board of Scientific Counselors (BSC) regarding the measurement of discrimination, and one of the BSC’s recommendations was to conduct question evaluation studies examining how respondents interpret and respond to common discrimination scales, including the Everyday Discrimination Scale and the Heightened Vigilance Scale. DRM will use RANDS 8 to conduct mixed-method evaluations in response to this recommendation, expanding upon the findings from the cognitive interviews conducted as part of the RANDS 7 project13, as well as from a previous mixed-method evaluation conducted as part of RANDS 614. DRM will investigate whether existing measures of discrimination capture intersectional experiences, and whether and how interpretations of individual items vary across population subgroups.
NCHS anticipates that future rounds of RANDS will follow up on the analyses of these data from RANDS 8. Specifically, gender identity, the novel calibration variables, emotional well-being, and discrimination are candidates for further testing and refinement in the next round of RANDS (presumably called “RANDS 9,” to be conducted in the late summer of 2023) based on the outcomes of RANDS 8.
3. Use of Improved Information Technology and Burden Reduction
Respondents will complete the survey using NORC’s web interface or telephone. Respondents will submit their responses electronically; and these responses will be tallied electronically and put into a database by survey software. Electronic collection will minimize the burden on survey respondents and facilitate the most rapid processing of survey results.
4. Efforts to Identify Duplication and Use of Similar Information
The CCQDER at NCHS is the only government facility that currently conducts testing and development of NCHS or other CDC questionnaires. Similar facilities at the Bureau of the Census and the Bureau of Labor Statistics bear the responsibility for testing survey questionnaires associated with their own agencies. The demand for CCQDER activities exceeds available resources.
In order to identify duplication across federal agencies, CCQDER hosts a publicly accessible online searchable database, Q-Bank, that contains all CCQDER evaluation reports. CCQDER encourages all agencies to submit their evaluation reports so that it is possible to track the work done across agencies as well as to build in existing knowledge.
5. Impact on Small Businesses and Other Small Entities
This data collection does not impact small business or other small entities.
6. Consequences of Collecting the Information Less Frequently
This is one-time data collection activity.
7. Special Circumstances Relating to Guidelines of 5 CFR 1320.5
This request fully complies with the regulation 5 CFR 1320.5.
8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agencies
A Federal Register notice for the CCQDER Generic data collection was published on December 12, 2022. No comments were received. No additional comment period is required for GENICs submitted under this generic.
Consultants:
The following individuals have been consulted within the past year on survey methodology and/or on a specific project:
Contact information:
Katherine Irimata
Associate Service Fellow
NCHS
301-458-4075
John Dombrowski
Client Services Manger
AmeriSpeak
312-230-9059
9. Explanation of Any Payment or Gift to Respondents
NCHS will not provide an incentive to panelists participating in RANDS, though NORC does provide non-cash incentives in the form of “AmeriPoints” that panelists can accumulate and exchange for items such as gift cards.
10. Protection of the Privacy and Confidentiality of Information Provided by Respondents
The NCHS Privacy Act Coordinator has reviewed this request and has determined that the Privacy Act is applicable. The related System of Records Notice is 09-20-0164 Health and Demographic Surveys Conducted in Probability Samples of the U.S. Population.
A Privacy Impact Assessment was submitted on May 16, 2016. The CCQDER continues to collect, on a confidential basis, data needed in order to conduct CCQDER studies. The process of informing respondents of the procedures used to keep information confidential begins with the telephone screener and will carry through to the interviewer and all communications with potential respondents. Materials will include all elements of informed consent, including the purpose of the data collection, the voluntary nature of the study, audio or video recording of the interview, and the effect upon the respondent for terminating the interview at any time.
Confidentiality provided to respondents is assured by adherence to Section 308(d) of the Public Health Service Act (42 U.S.C. 242m) which states:
"No information, if an establishment or person supplying the information or described in it is identifiable, obtained in the course of activities undertaken or supported under section...306 (NCHS legislation),...may be used for any purpose other than the purpose for which it was supplied unless such establishment or person has consented (as determined under regulations of the Secretary) to its use for such other purpose and (1) in the case of information obtained in the course of health statistical or epidemiological activities under section...306, such information may not be published or released in other form if the particular establishment or person supplying the information or described in it is identifiable unless such establishment or person has consented (as determined under regulations of the Secretary) to its publication or release in other form,..."
In addition, legislation covering confidentiality is provided according to the Confidential Information Protection and Statistical Efficiency Act or CIPSEA, (Pub. L. No. 115-435, 132 Stat. 5529 § 302), which states:
“Whoever, being an officer, employee, or agent of an agency acquiring information for exclusively statistical purposes, having taken and subscribed the oath of office, or having sworn to observe the limitations imposed by this section, comes into possession of such information by reason of his or her being an officer, employee, or agent and, knowing that the disclosure of the specific information is prohibited under the provisions of this subchapter, willfully discloses the information in any manner to a person or agency not entitled to receive it, shall be guilty of a class E felony and imprisoned for not more than 5 years, or fined not more than $250,000, or both.”
The CIPSEA legislation authorizes the designation of agents (“designated agents” or “agents”) to perform statistical activities on behalf of an agency. These agents function under the supervision of the agency’s employees and are subject to the same provisions of law with regard to confidentiality as an agency’s employees. A Designated Agent Agreement between the agency and the designated agents (e.g. contractors) must be executed before the agents can acquire information for the agency for exclusively statistical purposes under a pledge of confidentiality. This requirement is outlined in an OMB Notice, published in the Federal Register on June 15, 2007, entitled “Implementation Guidance for Title V of the E-Government Act, Confidential Information Protection and Statistical Efficiency Act of 2002 (CIPSEA).”
A Designated Agent Agreement between NCHS and any DRM contractor will be executed if any contractors are hired to acquire information for the NCHS for exclusively statistical purposes under a pledge of confidentiality (i.e. complete any of the five types of activities described in this generic clearance request). Additionally, the agents (contractors) will be required to complete NCHS Confidentiality Training (https://www.cdc.gov/nchs/training/confidentiality/training/), submit a certificate of completion, and sign a pledge to maintain confidentiality (Nondisclosure Affidavit; see Attachment 7) prior to completing DRM work. If the CCQDER contractor hires subcontractors to complete DRM work, the subcontractors must adhere to the same confidentiality and security requirements as DRM staff and contractors.
As outlined in the informed consent form, access to personal information is restricted to DRM staff who can only access the personal information for statistical, training and research purposes. Additionally, other NCHS staff, designated agents such as DRM contractors, or subcontractors may access the personal information for statistical purposes only after signing a Designated Agent Agreement with NCHS. DRM staff, designated agents, and staff from collaborating agencies must complete annual NCHS confidentiality training (https://www.cdc.gov/nchs/training/confidentiality/training/), submit a certificate of completion, and sign the NCHS affidavit of nondisclosure (see Attachment 7) prior to being granted access to any personal information.
The collection of information in identifiable form requires strong measures to ensure that private information is not disclosed in a breach of confidentiality. Storage of confidential data is protected through procedures such as an internal QDRL LAN, passwords and restricted access.
Confidentiality of responses and safeguarding of data at NCHS
Following the completion of this round’s field period, NORC will process the survey data and prepare data files. The data files will not include the respondents’ names, addresses, or any other primary personally identifiable information (PII), including any ISP (internet service provider) address data NORC has about the computer from which the respondent replied to the survey. All metadata tying the respondents to their inclusion in the RANDS sample will be eliminated from the NORC servers, including the backups, following final delivery. The data files will be transferred to NCHS via either a secure File Transfer Protocol (FTP) web portal or by loading them directly on an encrypted memory stick.
Following the delivery of the RANDS datasets and the final methodological report, NORC will remove all RANDS data from its servers, including backups. This will include not only the responses to the survey itself, but the metadata associated with the RANDS (including response, non-response, participation, and sampling flags identifying the RANDS sample). NORC has extensive cyber and physical security in place, including a CIPSEA Information Protection Plan approved by the NCHS Confidentiality Officer and the NCHS Information Systems Security Officer, in order to protect both the security of the front-end survey interface and the back-end storage of the survey’s data (Attachment 8). Additionally, all NORC employees working on the RANDS will complete NCHS confidentiality training, sign the NCHS affidavit of nondisclosure (see Attachment 7), and will be NCHS designated agents via the Designated Agent Agreement between NORC and NCHS. The sampling frame information for AmeriSpeak will be protected under Section 308(d) of the Public Health Service Act [42 U.S.C. 242m(d)] and the Confidential Information Protection and Statistical Efficiency Act of 2018 (CIPSEA Pub. L. No. 115-435, 132 Stat. 5529 § 302).
11. Institutional Review Board (IRB) and Justification for Sensitive Questions
NCHS Ethics Review Board (ERB) and CDC Human Research Protection Office (HRPO) approved this data collection on September 19, 2022.
Informed Consent and Voluntary Nature
All Panel participants have been fully screened and a substantial amount of background data have already been collected (e.g., health and well-being, socio-economic and occupational status, media usage, political views, age, gender, race, ethnicity, etc.), which will be attached to the final files delivered by NORC to NCHS, allowing for extensive non-response bias analysis.
As with previous rounds, the RANDS survey itself will begin with an introduction screen (or introduction text for telephone respondents) similar to what is seen at the beginning of Attachment 1, explaining the general purpose of the survey, and providing the confidentiality and Paperwork Reduction Act language. As signed consent is not possible for internet surveys where the population of respondents is anonymous to NCHS, as in a commercial panel, we will receive a waiver of signed consent from the NCHS ERB (as has been the case in all previous rounds of RANDS). The introduction page will require the respondent to manually click through to the first page of questions (or agree to continue and not hang up for telephone respondents); this action therefore implies consent.
12. Estimates of Annualized Burden hours and costs (Table 1 was OMB approved on January 27, 2023, for CCQDER Generic data collection.)
Estimated Annualized Burden Table 1 (Approved hours under OMB No. 0920-0222)
Types of Respondents |
Form Name |
Number of Respondents |
Number of Responses per Respondent |
Average hours per response (in hours) |
Total Burden Hours
|
Individuals or households |
RANDS |
49,800 |
1 |
15/60 |
12,450 |
Estimated Annualized Burden Table 2 – RANDS 8
Types of Respondents |
Form Name |
Number of Respondents |
Number of Responses per Respondent |
Average hours per response (in hours) |
Total Burden Hours
|
Individuals or households |
RANDS 8
|
16,600 |
1 |
15/60 |
4,150 |
13. Estimates of Other Total Annual Cost Burden to Respondents and Record keepers
There is no annual capital or maintenance costs to the respondent resulting from this collection of information.
14. Annualized Costs to the Federal Government
Funding for RANDS is from the Data Modernization Initiative funds. It is expected that collecting and analyzing the data from RANDS 8 will cost to the government $600,000. It consists mainly of the salaries of the DRM and contracted staff that will (1) assist the questionnaire designers in the design of appropriate laboratory instruments, (2) recruit, schedule, and assist in interviewing volunteer respondents, and (3) assist in the analysis of the results and recommend changes in questionnaire wording.
15. Explanation for Program Changes or Adjustments
This is a generic IC. This one-time data collection will use 4,150 hours.
16. Plans for Tabulation and Publication and Project Time Schedule
This clearance request is for questionnaire development activities to be conducted prior to survey production and for developmental work that will guide future questionnaire design. The majority of laboratory investigations will be analyzed qualitatively. Final reports will be written that document how the question performed in the RANDS survey, including question problems as well as the phenomena captured by the survey question, web probes and experiments as applicable. All reports will be placed on Q-Bank for public access. Reports are used to provide necessary information to guide designs for redesigning a question prior to fielding as well as to assist end users when analyzing the survey data. For field tests/pilot interviewing activities, qualitative and quantitative analysis will be performed on samples of observational data from household interviews in order to determine where additional problems occur. Because NCHS is using state-of-the-art questionnaire development techniques, methodological papers will be written which may include descriptions of response problems, recall strategies used, and quantitative analysis of frequency counts of several classes of problems that are uncovered through the cognitive interview and observation techniques.
17. Reason(s) Display of OMB Expiration Date is Inappropriate
The expiration date will be displayed.
18. Exceptions to Certification for Paperwork Reduction Act Submissions
The certifications are included in this submission.
3 https://www.cdc.gov/nchs/covid19/rands.htm
4 Under OMB# 0902-0222, Amendment Approved 11/30/21; Willson, S. and K. Miller. (2022). Cognitive Interview Evaluation of X Gender Marker Definitions for the U.S. Passport Application Form. National Center for Health Statistics - CCQDER. Hyattsville, MD. https://wwwn.cdc.gov/qbank/report.aspx?1225
5 Under OMB# 0902-0222, Amendment Approved 7/25/22; Willson, S., Miller, K. (2022). Cognitive interview Evaluation of Demographic Questions for the US Department of State Global Employee Management System. National Center for Health Statistics - CCQDER. Hyattsville, MD. https://wwwn.cdc.gov/QBank/Report.aspx?1234
6 Under OMB# 0902-0222, Amendment Approved 8/12/22; Miller, K., & Willson, S.,2022, Development and Evaluation of a Single, Non-Binary Gender Question for Population-Based Federal Health Surveys, Hyattsville, MD: National Center for Health Statistics - CCQDER. https://wwwn.cdc.gov/QBank/Report.aspx?1235
7 https://www.nationalacademies.org/our-work/measuring-sex-gender-identity-and-sexual-orientation-for-the-national-institutes-of-health#sectionProjectScope
8 Under OMB# 0902-0222, Amendment Approved 6/22/21; Miller, K., Willson, S., Ryan, V.. (2021). An Initial Cognitive Evaluation of a 2-Step Gender Identity Measure. National Center for Health Statistics - QDRL. Hyattsville, MD. https://wwwn.cdc.gov/qbank/report.aspx?1219; Ryan, V. “Examining Measurement Error in a Sexual Identity Question.” AAPOR 77th Annual Conference, Chicago, IL. May 13, 2022.
9 Collected under OMB# 0902-0222, Amendment Approved 8/12/22.
10 Collected under OMB# 0902-0222, Amendment Approved 8/12/22.
11 Scanlon, P., and Jamoom, E.. (2019). The Cognitive Evaluation of Survey Items Related to Vaccine Hesitance and Confidence for Inclusion on a Series of Short Question Sets. NCHS. Hyattsville, MD. http://dx.doi.org/10.15620/cdc:110543; Santibanez et al 2020, https://doi.org/10.1542/peds.2020-007609
12 Collected under OMB# 0902-0222, Amendment Approved 8/12/22.
13 Collected under OMB# 0902-0222, Amendment Approved 8/12/22.
14 Collected under OMB# 0902-0222, Amendment Approved 3/23/21.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Modified | 0000-00-00 |
| File Created | 2023-09-03 |