Supporting Statement A for Request for Emergency Clearance:
NATIONAL CENTER FOR HEALTH STATISTICS RESEARCH AND DEVELOPMENT SURVEY
OMB No. 0920-XXXX
Expiration Date: XX/XX/XXXX
Contact Information:
Paul J Scanlon Jr, PhD.
Senior Behavioral Scientist
Collaborating Center for Questionnaire Design and Evaluation Research
Division of Research and Methodology
National Center for Health Statistics/CDC
3311 Toledo Road
Hyattsville, MD 20782
301-458-4649
May 08, 2020
Table of Contents
A. Justification
A.1. Circumstance Making the Collection of Information Necessary 5
A.2. Purpose and Use of Information Collection 5
A.3. Use of Improved Information Technology and Burden Reduction 8
A.4. Efforts to Identify Duplication and Use of Similar Information 8
A.5. Impact on Small Businesses or Other Small Entities 9
A.6. Consequences of Collecting the Information Less Frequently 9
A.7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5 9
A.8. Comments in Response to the Federal Register Notice and
Efforts to Consult Outside the Agency 9
A.9. Explanation of Any Payment or Gift to Respondents 10
A.10. Protection of the Privacy and Confidentiality of Information Provided
by Respondents…………………………………………….. 11
A.11. Institutional Review Board (IRB) and Justification for Sensitive Questions 16
A.12. Estimates of Annualized Burden Hours and Costs 17
A.13. Estimates of Other Total Annual Cost Burden to Respondents
or Record Keepers 18
A.14. Annualized Cost to the Federal Government 18
A.15. Explanation for Program Changes or Adjustments 19
A.16. Plans for Tabulation and Publication and Project Time Schedule 19
A.17. Reason(s) Display of OMB Expiration Date in Inappropriate 19
A.18. Exceptions to Certification for Paperwork Reduction Act Submissions 19
LIST OF ATTACHMENTS
Attachment A - Pub Health Serv Act
Attachment B - Intro Screen and Questionnaire
Attachment C - Telephone Screening Script
Attachment D Cognitive Interview Telephone/Video Introduction Script
Attachment E - NORC CIPESA Protection Plan
Attachment F - NCHS Non-Disclosure Affidavit for NORC Organization Staff working on RANDS
Attachment G - CCQDER Data Storage and Access Policy
Attachment H - NCHS ERB Approval Notice
Attachment I - Recruitment Advertisement
Attachment J - Sample script of CCQDER voice mail.doc
Attachment K - Respondent Data Collection Sheet
Attachment L - Abstracts of Previous RANDS Research on Estimation and Calibration
Attachment M - RANDS during COVID-19 Variable Crosswalk, Uses, and Power Calculation
Attachment N - Health and Health Care Access Questions Across Federal Surveys During COVID-19
Supporting Statement A
Executive Summary
The National Center for Health Statistics’ (NCHS) requests emergency approval for six months to conduct rounds of the National Center for Health Statistics’ Research and Development Survey (RANDS), as well as associated cognitive interviews, specifically during the response to the ongoing Coronavirus pandemic (COVID-19 crisis). While previous rounds of RANDS have been conducted under the Collaborating Center for Questionnaire Design and Evaluation Research’s (CCQDER) Generic Clearance (0920-0222, current expiration: 8/31/2021), we are now requesting a new emergency clearance for this data collection for the following reasons. In addition to continuing the methodological objectives of previous RANDS, given the demands for information related to the COVID-19 pandemic, NCHS proposes to take advantage of past methodological advancements of the RANDS program to use the survey to provide information on population characteristics related to COVID-19. In addition, this round of RANDS will be interpreted in the context of other information collections during the COVID-19 response, particularly the Census Bureau’s Household Pulse Survey during the COVID-19 Epidemic (referred to throughout as the “Census Pulse Survey”, OMB Control # 0607-1013), which collects weekly data on temporal and geographic trends. This request includes burden for both the cognitive evaluation of RANDS survey items and the survey itself.
The overall goal of the broader NCHS-RANDS project is to leverage commercially-created and maintained survey panels to supplement and expand NCHS’ methodological research. Furthermore, RANDS during COVID-19 will allow NCHS to quickly obtain and disseminate information about population health characteristics during the ongoing Coronavirus pandemic and to provide documentation supporting the validity of pandemic-related survey questions.
Specifically, this emergency information collection request has two purposes: (a) generation of data that can help explain health-related experiences of the US population during the pandemic and (b) continuation of developmental survey methods research. These purposes encompass three distinct, but related, activities:
The RANDS-COVID-19 survey, which will be conducted by NORC using their commercially-available probability Amerispeak survey panel and whose data will be calibrated to the National Health Interview Survey (NHIS) based on previous NCHS RANDS research findings (estimation production)
Evaluation and calibration of the differences between Amerispeak, NHIS, and the non-probability panel (estimation research)
The evaluation and validation of the RANDS during COVID-19 questions via cognitive interviewing and probing (measurement research).
The subject matter covered in RANDS during COVID-19 will focus on health related population characteristics and the questionnaire will include items to be used in the calibration to NHIS, to compare to data collected from the NHIS, and to compare to other COVID-19 related data collections that do not focus on health. RANDS is designed to include a set of health questions from the NHIS for the purpose of calibration to increase alignment, and reduce differences due to differential response, between sources associated with heath. Further, RANDS during COVID-19 includes questions included on other Federal COVID-19 surveys for ‘cross-walking’ and triangulation of information among sources. This triangulation will allow NCHS to make more robust and detailed interpretations of the items shared across the surveys.
For the measurement research and cognitive interviewing portion of this request, CCQDER staff will use cognitive interviewing methodology to design and evaluate both interviewer- and self-administered questions. Due to the need to field the survey as quickly as possible it will not be possible to conduct the cognitive interviewing prior to fielding the survey. Rather the interviewing will be conducted as soon as possible to inform the interpretation of the survey data. Cognitive interviewing allows researchers to identify the presence of interpretive patterns as opposed to making estimations or causal statements Additionally, as CCQDER has done in previous rounds of RANDS1, set cognitive probes in the RANDS during COVID-19 survey itself will be used to examine questionnaire concepts (such as reference period utility) and the construct validity of survey items. This measurement error work will provide some of the first qualitative insights into how survey items related to the novel Coronavirus and COVID-19 function. In particular, the findings from the measurement error research will inform the design of planned Coronavirus-related questions for the NHIS.
For the survey portion of this project, NCHS will contract with a commercial firm that maintains its own proprietary survey panel. Using this platform, NCHS will conduct a survey to collect data about the demographic, health, and behavioral characteristics of the sample during the COVID-19 pandemic for the purpose of generating data that can help explain health-related experiences of the US population during the pandemic. For the purpose of continuing our survey methods research, we plan to build on work already done with previous rounds of RANDS on combining information from high quality in-person interview surveys and commercial survey panels (estimation research activity). In both the survey and cognitive interview portions of this information collection, NCHS will conduct sub-group analyses in an effort to understand health outcome and access disparities and well as differences in survey response across groups. Both opt-in and recruited (statistically sampled) panels will be used as the sample source for this survey. The statistically sampled panel will be used for the purpose of generating data that can help explain health-related experiences of the US population during the pandemic and both panels will be used for the purpose of continuing our survey methods research activities.
As explained in detail in Part B, NCHS recognizes even the recruited panels’ data quality are not equivalent to the data quality from its traditional survey systems (such as the National Health Interview Survey and the National Health and Nutrition Examination Survey). When initially developed, NCHS did not intend to use RANDS to create independent official statistics, but rather to develop methods to calibrate web based surveys using the higher quality information obtained from other NCHS data collections in order to increase the content of information obtained as well as the sample size available for some analysis. RANDS was also intended to increase our understanding of how some health outcomes, behaviors, and attitudes are related and how these relationships might differ across populations. Finally, RANDS provided a platform for expanding our question evaluation program.
Building on what we have learned through our work with RANDS and the need for current information on COVID-19 when other data collection systems are not able to operate at all or efficiently, we are expanding the objectives of RANDS to provide substantive estimates of health and health care characteristics related to the COVID-19 pandemic. There is need for rapid information about the health of the population during the COVID-19 pandemic and response that can only be obtained through surveys, and RANDS is able to provide this information given the design of the survey and the methodological insights obtained from past RANDS rounds. Therefore, one of our purposes is to use RANDS data calibrated to the NHIS, in conjunction with information known about the error structure of the resulting data, to generate data that can help explain health-related experiences of the US population during the pandemic. Technical material accompanying all data disseminations will describe the limitations of the data due to coverage and sample design and methods used for calibration and estimation. The presentation of RANDS data will be done in conjunction with information from NCHS’ traditional information collections when those data become available and in coordination with other federal data, when appropriate. Data collections across all of NCHS surveys and with other Federal data collections are being coordinated to achieve this end. In particular, there is a close synergistic relationship between the RANDS, the NHIS, the Census Pulse Survey, the Current Population Survey (CPS), the Medicare Current Beneficiary Survey (MCBS), and BLS’ National Longitudinal Survey of Youth | 1979 (NLSY79).
This ICR is time-sensitive and is therefore being requested under emergency procedures. The United States is currently experiencing an pandemic due to the novel Coronavirus, and CDC requires timely information about the public’s experiences with the virus, the COVID-19 disease, and the pandemic response. The information provided by this data collection will allow CDC and NCHS to target its ongoing public communication and to collect valid data in future information collections.
NCHS’ Research and Development Survey
A new six-month OMB clearance revision is requested for the Centers for Disease Control and Prevention’s (CDC) National Center for Health Statistics’ (NCHS) Research and Development Survey (RANDS). Since COVID-19 has resulted in a public health crisis, NCHS is requesting an emergency clearance. Data related to this pandemic is needed sooner than a regular OMB approval can be obtained; we do not want to delay the urgency of the needed data.
This information collection request encompasses:
The collection of RANDS data via NORC’s statistically sampled AmeriSpeak for the primary purpose of generating data that can help explain health-related experiences of the US population during the pandemic and to continue our methods research on the appropriate use of panels
The collection of RANDS data via NORC’s opt-in panel TrueNorth for the purpose of continuing our survey methods research, conducting estimation research, and measurement research
The evaluation of the RANDS survey questions via cognitive interviews for the purpose of measurement research
These activities will be conducted by the staff of NCHS’ Division of Research and Methodology and its designated agents.
A. JUSTIFICATION
1. Circumstances Making the Collection of Information Necessary
The United States is currently suffering from an pandemic of a novel Coronavirus (SARS-CoV-2) that by some estimates has infected over 1,129,000 Americans and led to over 73,000 deaths in the United States as of May 8, 2020. On January 31, 2020 the Secretary of Health and Human Services determined that this pandemic was a public health emergency2. The Centers for Disease Control and Prevention, of which the National Center for Health Statistics is a component, is one of the federal agencies responsible for the federal response to this pandemic.
One of the main responses to this pandemic is an increase in “social distancing” and associated “stay-at-home” orders whereby places of businesses are closed, people are working from home, and Americans are asked to limit their physical interactions with others from outside their household. As a result, the collection of federal survey data that use face-to-face modes of collection, have slowed or halted. However, data on health outcomes and health care access associated with the pandemic, and the public’s understanding of, and reaction to, the pandemic are vital. NCHS’ Research and Development Survey provides NCHS and CDC with an opportunity to collect this necessary data in a timely way via survey modes unaffected by social distancing.
Data collection for this project is authorized under 42 U.S.C. 242k (Section 306 of the Public Health Service Act). A copy of the legislation is provided in Attachment A. CDC is requesting emergency clearance with the understanding that clearance is limited to six-months.
2. Purpose and Use of Information Collection
This information collection request encompasses three separate, but related information collections: first, a two-round methodological survey (the RANDS during COVID-19) using NORC’s AmeriSpeak Panel. Second, a complementary two-round methodological survey using NORC’s TrueNorth supplemental panel. Third, a set of cognitive interviews that will be used to validate the items on the RANDS during COVID-19 questionnaire. While NCHS would of course prefer to conduct iterative rounds of cognitive interviews prior to the fielding of the RANDS during COVID-19 questionnaire, given both the limitations of the current testing environment due to social distancing and the public health need for the RANDS during COVID-19 data itself, NCHS will be unable to follow its typical workflow. Instead, the cognitive interviewing project proposed here will function as a validity study so that NCHS staff and other subject matter experts can understand what constructs the questions captured as they analyze, interpret, and disseminate the RANDS during COVID-19 data.
All three information collections will be used for the purpose of continuing our developmental survey methods research: The first two will inform our estimation research activities, and all three will inform our measurement research activities.
The data from the Amerispeak panel will also be used for the purpose of contributing to understanding CDC’s ongoing surveillance of the COVID-19 pandemic and generate data that can help explain health-related experiences of the US population during the pandemic. Given the current pandemic and the resulting limitations placed on NCHS’ other data collections, RANDS will provide NCHS and CDC with early estimates of COVID-19-related concepts (such as healthcare access, psychological distress, chronic conditions, health behaviors, COVID-19 testing and health care received in relation to COVID-19, and disease prevention behaviors during this period).
RANDS during COVID-19
As with previous rounds of RANDS (all of which have been collected previously under CCQDER’s generic clearance, OMB control number 0920-0222, current expiration: 8/31/2021), NCHS developed RANDS during COVID-19 with a methodological component that will allow the agency to continue the long process of examining whether or not, and how, commercial survey panels may be integrated into NCHS’ existing survey systems. While one purpose of this information collection is still to discover new, and to improve existing, methods that will increase data quality in the midst of declining response rates and increased costs, the other purpose of RANDS during COVID-19 is to focus on providing NCHS and CDC with information on American’s experiences and health outcomes related to the ongoing Coronavirus pandemic. These data will then be used by NCHS and CDC to inform policy and communication efforts as well as future surveys and studies.
This current proposal is for two new rounds of data collection using NORC’s statistically sampled AmeriSpeak Panel3 as well as its supplemental TrueNorth non-probability (opt-in) panel4. The rounds will be conducted between six and eight weeks apart and will begin either at the end of April or as soon as emergency OMB approval is received. NCHS will contract with NORC to obtain 12,000 complete responses to the first round of the survey and 10,000 to the second. (The previous round of RANDS conducted with NORC had a panel-to-survey response rate of 62%, and NORC is expecting a similar response to this set of surveys. Importantly, the contract with NORC will specify a number of complete responses and not an initial sample size—NORC will be responsible for providing the specified number of completes regardless of the response rate to participate in the surveys.) Of this sample, half of the complete cases in each round will come from NORC’s statistically-sampled AmeriSpeak Panel (the same panel approved for use in the previous two rounds of RANDS under the generic clearance 0920-0222), and the other half will come from their supplemental non-probability panel, TrueNorth. Additionally for the AmeriSpeak panelists only, NORC will contact the same sample for both rounds—giving NCHS the ability to explore how the same respondents’ experiences, perceptions, and understandings of the pandemic change over time.
The questionnaires for the two rounds will be very similar. The questionnaire for the first round is found in Attachment B. Following the first round and an initial analysis of the results, small changes in question wording, the addition of questions (such as probing questions), or the removal of questions may be made. If major changes are made to the questionnaire between rounds, the revised instrument will be shared with OMB via a separate non-substantive change.
The information obtained from RANDS will be used by NCHS and CDC for the purpose of generating data that can help explain health-related experiences of the US population during the pandemic. These data can be used for informing future data collections, policy decisions, and communication strategies. As the sample underlying RANDS is not of the same quality as those in NCHS’ large population health data collections, any data products will include explicit language explaining the resulting limitations and their potential impacts on data quality and accuracy.
Furthermore, dissemination and interpretation of RANDS will be coordinated with COVID-19 data from NCHS’s traditional data collections when they become available and with other related health data, when appropriate. In particular, RANDS during COVID-19 includes a small number of questions that are being fielded (or are planned to be fielded) by other Federal information collections, including the CPS, the Census Pulse Survey, and the NHIS. As shown in Attachment M (and with the notable exception of the extensive overlap with the NHIS, which is largely due to RANDS’ planned calibration and measurement error research efforts), most of these shared variables relate to RANDS during COVID-19’s primary conceptual focus of health care access. Doing so will allow NCHS to triangulate responses across all of these surveys, allowing it a more robust and detailed examination of health care access than if this was not possible. A more detailed explanation of what each survey contributes on its own, and in triangulation with RANDS, is presented in Attachment N. Briefly:
The Census Pulse Survey and CPS are (or will, in the case of the CPS) collecting basic information of both health insurance coverage and receipt or non-receipt of non-coronavirus health care in the last 4 weeks due to the Coronavirus pandemic. Given the broad focus of the Census Pulse Survey, and the economic focus of the CPS, NCHS also expects to be able to get information about how health insurance and receipt/non-receipt of non-Coronavirus health care relate to other non-health variables—in particular employment and economic status. By triangulating these results with those from RANDS during COVID-19, NCHS believes that it can gain insight into potential relationships between economic and health-related barriers to health care.
RANDS uses the NHIS as it major source questionnaire, so a large number of items are shared across the two surveys. The largest proportion of this overlap is due to RANDS during COVID-19’s alignment and calibration plans, wherein NCHS can investigate the overall similarity of the samples and to then correct for some differences using calibrated weights (see Attachment M for the list of “alignment” and “calibration” variables).
Other overlapping variables between these two surveys relate to RANDS’ estimation goals and will provide NCHS with the ability to explore related covariates—for instance whether or not a respondent has a usual place of care. Shared health care access questions will also provide NCHS with a baseline when considering the estimates RANDS will produce during the COVID-19 pandemic. It should be noted that RANDS will not ask the exact version of the non-receipt of non-Coronavirus health care question that NHIS is planning, but rather will ask the CPS/Census Pulse Survey version. There are two reasons for this. First, given that RANDS has a large number of items that overlap with NHIS, but not with CPS or the Census Pulse Survey, matching the wording with the version found on the latter two will allow triangulation to those surveys. Second, NHIS will not collect data from its Coronavirus-related questions until later in 2020; however CPS and the Census Pulse Survey are collecting data contemporaneously with the planned fielding of RANDS during COVID-19—providing more robust points of comparison to a variable that is designed to capture information about the effects of the ongoing pandemic.
Finally, NHIS staff are currently planning for questionnaire changes due to COVID-19, and results from the measurement error research component of RANDS during COVID-19 (see Attachment M for those questions related to this component) will help NHIS finalize these changes.
RANDS during COVID-19 data will inform methodology, policy, and communications decisions by NCHS and CDC related to the Coronavirus pandemic going forward. While the underlying sample quality has known limitations, in combination with calibration methods that NCHS has developed based on previous rounds of RANDS and with coordination with other data being collected and released, we believe that the data quality will be sufficiently fit for our purposes.
Cognitive Interviews for the Evaluation of Coronavirus-Related Survey Items
NCHS plans to conduct a series of cognitive interviews on the RANDS-COVID-19 questionnaire (found in Attachment B). The purpose of the cognitive interviewing project is not to obtain survey data, but rather to obtain information about the processes people use to answer survey questions as well as to identify any potential problems in the questions, e.g., questions which are vague or ambiguous, cannot be answered readily or accurately by the respondent, or otherwise contribute to the non-sampling errors of the survey. Data collection procedures for cognitive interviewing are different from survey interviewing. While survey interviewers strictly adhere to scripted questionnaires, cognitive interviewers use survey questions as starting points to begin a more detailed discussion of questions themselves: how respondents interpret key concepts, their ability to recall the requested information, and the appropriateness of response categories. Because the interviews generate narrative responses rather than statistics, results are analyzed using qualitative methods. This type of in-depth analysis reveals problems in particular survey questions and, as a result, can help to improve the overall quality of surveys.
NCHS typically conducts cognitive evaluations using cognitive interviewing before a survey is fielded—not only to fulfill the pretesting requirements under OMB’s Statistical Standards and Guidelines, but also to as an exercise that allows subject matter experts to plan their analyses. However, given both CDC’s and NCHS’ immediate need for Coronavirus-related data and the current social environment, cognitive interviewing for this project will not take place before the fielding of the RANDS during COVID-19 survey itself. Rather, cognitive interviews will be conducted either when the social environment allows or when CCQDER has the resources and procedures in place to conduct interviews during periods of social distancing. These interviews will serve as a validity test of the RANDS during COVID-19 questionnaires and will provide insight and guidance to NCHS staff and subject matter experts as they analyze the RANDS data. The questions to be evaluated in this case are the items on the RANDS during COVID-19, and the questionnaire for the first round is included as Attachment B. The sample will not be statistically-based, but rather will be purposive, with up to 100 respondents. Findings are not meant to be generalizable to the population, and will not be presented as such.
Alongside the information collected by the set cognitive probes on the RANDS during COVID-19 survey itself, the findings from this cognitive evaluation will be used not only by CCQDER to validate the items on RANDS during COVID-19, but will also contribute some of the first qualitative findings into the functioning of Coronavirus-related survey items. As noted in the survey crosswalk presented in Attachment M, a number of both Federal and non-Federal surveys are currently asking (or are planning to ask) questions related to the pandemic. However, no systematic cognitive interviewing project or web probing project has yet been completed; therefore how these questions function is largely unknown. The cognitive interviews and web probes will provide insights into the construct validity of Coronavirus-related questions. These findings will then not only inform the design of NCHS’ other information collections related to COVID-19, such as potential items on the NHIS and the National Health and Nutrition Examination Survey (NHANES), but also other agencies and organizations when designing and interpretation their own COVID-19-related questions. To this end, CCQDER has closely coordinated with NHIS staff about the specific Coronavirus-related questions on RANDS and areas where measurement research is needed. In order to facilitate this, CCQDER will produce a report detailing the patterns of interpretation of each of the items, which will be made available to the public via CDC’s Q-Bank.
Use of Improved Information Technology and Burden Reduction
Both activities encompassed by this ICR will use information technology to reduce burden.
RANDS during COVID-19
RANDS during COVID-19 will be conducted across two modes: self-response web and interviewer-administrated telephone. About 90% of the AmeriSpeak panelists and all TrueNorth panelists will complete their surveys online using NORC’s proprietary survey software. Among other features, this software skips respondents past questions for which they do not qualify and allows for the embedding of help text. The 10% of AmeriSpeak panelists who will receive the phone survey will use a CATI questionnaire, where again they will be skipped out of questions for which they do not qualify.
Cognitive Interviewing
Given the current inability to conduct face-to-face cognitive interviews, CCQDER is actively working to establish ways to conduct remote cognitive interviews via telephone or video conferencing. Some considerations that NCHS are currently working through vis-à-vis remote collection include the system security of the video conferencing software as it applies to CIPSEA, recruitment challenges, the storage of Information in Identifiable Form (IIF), and the delivery of renumeration to respondents. When possible, CCQDER plans on using this technology to conduct interviews until a return to face-to-face interviewing is possible.
4. Efforts to Identify Duplication and Use of Similar Information
Given the current nature of the COVID-19 pandemic, there are currently no data from large ongoing population health surveys about the novel Coronavirus pandemic and its health and social implications. Some private polling and survey firms have collected COVID-19-related data, but this will not serve NCHS’ and CDC’s purposes given a) the low quality of these typically-RDD surveys and b) the fact that NCHS cannot access both the frame and record-level data that will permit it to create its own estimates that comport to our scientific rigor and fit our purposes.
Although there are other federal data collections obtaining COVID-19 data using methods similar to ours, such as the Census Bureau’s Covid-19 Household Pulse Survey, RANDS differs from other collections in several important ways.
First, the focus of RANDS during COVID-19 is on health so a wider range of health information on multiple constructs will be obtained compared to other more general surveys. In particular, the Household Pulse Survey during the COVID-19 Epidemic conducted by the Census Bureau (and on which NCHS is collaborating) will obtain limited information in a large number of domains using a sample large enough to get information for geographic areas. However, this Census Bureau-led survey only includes eight health questions in the final questionnaire approved by OMB. While most of these questions will also be fielded in the RANDS during COVID-19 (with some minor differences in question text due to mode and reference period), the RANDS’ questionnaire includes dimensions of health not found in the Census survey. By including a subset of the RANDS questions in the Census survey, we will be able to cross-walk between the two data collections; gaining breadth from the Census survey, but greater depth from RANDS. The longitudinal collection of information for the same participants will allow us to describe changes in health care access, behaviors, and attitudes during this rapidly changing environment.
Second, while NHIS will also include COVID-19 questions and its wide range of health questions, there are important differences between the two sources. The RANDS data will be able to provide more timely information relative to other NCHS population health surveys, including NHIS. As described above, the longitudinal collection of information for the same participants will also allow RANDS during COVID-19 to report changes in health during this period. The RANDS program has experience leveraging the strength of the NHIS to improve and evaluate its estimates and includes NHIS questions specifically for this purpose in each round.
Third, we are building on an established relationship with NORC and obtaining data (largely) using methods previously determined by us. This experience will allow NCHS to quickly analyze the RANDS during COVID-19 data in a way that acknowledges and compensates for the survey design.
Lastly, as has been the case with previous rounds of RANDS, data from RANDS during COVID-19 will be used to inform the design of other NCHS (and federal statistical system) survey questionnaires. In particular, since this will be one of the only NCHS surveys collecting information on the public’s interpretations and understandings of concepts around the Coronavirus pandemic, the findings from both components of this information collection request will give other parts CDC (as well as other agencies) their first look at the best practices for collecting data about this crisis.
5. Impact on Small Businesses and Other Small Entities
Information collection for RANDS does not involve small businesses or other small entities.
6. Consequences of Collecting the Information Less Frequently
The information that will be collected under this request is about an ongoing public health emergency that has impacted many aspects of American life, including healthcare access and health outcomes. The plan is to conduct two rounds of the RANDS during COVID-19 survey approximately two months apart. By not collecting both rounds or by collecting the second round at a much later date, NCHS will be unable to understand the in-the-moment changes to health outcomes, access, behaviors, as well as changes in how the public understands key concepts related to the pandemic.
7. Special Circumstances Relating to Guidelines of 5 CFR 1320.5
There are no special circumstances.
8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agencies
This is an emergency request, and a Federal Register Notice will be published after OMB approval.
Consultants outside of CDC:
The following individuals have been consulted about the RANDS during COVID-19:
Debra Reed-Gillette
Director, Medicare Current Beneficiary Survey
Centers for Medicare & Medicaid Services
(410) 786-5525
Jenny Hunter Childs
Assistant Center Chief, Center for Behavioral Science Methods
United States Census Bureau
(301) 763-4927
Keenan Dworak-Fisher
Economist
Bureau of Labor Statistics
Consultants within CDC:
The following individuals within NCHS have been consulted about RANDS during COVID-19:
Jessica Graber
Division of Health and Nutrition Examination Surveys
3311 Toledo Road
Hyattsville, MD 20782
(301) 458-4258
Stephen Blumberg
Division of Health Interview Statistics
3311 Toledo Road
Hyattsville, MD 20782
(301) 458-4107
9. Explanation of Any Payment or Gift to Respondents
RANDS during COVID-19
AmeriSpeak and TrueNorth respondents will not be paid for participating in either round of the RANDS during COVID-19 survey.
Cognitive Interviewing
CCQDER will follow its typical protocol and provide a $40 incentive for participating in the one-hour cognitive interviews. Cognitive interview respondents receive incentives for several reasons:
Typically, respondents are recruited for specific characteristics that are related to the subject matter of the survey (e.g., questions may be relevant only to people with certain health conditions). The more specific the subject matter, the more difficult it is to recruit eligible respondents. Incentives helps to attract a greater number of potential respondents.
Cognitive interviews require an unusual level of mental effort, as respondents are asked to explain their mental processes as they hear the question, discuss its meaning and any ambiguities, and describe why they answered the questions the way they did.
They are usually asked to travel to the laboratory testing site, which involves transportation and parking expenses. (Many respondents incur additional expenses due to leaving their jobs during business hours, making arrangements for child care, etc.).
It is important to offer remuneration sufficient to attract the full range of needed respondent types for cognitive interviewing projects. Inadequate respondent recruitment limits the effectiveness of the questionnaire evaluation.
10. Protection
of the Privacy and Confidentiality of Information Provided by
Respondents
The NCHS Privacy Act Coordinator has reviewed this request and has determined that the Privacy Act is applicable. The related System of Records Notice is 09-20-0164 Health and Demographic Surveys Conducted in Probability Samples of the U.S. Population.
A Privacy Impact Assessment was submitted on October 10, 2019. CCQDER and DRM continue to collect, on a confidential basis, data needed in order to conduct CCQDER and DRM studies. For the RANDS during COVID-19 survey, the process of informing respondents of the procedures used to keep information confidential respondents begins with language explicating the voluntary nature of the survey and providing the legal basis and confidentiality assurance on the initial screen (shown in Attachment B), and will asked to review it before beginning the survey on the next screen. In the cognitive interviews, the process of informing respondents of the procedures used to keep information confidential begins with the telephone screener (Attachment C) and will carry through to the interviewer and all communications with potential respondents. Cognitive interviewing respondents will be asked to give verbal consent (a waiver of signed informed consent has been requested from the NCHS ERB as CCQDER anticipates most cognitive interviews will occur over the phone or via video conferencing, see Question 2 in Supplementary Statement B) after being read a script that that presents privacy and confidentiality information (Attachment D). For both activities in this information collection request, materials and scripts will include all elements of informed consent, including the purpose of the data collection, the voluntary nature of the study, audio or video recording of the interview, and the effect upon the respondent for terminating the interview at any time.
Across both activities requested in this ICR, confidentiality provided to respondents is assured by adherence to Section 308(d) of the Public Health Service Act (42 U.S.C. 242m) which states:
"No information, if an establishment or person supplying the information or described in it is identifiable, obtained in the course of activities undertaken or supported under section...306 (NCHS legislation),...may be used for any purpose other than the purpose for which it was supplied unless such establishment or person has consented (as determined under regulations of the Secretary) to its use for such other purpose and (1) in the case of information obtained in the course of health statistical or epidemiological activities under section...306, such information may not be published or released in other form if the particular establishment or person supplying the information or described in it is identifiable unless such establishment or person has consented (as determined under regulations of the Secretary) to its publication or release in other form,..."
In addition, legislation covering confidentiality is provided according to section 513 of the Confidential Information Protection and Statistical Efficiency Act or CIPSEA, which states:
“Whoever, being an officer, employee, or agent of an agency acquiring information for exclusively statistical purposes, having taken and subscribed the oath of office, or having sworn to observe the limitations imposed by section 512, comes into possession of such information by reason of his or her being an officer, employee, or agent and, knowing that the disclosure of the specific information is prohibited under the provisions of this title, willfully discloses the information in any manner to a person or agency not entitled to receive it, shall be guilty of a class E felony and imprisoned for not more than 5 years, or fined not more than $250,000, or both.”
RANDS during COVID-19 Information in Identifiable Form
NORC has access to Information in Identifiable Form (IIF) for their panel membership, including information such as name, date of birth, mailing address, and phone numbers. They collect this information in order to maintain their propriety panel. However, no direct IIF is associated with the RANDS data collection, and no direct personally identifiable information is transmitted to NCHS from NORC’s servers. NORC has extensive cyber and physical security in place, including a CIPSEA Information Protection Plan approved by the NCHS Confidentiality Officer and the NCHS Information Systems Security Officer (Attachment E), in order to protect both the security of the front-end survey interface and the back-end storage of the survey’s data. In order to comply with this information protection plan, a sample from the panels is pulled from NORC’s server and placed on a separate server share. At this point, all direct identifiers are removed and NORC is unable to directly link RANDS information to individual AmeriSpeak or TrueNorth panelists. Additionally, as subcontractors to NCHS (via CCQDER’s contract with Swan Solutions) all NORC employees working on the RANDS during COVID-19 will complete NCHS confidentiality training, sign the NCHS affidavit of nondisclosure (see Attachment F), and will be NCHS designated agents via the Designated Agent Agreement between Swan Solutions, LLC and NCHS.
While no direct PII will be transmitted to NCHS from NORC, some IIF will be collected in RANDS and securely sent to NCHS as part of the final survey files. All of these items have been routinely approved and collected in the past for other NCHS information collections. The identifiable information includes:
Date of birth
State/Territory
Medical information
Information in Identifiable Form from Cognitive Interviews
Data in identifiable form is collected for linkage of the respondent data collection form and audio and video recordings. The CCQDER also uses some identifiable data (name, phone number, email address) to contact previous respondents for CCQDER studies. The ability to match respondents to other data (respondent demographics, and audio/video recordings) greatly expands the usefulness of the data at a very low cost.
Access to personal information is restricted to CCQDER staff and designated agents (who have signed a Designated Agent Agreement) who can only access the personal information for statistical purposes.
General Privacy and Confidentiality Protection Procedures for RANDS during COVID-19 and Cognitive Interviews
The collection of information in identifiable form across both activities encompassed by this ICR requires strong measures to ensure that private information is not disclosed in a breach of confidentiality. Only those NCHS employees, those specially designated agents (including Swan Solutions and NORC staff), and research partners who must use the personal information for a specific purpose can use such data. Furthermore, storage of CCQDER’s confidential data is protected through procedures such as an internal QDRL LAN, passwords and restricted access.
As noted above, all NCHS employees as well as all contract staff, receive appropriate training and sign a “Nondisclosure Statement.” Staff from collaborating agencies are also required to sign this statement, and members of outside agencies are required to enter into a more formal agreement with NCHS. Everyone else who uses RANDS data can do so only after all identifiable information is removed (as described below). In addition, the Cybersecurity Act of 2015 permits monitoring information systems for the purpose of protecting a network from hacking, denial of service attacks and other security vulnerabilities5. Monitoring under the Cybersecurity Act may be done by a system owner or another entity the system owner allows to monitor its network and operate defensive measures on its behalf. The software used for monitoring may scan information that is transiting, stored on, or processed by the system. If the information triggers a cyber threat indicator, the information may be intercepted and reviewed for cyber threats. The cyber threat indicator or defensive measure taken to remove the threat may be shared with others only after any information not directly related to a cybersecurity threat has been removed. In addition, sharing of information can occur only after removal of personal information of a specific individual or information that identifies a specific individual.
It is NCHS policy to make RANDS data available via public use data files to the scientific community. Publicly released data sets will be available indefinitely on the NCHS website. A concerted effort is made to avoid any disclosures that may allow a researcher to go back and find individuals in the general population. To this end, prior to their release, the RANDS data files will be reviewed by the NCHS Disclosure Review Board to evaluate tabulations of data estimates along with the survey methods in order to determine where disclosure risks might arise and how to minimize them. Several techniques are used to minimize these risks, including collapsing categories, top and bottom coding, adding noise to variables, removing detailed geographic information that may allow someone to identify individuals in the general population, along with other statistically sound means. Researchers wishing to conduct analysis on variables not available in the public use data files may submit a research proposal to use the NCHS Research Data Center6.
The CIPSEA legislation authorizes the designation of agents (“designated agents” or “agents”) to perform statistical activities on behalf of an agency. These agents function under the supervision of the agency’s employees and are subject to the same provisions of law with regard to confidentiality as an agency’s employees. A Designated Agent Agreement between the agency and the designated agents (e.g. contractors) must be executed before the agents can acquire information for the agency for exclusively statistical purposes under a pledge of confidentiality. This requirement is outlined in an OMB Notice, published in the Federal Register on June 15, 2007, entitled “Implementation Guidance for Title V of the E-Government Act, Confidential Information Protection and Statistical Efficiency Act of 2002 (CIPSEA).” Additionally, the agents (contractors) will be required to complete the NCHS Confidentiality Training (https://www.cdc.gov/nchs/training/confidentiality/training/), submit a certificate of completion, and sign a pledge to maintain confidentiality (Nondisclosure Affidavit; see Attachment F) prior to completing work. If the contractor hires subcontractors to complete work, the subcontractors must adhere to the same confidentiality and security requirements as NCHS staff and contractors.
CCQDER-Specific Procedures for Safeguarding Cognitive Interviewing Data
As CCQDER collects IIF that includes video and voice data, CCQDER has established a
routine set of administrative, technical, and physical measures to safeguard confidentiality. These have been approved in CCQDER’s existing generic clearance (0920-0222, expiration 8/31/2021), and include the following:
Storage of confidential data (informed consent form, respondent database, video and audio recordings) on the QDRL LAN are protected through procedures such as an internal QDRL LAN, passwords, and carefully restricted access;
The QDRL LAN is not located on the NCHS LAN, the QDRL LAN is inaccessible to others (not CCQDER staff) inside or outside NCHS;
All CCQDER personnel (including CCQDER contractors/designated agents) who have access to confidential data complete NCHS Confidentiality Training (https://www.cdc.gov/nchs/training/confidentiality/training/), submit a certificate of completion, and sign a pledge to maintain confidentiality (Nondisclosure Affidavit; see Attachment F), and are given instruction by the CCQDER Laboratory Manager on the requirement to protect confidentiality. Contracted personnel send hardcopies of the NCHS Confidentiality Training certificates and original signed hardcopies of the Nondisclosure Affidavits to Karen Whitaker, QDRL Program Specialist/Contracting Officer Representative (COR);
Only such authorized CCQDER personnel are allowed access to confidential data, and only when their work requires it. CCQDER Personnel holding proper passwords may access the QDRL LAN through their CCQDER Computer Desk Top which is hardwired to the QDRL LAN;
Data from cognitive interviews that are not conducted in the physical QDRL are stored in a secured travel case to ensure that there is no loss in transit until returned to NCHS, at which point the data is stored in secure conditions (CCQDER Control Room or locked staff office in a locked drawer) until the recordings can be manually ingested;
NCHS government issued encrypted laptops will be used to video or audio record interviews conducted off-site. Due to the size of the video recordings, the internal drive of the encrypted laptop is not sufficient for storage of the recordings. Recordings will be saved to an NCHS government issued encrypted flash drive. The encrypted flash drive is FIPS 140-2 compliant and approved for use by CDC’s OCISO.
Extreme care will be taken with all recordings and paperwork from interviews conducted outside of the QDRL. Recordings and any identifying paperwork will be stored in a secured travel case until returned to NCHS, at which point they will be transferred to the usual secured locked storage cabinets. Once the video and audio recordings are transferred to the QDRL Network, the recordings will be deleted from encrypted flash drive. Once deleted, the files are no longer available for use.
QDRL Lab Access Protocol
This Lab is a restricted access secure facility. Access is by CCQDER staff or by CCQDER staff escort only.
Should access to the Lab be needed by anyone other than CCQDER staff members for any reason including emergency please adhere to the following:
During normal working hours contact one of the following CCQDER staff who will provide escorted access to the Lab.
Sean Murphy x4391 Mobile: 202-503-0321
Kristen Miller x4625 Mobile: 301-275-8182
Karen Whitaker x4569 Mobile: 410-212-3643
During off hours, weekends, and holidays contact one of the following CCQDER staff:
Sean Murphy Mobile 202-503-0321
Lee Burch Mobile 301-233-0311
Kristen Miller Mobile 301-275-8182
RANDS during COVID-19 Cognitive Interviewing Project Data Controls and Retention
In accordance with the CCQDER Data Storage and Access Policy (Attachment G), CCQDER has determined that this project falls in the “unrestricted” category. Therefore, interviews will be video or audio recorded, and researchers from CCQDER, and staff from NCHS and CDC who are working on the project, will use the recordings for research purposes. Non-CCQDER NCHS and CDC staff viewing/listening to recordings in the QDRL under CCQDER supervision have read and signed a non-disclosure affidavit.
Video or audio recording is required for this project except in the rare case that a study participant initially agrees to be video recorded during the telephone screening, but changes their mind. In that case, the respondent will be asked if they agree to be audio recorded. If they decline to be audio recorded the interview will proceed without recording. In this case the interviewer will depend on their handwritten notes when conducting analysis.
NCHS government issued encrypted laptops will be used to video and audio record the interviews conducted off-site. Due to the size of the video recordings, the internal drive of the encrypted laptop is not sufficient for storage of the recordings. Recordings will be saved to an NCHS government issued encrypted flash drive. The encrypted flash drive is FIPS 140-2 compliant and approved for use by CDC’s OCISO.
CCQDER staff will also use the NCHS government issued encrypted laptops to input their interviewer notes into Q-Notes, CCQDER’s cognitive interview qualitative data analysis software, which is hosted on CDC’s servers7. Within 24 hours, a CCQDER staff member will review interview notes and will delete any direct or indirect PII if found.
Extreme care will be taken with all recordings and paperwork from the interviews conducted off-site. Recordings and identifying paperwork will be stored in a secured travel case until returned to NCHS, at which point they will be transferred to the usual secured locked storage cabinets. Once the video and audio recordings are transferred to the QDRL Network, the recordings will be deleted from encrypted flash drive. Once deleted, the files are no longer available for use.
Initial retention period of the audio/video recordings is 5 years after project completion. After the initial retention period, the recordings will be re-evaluated by the CCQDER Director to determine relevance, ongoing usefulness, and qualitative value for likely use in question evaluation research. If it is determined by the CCQDER Director in conjunction with CCQDER project-relevant staff that there is no valid reason to retain the recording, it will be destroyed by designated CCQDER staff. If the interview continues to be of value (defined as ongoing use by research staff, topic relevance, likely use for federal questions evaluation research), reassessment of the recording will occur again in 5 years.
11. Institutional Review Board (IRB) and Justification for Sensitive Questions
This research is being conducted under protocol #2016-16 Laboratory Based Questionnaire Design (CCQDER), which was approved by the NCHS Research Ethics Review Board on October 25, 2019 (Attachment H).
Given the fact that RANDS during COVID-19 is designed to survey the public about an ongoing pandemic that has not only affected day-to-day life in the United States, but also lead to a large number of deaths, some of the survey’s (and the accompanying cognitive interviews’) topics may include potentially sensitive questions for some respondents. However, the potential sensitivity of questions was an evaluation criterion in determining content of the survey. The multi-purpose nature of the RANDS makes it necessary to exclude topics so sensitive that they may interfere with participation. No topics that have been deemed to be universally or extremely sensitive have been included in the questionnaire (Attachment B).
In the informed consent procedure for both activities, participants are advised of the voluntary nature of their participation in the survey or any of its components. Sample persons are informed that they can choose not to answer any questions they do not wish to answer and that they may stop the interview at any time.
12. Estimates of Annualized Burden hours and costs:
The estimated overall average annual burden for 2020, including the both rounds of the RANDS during COVID-19 survey and the cognitive interviews is 7,447 hours. This project is not expected to extend beyond 2020. Any future modification that might impact the instruments and/or burden estimates will be submitted as a non-substantive change request for OMB review, as applicable.
For cognitive interviews conducted at NCHS, time required to travel to the lab is not covered, because distances and modes of transportation are unknown. No retrieval of information by respondents for either activity is anticipated; although it is possible that validation of data at some point may require respondents to check records. In that case, the study will be designed so that the response time includes record retrieval. All estimates are based on NCHS' past experience (1988 through 2020).
Estimated Annualized Burden Table
Types of Respondents |
Form Name |
Number of Participants |
Number of Responses/ Participant |
Average hours per response |
Response Burden (in hours) |
Individuals or households |
RANDS during Covid Round 1 |
12,000 |
1 |
20/60 |
4,000 |
Individuals or households |
RANDS during Covid Round 2 |
10,000 |
1 |
20/60 |
3,334 |
Individuals or households |
Screener (recruited from newspaper/flyer) |
150 |
1 |
5/60 |
13 |
Individuals or households |
Questionnaire |
100 |
1 |
55/60 |
92 |
Individuals or households |
Respondent Data Collection Sheet |
100 |
1 |
5/60 |
8 |
Total |
|
22,350 |
|
|
7,447 |
Estimated Annualized Burden Costs to Respondents.
The average annual response burden cost for the CCQDER is estimated to be $185,702.38. The hourly wage estimate is based on the Bureau of Labor Statistics May 2016 National Occupational Employment and Wage Estimates (http://www.bls.gov/oes/current/oes_nat.htm). There is no cost to respondents other than their time to participate.
Type of Respondent |
Form Name |
Total Burden Hours |
Hourly Wage Rate |
Total Respondent Costs |
Individuals or households |
RANDS during Covid Round 1 |
4,000 |
$23.86 |
$95,440.00 |
Individuals or households |
RANDS during Covid Round 2 |
3,334 |
$23.86 |
$79,549.24 |
Individuals or households |
Screener (recruited from newspaper/flyer) |
13 |
$23.86 |
$310.18 |
Individuals or households |
Questionnaire |
92 |
$23.86 |
$2,195.12 |
Individuals or households |
Respondent Data |
8 |
$23.86 |
$190.88 |
Total |
|
|
|
$177,685.42 |
13. Estimates of Other Total Annual Cost Burden to Respondents and Record keepers
None.
14. Annualized Costs to the Federal Government
The estimated annualized cost to the federal government for the activities outlined in this information collection request is $1,290,859.94.
Expense |
Cost |
RANDS NORC Contract |
$819,000.00 |
RANDS Planning/Analysis Staff Time |
$157,363.14 |
Cog Interviewing Project Planning/Conduct/Analysis |
$74,499.34 |
Contract Staff Cost for both RANDS and Cognitive Interviewing Project (including remuneration for cognitive interviews) |
$239,997.46 |
Total |
$1,290,859.94 |
15. Explanation for Program Changes or Adjustments
This is a new emergency clearance request. Previous rounds of both the RANDS survey and associated cognitive interviewing projects have been conducted under CCQDER’s generic clearance (OMB# 0920-0222, current expiration: 08/31/2021). RANDS during COVID-19 is being submitted as a new emergency ICR under the advice of NCHS’ OIRA desk officer and in order to conceptually link this work with other COVID-19-related emergency work from the Centers for Disease Control and Prevention.
This ICR is requesting a total of 7,447 burden hours.
16. Plans for Tabulation and Publication and Project Time Schedule
The following are key activities and projected completion dates for the RANDS during COVID-19 project:
Activity |
Projected Completion Date |
Round 1 Survey Data Collection |
Three weeks after OMB approval |
Round 2 Survey Data Collection |
11 weeks after OMB approval |
Cognitive Interviews |
12 weeks after OMB approval |
RANDS during COVID Data File Available |
Five months after OMB approval |
Publication of Cognitive Interviewing Report |
Five months after OMB approval |
17. Reason(s) Display of OMB Expiration Date is Inappropriate
The expiration date will be displayed.
18. Exceptions to Certification for Paperwork Reduction Act Submissions
The certifications are included in this submission.
1 See for instance, Scanlon, Paul (2019). The Effects of Embedding Closed-ended Cognitive Probes in a Web Survey on Survey Response. Field Methods 31(4): 328-343. doi: https://doi.org/10.1177/1525822X19871546
4 Information on NORC’s TrueNorth approach is at http://amerispeak.norc.org/our-capabilities/Pages/TrueNorth.aspx.
5 To “monitor” means “to acquire, identify, or scan, or to possess, information that is stored on, processed by, or transiting an information system”; “information system” means “a discrete set of information resources organized for the collection, processing, maintenance, use, sharing, dissemination or disposition of information;” “cyber threat indicator” means information that is necessary to describe or identify security vulnerabilities of an information system, enable the exploitation of a security vulnerability, or unauthorized remote access or use of an information system.
6 Procedures for submitting the proposal and other important information can be found here http://www.cdc.gov/rdc/
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Whitaker, Karen R. (CDC/DDPHSS/NCHS/DRM) |
| File Modified | 0000-00-00 |
| File Created | 2021-01-14 |