Behavioral Risk Factor Surveillance System (BRFSS) Asthma Call-back Survey (ACBS)
OMB Control No. 0920-1204, Exp. Date 11/30/2020
Supporting Statement
Revision
Part B: Collections of Information Employing Statistical Methods
August 5, 2020
Xiaoting Qin
Centers for Disease Control and Prevention
National Center for Environmental Health
Environmental Hazards and Health Effects
Asthma and Community Health Branch
4770 Buford Highway F- 60
Atlanta, GA 30341
Xqin@cdc.gov
770-488-0797
Table of Contents
B. Collections of Information Employing Statistical Methods 3
B.1 Respondent Universe and Sampling Methods 4
B.2 Procedures for the Collection of Information 5
B.3 Methods to Maximize Response Rates and Deal with Nonresponse 10
B.4 Tests of Procedures or Methods to be Undertaken 11
B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 12
The Behavioral Risk Factor Surveillance System (BRFSS) is a collaborative project of the Centers for Disease Control and Prevention (CDC) and U.S. states, the District of Columbia, and U.S. territories (collectively called “states” or “jurisdictions” in this document). The BRFSS is a coordinated series of interviews that collects information about preventive health practices and behavioral risk factors that are linked to chronic diseases, injuries, and preventable infectious diseases. Respondents are adults, ages > 18 years. Information collection is conducted annually.
The Asthma Call-back Survey (ACBS) is an ongoing data collection administered on behalf of the National Asthma Control Program (NACP), by CDC’s National Center for Chronic Disease Prevention and Health Promotion (NCCDPHP) through their BRFSS cooperative agreement with state health departments under CDC-RFA DP15-1513. The respondent will be either an adult (BRFSS respondent) or child (chosen using the BRFSS Random Child Selection and Childhood Asthma Prevalence modules) who has ever had asthma. The ACBS sample includes all cases meeting the qualification criteria in BRFSS. ACBS interviewers will only conduct one call-back interview per household. For a state to include children in the ACBS, it must have the Random Child Selection and Childhood Asthma Prevalence modules in the BRFSS. If a randomly selected child with response on the BRFSS has a diagnosis of asthma, then he/she is eligible for the ACBS. If both the randomly selected child and adult responding to the BRFSS have a diagnosis of asthma, then one is eligible for the ACBS through a 75/25 or 100/0 split.
ACBS is a unique collaboration between the federal government and states. It is highly responsive to diverse needs and priorities for states, the federal government, and non-governmental agencies. The decisions about the ACBS encompass consideration of need for all these partners. All ACBS collaborators (See Supporting Statement A Section A.8 Table 8.1) recognize the need for a high level of data standardization while still being flexible to meet the asthma surveillance needs of individual states. For example, a regular monthly conference meeting among the CDC project officers, data collectors, and state BRFSS coordinators helps to establish agreed-upon protocols for data collection.
Respondent Universe
CDC’s NCCDPHP Division of Population Health administers the BRFSS parent survey, which provides the foundation for the ACBS administration and data collection (Attachment 7). Since the ACBS sample is a subset of BRFSS sponsored by the NACP, respondents for the ACBS are BRFSS adults (18 years of age or older) living in private households or college housing in participating states who report ever being diagnosed with asthma. Some states include children, under 18 years of age, who are randomly selected subjects in the BRFSS household. In participating states, parents or guardians serve as ACBS proxy respondents for their children ever diagnosed with asthma. Children do not respond directly to the ACBS questionnaire. If a randomly selected child with response on the BRFSS has a diagnosis of asthma, then he/she is eligible for the ACBS. If both the randomly selected child and adult responding to the BRFSS have a diagnosis of asthma, then only one is eligible for the ACBS through a (75% Children vs 25% Adult) or (100% Children only) split. The ACBS sampling enrollment process is presented in Attachment 9.
ACBS State-tailored Samples
BRFSS data collection is conducted through telephone interviews (except in very limited circumstances in which interviews must be conducted in-person). An independent sample is drawn for each state by the state’s health department. A sample record is one telephone number in the list of all telephone numbers selected for dialing. To meet the BRFSS sample design standards, sample records must be justifiable as a probability sample of all persons with telephones in the state. The BRFSS sample is randomly selected from working phone numbers within each jurisdiction. CDC uses an overlapping sample of landline and cell phone numbers. No direct method of accounting for non-telephone coverage is employed for the BRFSS. The size of each state sample is estimated according to the number of completed interviews for the previous year but may be adjusted depending on state objectives and funding. States may subdivide their state by geographic region/geostrata (such as public health districts, counties or groups of counties). States may also target population groups within their sample.
The size of each state’s ACBS sample varies based on their BRFSS sample size and can be estimated as follows.
Using State X’s 2016 BRFSS sample size of 5,000 adults:
Assuming lifetime prevalence (weighted percentage) in State X is 13 percent for 2013 and the sample size is 5,000 adults, the number of adults eligible for the ACBS in State X will be 650 (13.0% x 5,000). The lifetime prevalence is the weighted population average based on 33 participating states.
Considering State X’s CASRO response rate of 45 percent, about 293 (45% x 650) adults will complete the ACBS (CASRO response rate see Attachment 13)
For children, a similar calculation is used:
Approximately 1/3rd of the 5,000 adults will have children (1,650).
Using State X’s child lifetime weighted percentage rate of 11.8 percent, approximately 195 of the 1,650 children will have lifetime asthma.
Using a 45 percent response rate, State X would have approximately 90 child completed interviews.
To ensure an adequate number of responses for weighting purposes, the state must have at least 75 completed child interviews. If a state has less than 75 records for child data, the data are not weighted and the data from that state is excluded from the public release file. In this case, the data is combined with multiple subsequent years and weighted to obtain reliable estimation.
The ACBS data collection follows all standard BRFSS data collection protocols (such as call attempts, assigning dispositions to cases, etc.). Data collection for ACBS must meet guidelines and data quality criteria established for the annual state-wide survey.
The ACBS steps, roles, and responsibilities are described below:
CDC annually provides the ACBS questionnaire and compiles requests regarding the questionnaire modifications from states and sends the requests to the questionnaire work group. This work group is comprised of state epidemiologists and CDC Health Scientists, who review and vote on proposed changes and new questions. Questionnaire revisions are described in the supporting documents part A1.15 (Table 15.1, Table 15.2). All states use the same ACBS questionnaire. CDC also produces data processing layouts (Attachment 5g-5h).
Information collection is conducted by telephone interview. CDC provides Computer-Assisted Telephone Interviewing (CATI) programming to states for their use. States may also opt to use their own CATI programming software.
ACBS recipients are responsible for field operations and to determine how their data will be collected within the BRFSS (Attachment 6) and ACBS guidelines (Attachment 4). States may collect data using in-house calling centers, hire vendors using RFP procedures, or contract with universities. The data collector is the same for BRFSS and ACBS. Data collectors must develop and maintain procedures to ensure respondents’ privacy, assure and document the quality of the interviewing process, and supervise and monitor the interviewers. Files containing phone numbers must be maintained separately from any files containing responses.
States submit de-identified data files to CDC on a monthly or quarterly basis for cleaning and weighting. The CDC BRFSS ACBS operation team returns clean, weighted data files to the state of origin for its use. Through the BRFSS website, BRFSS ACBS operation team also makes cleaned subsets of state data files available for public use, along with information about the source of sample (landline or cell phone), weighting, and any restrictions on publication or use of the (https://www.cdc.gov/brfss/acbs/index.htm).
ACBS Screener and Permission for Adults and Children:
The ACBS subset is selected from the BRFSS respondent pool. All BRFSS adults, 18 years and older, in participating states who report ever being diagnosed with asthma are eligible for ACBS. If a child is the selected sample member for the ACBS, the interview will be conducted with the most knowledgeable parent or guardian in the household; persons under age 18 years are not interviewed directly. A BRFSS respondent at the core must be the parent/guardian of the selected child. If the BRFSS respondent is not the parent/guardian of the selected child, an ACBS survey for the child with asthma is not to be conducted. For example, a core BRFSS respondent who is a sibling of the selected child, over 18 years but is not the guardian of the selected child could not transfer the child ACBS over to the parent/guardian of the child. The reason for this is that the core BRFSS data must also be for the parent/guardian of the selected child. However, as a new revision, the ACBS proposes to allow the parent/guardian of the child to transfer the interview to the Most Knowledgeable Person (MKP) and grant this person permission to conduct the interview. Beginning in 2021, the State should identify the MKP only using the BRFSS state-added questions. This option will be removed from the ACBS in this revision (see Supporting Statement A Section A15 for details).
All states inform the BRFSS respondent of the request to participate in the ACBS during the BRFSS interview. A template with recommended wording for the question requesting permission to call the respondent back some time in the next two days is provided by the CDC to the states. States may require slight changes in the wording of this question, based on their institutional review board (IRB). The ACBS screeners are provided in Attachments 5a–5d.
ACBS Questionnaire:
The ACBS questionnaire has two versions, one for the adult eligible respondents (Attachment 5e), and one for child eligible respondents (Attachment 5f). The ACBS was pre-tested as the National Asthma Survey (NAS) in 2003, and administered in three states during 2005, 25 states in 2006, 33 states or justification in 2016, and 27 states or jurisdictions in 2020. It has been running consecutively for 15 years. California and Puerto Rico provide a Spanish translation of each instrument. Questionnaire changes compared to the last OMB approved version was provided in Support Statement part A (Table 15.1; Table 15.2).
The NAS initial questionnaires were designed by the NACP and further refined based on many questions included in other national surveys (National Health Interview Survey [NHIS], National Health and Nutrition Examination Survey [NHANES]) and BRFSS to facilitate comparison and because many of these questions had already undergone extensive testing. Some questions were written to translate the National Asthma Education and Prevention Program (NAEPP) guidelines into practice. Cognitive testing and the series of four pretests (in 2003) are described in Attachment 15 (page 6).
ACBS Call/Interview Guidelines:
All standard BRFSS data collection protocols (such as call attempts, assigning dispositions to cases, etc.) are followed. Data collection for the ACBS typically starts by February 1. The is typically conducted within two days of the BRFSS interview completion date. If the respondent is willing to participate immediately after the BRFSS survey, the ACBS interview can be conducted.
The BRFSS protocol suggests up to 15 calling attempts for each landline phone number and up to 8 for each cell phone number in the sample, depending on state regulations for calling and outcomes of previous calling attempts. Some states make calling attempts over the totals suggested by the BRFSS protocol. Although states may have some flexibility in distribution of calling times, in general, surveys are conducted using the following calling occasions:
Conduct 20 percent of the landline interviews on weekdays (prior to 5:00 pm)
Conduct 80 percent of the landline interviews on weeknights (after 5:00 pm) and weekends
Conduct cell phone interviews during all three calling occasions (weekday, weeknight and weekend) approximately 30 percent of cell phone calls on weekend calling occasions.
Change schedules to accommodate holidays and special events
Make weeknight calls just after the 5:00 pm
Make callbacks during hours that are not scheduled for other interviews, generally on weekdays
Except for verbally abusive respondents, eligible persons who initially refuse to be interviewed may be contacted at least one additional time and given the opportunity to be interviewed. Preferably, this second contact will be made by a supervisor or a different interviewer. Some states have regulations on whether refusals should be called again.
Adhere to respondents’ requests for specific callback times whenever possible
ACBS Call Disposition:
States are required to give a final disposition for every number in the ACBS eligible sample. Each BRFSS asthma eligible respondents which agree to participate in the ACBS must be assigned a final disposition code, to indicate a particular result of calling the number:
A completed or partially completed interview; or
A determination that: a BRFSS respondent was eligible to be included, but an interview was not completed; or a BRFSS respondent was ineligible or eligibility could not be determined
Procedures for call dispositions that ACBS follows the BRFSS procedures (Attachment 10). However, some additions and adaptations are required for the ACBS survey. ACBS interviews are considered complete (COIN) if the respondents finish the entire interview or partial complete if they progressed through Section 8 (medication) of the ACBS interview. ACBS interviews are refusals if the respondent refuses participation at either the BRFSS interview or the time of the ACBS interview. Terminations are interviews that start the ACBS survey but are terminated during the interview before completing Section 8.
Final disposition codes are then used to calculate response rates, cooperation rates, and refusal rates (Attachment 11, 12). The distribution of individual disposition codes and the rates of cooperation, refusal, and response are published annually in the Summary Data Quality Reports. The ACBS uses standards set by the American Association of Public Opinion Research (AAPOR) [2] and Council of American Survey Research Organizations (CASRO) [3] to determine disposition codes and response rates. Data collectors must adhere to the rules for assigning disposition codes, and train and monitor interviewers in the use of specific dispositions.
ACBS Procedures to Promote Data Quality and Comparability:
In order to maintain consistency across states and allow for state-to-state comparisons, the BRFSS and ACBS sets standard protocols for data collection which all states are encouraged to adopt with technical assistance provided by CDC. ACBS follows the BRFSS procedures to promote data quality and comparability, with minor revisions, in order to maintain consistency across states and allow for state-to-state comparisons. The following items are included in the BRFSS survey protocol:
All states must ask the same questions without modification. States may choose to add state-added questions after the BRFSS core component. Interviewers may not offer information to respondents on the meaning of questions, words or phrases beyond the interviewer instructions provided by CDC and/or the state BRFSS coordinators.
Interviewers should be trained specifically for the ACBS and retrained each year.
Systematic, unobtrusive electronic monitoring is a routine and integral part of monthly survey procedures for all interviewers. States may also use recheck verification procedures to ensure data quality. Unless electronic monitoring of 10% of all interviews is being routinely conducted, a 5% random sample of each month’s interviews must be recalled verifying selected responses for quality assurance.
General calling rules, listed below, are established by the CDC ACBS operation team and states are encouraged to adhere to them whenever possible. It is understood that the calling rules are not universally applicable to each state.
All cellular telephone numbers must be hand-dialed.
If possible, calls made to non-English speaking households and assigned the interim disposition code of 5330 (household language barrier), should be attempted again with an interviewer who is fluent in the household language (e.g. Spanish).
States should maximize calling attempts as outlined in Attachment 6. The maximum number of attempts (15 for landline telephone and 8 for cellular telephone) may be exceeded if formal appointments are made with potential respondents.
Calling attempts should allow for a minimum of 6 rings and up to 10 rings if not answered or diverted to answering devices.
The ACBS produces a Summary Data Quality Report which is published annually on its websites. The report includes information on sample quality, rates of completion and response. The 2016 Summary Data Quality Report is provided in Attachment 11.
The ACBS uses numerous techniques to deal with response rates and nonresponse. These techniques include providing the interview in languages other than English, creating a number of protocols designed to convert refusals, and alternating times and days of calling attempts. States get permission from BRFSS respondents to call them back during the BRFSS survey. Experienced interviewers are used for callbacks when respondents initially refuse to take part in the survey. Hard refusals (where potential respondents state that they are not interested in completing the interview) are not called back.
States must maintain training for all interviewers involved in the ACBS. Issues related to response rates are discussed in large annual meetings of the data collectors. Data collectors also participate in monthly conference calls organized by the CDC to discuss best practices and share experiences.
The CDC has conducted numerous pilot studies in recent years to identify methods that might improve response rates and alleviate potential nonresponse bias. These pilot studies have included comparing the response patterns between landline and cell phone samples for the BRFSS and ACBS and assessing how the lag days between BRFSS and ACBS interviews affect ACBS response rates. This pilot study indicated that ACBS response rate for landline and cell phone response were highest if the callback was within 2 days of BRFSS interviews (92.3 vs. 88.8 percent). As lag days increased, the response rate decreased [4]. Based on results of these pilot studies, the 2020 ACBS protocol requires that the ACBS should be called within two days of the BRFSS interview complete.
The ACBS child survey identifies eligible asthma children based on the BRFSS Random Child Selection and childhood Asthma Prevalence modules. The most knowledge person (MKP) will answer the child survey, the parents or guardian can be the most knowledge person or point to a most knowledge adult to conduct the child ACBS interview. Two scripts were provided to states before 2019. States can choose to identify the most knowledge adult at the end of BRFSS survey, or at beginning of child ACBS interview. By comparing the response rate of two scripts, identifying the most knowledge person at the beginning of ACBS has the lowest response rate (around 5%) compared to identifying at the end of BRFSS. Therefore, a new protocol (starting in 2020) proposed that states identify the most knowledge person at the end of BRFSS and record it at state add questions section on the ACBS survey.
A pilot study was conducted for 6 states. The ACBS interview was conducted immediately after completing the BRFSS interview, if the BRFSS respondent agreed to do so. For 2015 and 2016, these 6 pilot states have significantly high CASRO (average 91%) compared to the states conducting ACBS within two weeks (average 45%). Therefore, according to the 2020 protocol, states can choose to conduct the ACBS interview immediately after the BRFSS interview if the respondent agrees.
Response rates, cooperation rates, and refusal rates for BRFSS are calculated and published annually using standards set by the American Association of Public Opinion Research (AAPOR) [2]. The BRFSS calculates response rates using AAPOR Response Rate #4, which is in keeping with rates provided by BRFSS prior to 2011, using rates from the Council of American Survey Research Organizations (CASRO) [3]. Response rates, cooperation rates, and refusal rates for ACBS are calculated using a formula based on BRFSS (Attachment 12). See the 2016 Asthma Call-Back Survey (ACBS) Summary Data Quality Report for specific details (Attachment 14).
To control potential nonresponse bias, the annual data quality report released with the public use dataset (https://www.cdc.gov/brfss/acbs/2016/pdf/sdq_report_acbs_16-508.pdf) has two added data tables following adult and child participants since 2015. The first table reports ACBS unweighted and weighted percentages for ACBS final completes, BRFSS-eligible asthma respondents but nonresponding to ACBS, and BRFSS-eligible asthma respondents by age, sex, and race/ethnicity demographic categories for each state. The second table reports estimated current asthma percentage among individuals who have ever been diagnosed with asthma. These two tables will help communicate the potential impact of nonresponse bias on the ACBS published dataset. Annual ACBS risk factors prevalence tables have included a footnote with the URL to the nonresponse report, to communicate potential nonresponse bias associated with the data at state level. This information will more clearly communicate the caveats of state-to-state comparisons.
ACBS data collection procedures have been adapted over time to meet the needs of the data collection process and maximize response rates, while minimizing respondent burden. The pretest of questionnaires every year is not necessary as changes are not always made. As this document indicates, subject matter experts from CDC and other federal agencies, state health department representatives, and survey experts are involved in the process of question development. Some of the questions which are included in the ACBS appear on the National Health Interview Survey or National Survey of Children’s Health. The use of identical or similar questions is advantageous in that it allows researchers to make comparisons across different samples, different geographic areas, or over time.
CDC personnel are responsible for all statistical aspects of the BRFSS including data analyses and reporting. The following staff members are primarily responsible for ACBS data reporting.
Name |
Title |
Phone |
|
Xiaoting Qin |
Survey Statistician |
770-488-0797 |
|
Zahran, Hatice S. |
Team Lead |
770-488-1509 |
|
Malilay, Josephine |
Branch Chief |
770-488-3465 |
In addition, staff members attend the annual meetings of the American Association of Public Opinion Research (AAPOR) and the Joint Statistical Meetings (JSM). These meetings experts provide guidance, comments, and suggestions on staff methodological research presentations.
Table B.5.1 lists individuals involved in the data collection in partnership with the CDC Division of Population Health, National Center for Chronic Disease Prevention, the program responsible for implementing the BRFSS and ACBS survey.
Table B.5.1 CDC Consultants
CDC |
CDC |
Wilmon Murphy Programmer and Project Manager Northrop Grumman Contractor Centers for Disease Control and Prevention National Center for Chronic Disease Prevention Division of Population Health Population Health Surveillance Branch 4770 Buford Hwy MS F-78 Atlanta, GA 30341 |
Ernest Kelly Software Development Analyst Northrop Grumman Contractor Centers for Disease Control and Prevention National Center for Chronic Disease Prevention Division of Population Health Population Health Surveillance Branch 4770 Buford Hwy MS F-78 Atlanta, GA 30341 |
Ajay Sharma Software Development Analyst Northrop Grumman Contractor Centers for Disease Control and Prevention National Center for Chronic Disease Prevention Division of Population Health Population Health Surveillance Branch 4770 Buford Hwy MS F-78 Atlanta, GA 30341 |
Tatum, Sharquetta Statistician Northrop Grumman Contractor Centers for Disease Control and Prevention National Center for Chronic Disease Prevention Division of Population Health Population Health Surveillance Branch 4770 Buford Hwy MS F-78 Atlanta, GA 30341 |
CDC personnel and asthma grantee states were involved in a workgroup responsible for documenting statistical, analysis, and reporting aspects of the ACBS in a detailed user’s manual covering all sections of the survey. This manual is available by request to [email protected].
Behavioral Risk Factor Surveillance System Overview, 2018. https://www.cdc.gov/brfss/annual_data/2018/pdf/overview-2018-508.pdf
American Association for Public Opinion Research (AAPOR). (2016). “Standard definitions: final dispositions of case codes and outcome rates for surveys.” Available at: http://www.aapor.org/AAPORMain/media/publications/Standard-Definitions20169theditionfinal.pdf [Google Scholar]
The Council of American Survey Research Organizations. 2013. Code of standards and ethics for market, opinion, and social research, Available at:
Qin X, Bailey CM, Zahran HS. Comparison response patterns on landline and cell phone in a call back survey: effects of demographic characteristics and lag days. Survey Methods Insights Field. 2019:10.13094/SMIF-2019-00019. doi:10.13094/SMIF-2019-00019
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | cww6 |
| File Modified | 0000-00-00 |
| File Created | 2021-01-13 |