Download:
pdf |
pdfBehavioral Risk Factor Surveillance System
2019 Summary Data Quality Report
July 16, 2020
Page 1 of 26
Table of Contents
Introduction ............................................................................................................................................................. 3
Interpretation of BRFSS Response Rates ............................................................................................................... 3
BRFSS 2019 Call Outcome Measures and Response Rate Formulae .................................................................... 5
Tables of Outcomes and Rates by State................................................................................................................ 10
References ............................................................................................................................................................. 26
Page 2 of 26
Introduction
The Behavioral Risk Factor Surveillance System (BRFSS) is a state-based, CDC-assisted health-data collection
project and partnership of state health departments, CDC’s Division of Population Health, and other CDC
programs and offices. It comprises telephone surveys conducted by the health departments of all 50 states, the
District of Columbia, Puerto Rico, and Guam.
This Summary Data Quality Report presents detailed descriptions of the 2019 BRFSS calling outcomes and call
summary information for each of the states and territories that participated. All BRFSS public-use data are
collected by landline telephone and cellular telephone to produce a single data set aggregated from the 2019
BRFSS territorial- and state-level data sets. The variables and outcomes provided in this document are
applicable to a combined data set of responses from participants using landline telephones and cellular
telephones within each of the states and territories.
The inclusion of data from cellular telephone interviews in the BRFSS public release data set has been standard
protocol since 2011. In many respects, 2011 was a year of change—both in BRFSS’s approach and
methodology. As the results of cellular telephone interviews were added in 2011, so were new weighting
procedures that could accommodate the inclusion of new weighting variables. Data users should note that
weighting procedures are likely to affect trend lines when comparing BRFSS data collected before and after
2011. Because of these changes, users are advised NOT to make direct comparisons with pre-2011 data, and
instead, should begin new trend lines with that year. Details of changes beginning with the 2011 BRFSS are
provided in the Morbidity and Mortality Weekly Report (MMWR), which highlights weighting and coverage
effects on trend lines.1 Each year of data collection since 2011 has included a larger percentage of calls from the
cell phone sample. In 2019, a majority of the BRFSS interviews were conducted by cell phone. The annual code
books provide information on the number and percentage of calls conducted by landline and cell phone by year.
The measures presented in this document are designed to summarize the quality of the 2019 BRFSS survey
data. Response rates, cooperation rates, and refusal rates for BRFSS are calculated using standards set by the
American Association for Public Opinion Research (AAPOR).2 The BRFSS has calculated 2019 response rates
using AAPOR Response Rate #4, which is in keeping with rates provided by BRFSS before 2011 using rates
from the Council of American Survey Research Organizations (CASRO).3
On the basis of the AAPOR guidelines, response rate calculations include assumptions of eligibility among
potential respondents or households that are not interviewed. Changes in the geographic distribution of cellular
telephone numbers by telephone companies and the portability of landline telephone numbers are likely to make
it more difficult than in the past to ascertain which telephone numbers are out-of-sample and which telephone
numbers represent likely households. The BRFSS calculates likely households and eligible persons using the
proportions of eligible households/persons among all phone numbers where eligibility has been determined.
This eligibility factor appears in calculations of response, cooperation, resolution, and refusal rates.
Interpretation of BRFSS Response Rates
Because this report reflects the inclusion of BRFSS cellular telephone interviews, contextual information on
cellular telephone response rates is provided below. Although cellular telephone response rates are generally
Page 3 of 26
lower than landline telephone response rates across most surveys, the BRFSS has achieved a cellular telephone
response rate that compares favorably with other similar surveys (Table 1). Moreover, since the initial inclusion
of cell phone respondents, the proportion of the sample that is interviewed by cell phone has increased. In many
states, cell phone respondents are the majority of the sample. Since 2012, median BRFSS cell phone response
rates have risen slightly. Overall, BRFSS response rates have leveled off in the past few years, with landline
rates declining and cell phone rates improving. In 2019, the screening of eligible landline phone numbers has
improved—which may account for a slight improvement in the proportion of numbers identified as working
phone numbers in the landline sample. This change would not necessarily increase response rates. The levelingoff of telephone survey response rates is noted for other federal surveys as well.4
Table 1.
Examples of Survey Response Rates
Year(s)
Overall Response
Rates
California Health Interview Survey (CHIS)a
2017
7.1%
National Immunization Survey (NIS) b
2014
42.5%
2013-2014
36.1%
2019
49.4%
Survey
National Adult Tobacco Survey (NATS) c
BRFSS d
a CHIS
2017 Methodology Report Series. (2018)
https://healthpolicy.ucla.edu/chis/design/Documents/CHIS_2017_MethodologyReport4_ResponseRates.pdf
b
Unlike the BRFSS, the NIS does not include household sampling in the landline portion of the study but interviews the adult
who self-identifies as the most knowledgeable about household immunization information.
https://www.cdc.gov/nchs/data/series/sr_01/sr01_061.pdf
c https://www.cdc.gov/tobacco/data_statistics/surveys/nats/pdfs/2014-methodology-report-tag508.pdf
d BRFSS
response rates are presented here as median rates for all states and territories.
Research by the Pew Research Center indicates that response rates for all telephone-based surveys have
declined in recent years.5 Comparisons of federal surveys indicate that all surveys including the BRFSS have
experienced declining response rates in recent years.4 Generally, response rates are lower for telephone surveys
than for surveys conducted in person.5 Industry averages for response rates by in-person, telephone, mail and
online surveys average 57%, 18%, 50% and 29%, respectively.6 Despite lower response rates over time, this
research supports previous findings7 that weighting to demographic characteristics of respondents ensures
accurate estimates for most measures.
The following tables present landline telephone and cellular telephone calling outcomes and rates. The BRFSS
cellular telephone survey was collected in a manner similar to that of the BRFSS landline telephone survey. One
important difference, however, is that interviews conducted by landline telephones include random selection
among adults within households, while cellular telephone interviews are conducted with adults who are
contacted on personal (nonbusiness) cellular telephones. The report presents data on three general types of
measure by state:
1. Call outcome measures, including response rates, which are based on landline telephone disposition codes.
Page 4 of 26
2. Call outcome measures, including response rates, which are based on cellular telephone disposition codes.
3. A weighted response rate, based on a combination of the landline telephone response rate with the cellular
telephone response rate proportional to the total sample used to collect the data for a state.
For clarity, the BRFSS recommends that authors and researchers referencing BRFSS data quality include the
following language, below. Note the places where authors should include information specific to their projects.
Response rates for BRFSS are calculated using standards set by the American Association for Public Opinion Research
(AAPOR) Response Rate Formula #4 (http://www.aapor.org/AAPOR_Main/media/publications/StandardDefinitions20169theditionfinal.pdf). The response rate is the number of respondents who completed the survey as a
proportion of all eligible and likely-eligible people. The median survey response rate for all states, territories and
Washington, DC, in 2019 was 49.4 and ranged from 37.3 to 73.1.a Response rates for states and territories included in this
analysis had a median of [provide median] and ranged from [provide range],b For detailed information see the BRFSS
Summary Data Quality Report c
a
Response rates and ranges should reflect the year(s) included in the analyses.
Response rates for states selected for analysis should be included here. This sentence may be omitted if all states are used
in the analysis.
c
See the Summary Data Quality Report for the year(s) included in the analyses. The 2019 document is available at:
https://www.cdc.gov/brfss/annual_data/2019/pdf/2019-sdqr-508.pdf.
b
BRFSS 2019 Call Outcome Measures and Response Rate Formulae
The calculations of calling-outcome rates are based on final disposition codes that are assigned after all calling
attempts have been exhausted. The BRFSS may make up to 15 attempts to reach a respondent before assigning
a final disposition code. In 2019, the BRFSS used a single set of disposition codes for both landline and cell
phones, adapted from standardized AAPOR disposition codes for telephone surveys. A few disposition codes
apply only to landline telephone or to cellular telephone sample numbers. For example, answering-device
messages may confirm household eligibility for landline telephone numbers but are not used to determine
eligibility of cellular telephone numbers. Disposition codes reflect whether interviewers have completed or
partially completed an interview (1000 level codes), determined that the household was eligible without
completing an interview (2000 level codes), determined that a household or respondent was ineligible (4000
level codes), or was unable to determine the eligibility of a household or respondent (3000 level codes).
Partially completed interviews are those that have collected all information needed to weight responses (about
12 minutes into the survey questionnaire, not including time for eligibility screening). The table below
illustrates the codes used by the BRFSS in 2019, and it notes the instances where codes are used only for
landline telephone or cellular telephone sample numbers.
The Disposition Code Table below uses a number of terms to define and categorize outcomes. These include the
following:
•
•
Respondent: A person who is contacted by an interviewer and who may be eligible for interview.
Private residence: Persons residing in private residences or college housing are eligible. Persons living
in group homes, military barracks or other living arrangements are not eligible. Persons living in
vacation homes for 30 days or more are eligible. Eligibility is ascertained by asking each potential
Page 5 of 26
•
•
•
•
respondent whether they live in a private residence. If the respondent is unsure whether their residence
qualifies, additional definitions of residences are provided.
Landline telephone: A telephone that is used within a specific location, including traditional household
telephones, Voice Over Internet Protocol (VOIP), and Internet phones connected to computers in a
household.
Cellular telephone: A mobile device that is not tied to a specific location for use.
Selected respondent: A person who is eligible for interview. For the cellular telephone sample, a
selected respondent is an adult associated with the phone number who lives in a private residence or
college housing within the United States or territories covered by the BRFSS. For the landline telephone
sample, a selected respondent is the person chosen for interview during the household enumeration
section of the screening questions.
Personal cellular telephone: A cellular telephone that is used for personal calls. Cellular telephones that
are used for both personal and business calls may be categorized as personal telephones and persons
contacted on these phones are eligible for interview. Persons using telephones that are exclusively for
business use are not eligible for interview.
Table 2.
2019 Disposition Codes for Landline Telephones and Cellular Telephones
Category
Interviewed
(1000-level codes)
Eligible, Non-Interview
(2000 level codes)
Unknown Eligibility
Code
Description
1100
Completed interview
1200
Partially completed interview
2111
Household level refusal (used for landline only)
2112
Selected respondent refusal
2120
Break off/termination within questionnaire
2210
Selected respondent never available
2320
Selected respondent physically or mentally unable to
complete interview
2330
Language barrier of selected respondent
3100
Unknown if housing unit
3130
No answer
3140
Answering device, unknown whether eligible
3150
Telecommunication barrier (i.e. call blocking)
3200
Household, not known if respondent eligible
3322
Physical or mental impairment (household level)
3330
Language barrier (household level)
3700
On never-call list
Page 6 of 26
Table 2.
2019 Disposition Codes for Landline Telephones and Cellular Telephones
Category
Not Eligible
Code
Description
4100
Out of sample
4200
Fax/data/modem
4300
Nonworking/disconnected number
4400
Technological barrier
(i.e., fast busy, phone circuit barriers)
4430
Call forwarding/pager
4460
Landline telephone number
(used for cellular telephone only)
4500
Non-residence/business
4900
Miscellaneous, non-eligible
Factors affecting the distribution of disposition codes by state include differences in telephone systems, sample
designs, surveyed populations, and data collection processes. Table 3 defines the categories of disposition codes
used to calculate outcome and response rates illustrated in Tables 4A through 6.
Table 3.
Categories of 2019 Landline and Cellular Telephone Disposition Codes
Category
Disposition Code
Definitions
Formulae
Abbreviation
Completed
Interviews
1100+1200
COIN
Eligible
1100+1200+2111+2112+2120+2210+2320+2330
ELIG
Contacted Eligible
1100+1200+2111+2112+2120+2210+2320+2330
CONELIG
Terminations and
Refusals
2111+2112+2120
TERE
Ineligible Phone
Numbers
All 4000 level disposition codes
INELIG
Unknown Whether
Eligible
All 3000 level disposition codes
UNKELIG
Eligibility Factor
ELIG/(ELIG + INELIG)
E
The disposition codes are categorized according to the groups illustrated in Table 3 to produce rates of
resolution, cooperation, completion, refusal and response. In accordance with population surveillance standards,
Page 7 of 26
the proportions of people who may have been eligible for interview, but who were not able to be interviewed,
are accounted for in the formulae.
Eligibility Factor
E = ELIG/ (ELIG + INELIG)
The Eligibility Factor is the proportion of eligible phone numbers from among all sample numbers for which
eligibility has been determined. The eligibility factor, therefore, provides a measure of eligibility that can be
applied to sample numbers with unknown eligibility. The purpose of the eligibility factor is to estimate the
proportion of the sample that is likely to be eligible. The eligibility factor is used in the calculations of refusal
and response rates. Separate eligibility factors are calculated for landline telephones and cellular telephone
samples for each state and territory.
Resolution Rate
((ELIG + INELIG) / (ELIG+INELIG+UNKELIG))*100
The Resolution Rate is the percentage of numbers in the total sample for which eligibility has been determined.
The total number of eligible and ineligible sample phone numbers is divided by the total number of phone
numbers in the entire sample. The result is multiplied by 100 to calculate the percentage of the sample for which
eligibility is determined. Separate resolution rates are calculated for landline telephone and cellular telephone
samples for each state and territory.
Page 8 of 26
Interview Completion Rate
(COIN / (COIN + TERE)) * 100
The Interview Completion Rate is the rate of completed interviews among all respondents who have been
determined to be eligible and selected for interviewing. The numerator is the number of complete and partially
completed interviews. This number is divided by the number of completed interviews, partially completed
interviews, and all break offs, refusals, and terminations. The result is multiplied by 100 to provide the
percentage of completed interviews among eligible respondents who are contacted by interviewers. Separate
interview completion rates are calculated for landline telephone and cellular telephone samples for each state
and territory.
Cooperation Rate
(COIN / CONELIG) *100
The AAPOR Cooperation Rate is the number of complete and partial complete interviews divided by the
number of contacted and eligible respondents. The BRFSS Cooperation Rate follows the guidelines of AAPOR
Cooperation Rate #2. Separate cooperation rates are calculated for landline telephone and cellular telephone
samples for each state and territory.
Refusal Rate
(TERE / (ELIG + (E * UNKELIG))) * 100
The BRFSS Refusal Rate is the proportion of all eligible respondents who refused to complete an interview or
terminated an interview prior to the threshold required to be considered a partial interview. Refusals and
terminations (TERE) are in the numerator, and the denominator includes all eligible numbers and a proportion
of the numbers with unknown eligibility. The proportion of numbers with unknown eligibility is determined by
the eligibility factor (E as described above). The result is then multiplied by 100 to provide a percentage of
refusals among all eligible and likely to be eligible numbers in the sample. Separate refusal rates are calculated
for landline telephone and cellular telephone samples for each state and territory.
Response Rate
(COIN / ((ELIG + (E * UNKELIG))) * 100
A Response Rate is an outcome rate with the number of complete and partial interviews in the numerator and an
estimate of the number of eligible units in the sam
ple in the denominator. The BRFSS Response Rate calculation assumes that the unresolved numbers contain the
same percentage of eligible households or eligible personal cell phones as the records whose eligibility or
ineligibility are determined. The BRFSS Response Rate follows the guidelines for AAPOR Response Rate #4.
It also is similar to the BRFSS CASRO Rates reported prior to 2011. Separate eligibility factors are calculated
for landline telephone and cellular telephone samples for each state and territory and a combined Response Rate
for landline telephone and cellular telephone also is calculated. The combined landline telephone and cellular
telephone response rate is generated by weighting to the respective size of the two samples. The total sample
equals the landline telephone sample plus cellular telephone sample. The proportion of each sample is
calculated using the total sample as the denominator. The formulae for the proportions of the sample are found
below:
P1 = TOTAL LANDLINE SAMPLE /
Page 9 of 26
(TOTAL LANDLINE SAMPLE + TOTAL CELL PHONE SAMPLE);
P2 = TOTAL CELL PHONE SAMPLE /
(TOTAL LANDLINE SAMPLE + TOTAL CELL PHONE SAMPLE);
The formula for the Combined Landline Telephone and Cellular Telephone Weighted Response Rate, therefore,
is described below:
COMBINED RESPONSE RATE=
(P1 * LANDLINE RESPONSE RATE) + (P2 * CELL PHONE RESPONSE RATE).
Tables of Outcomes and Rates by State
The tables on the following pages illustrate calling outcomes in categories of eligibility, rates of cooperation,
refusal, resolution, and response by landline telephone and cellular telephone samples.
➢ Tables 4A and 4B provide information on the size of the sample and the numbers and percentages of
completed interviews, cooperation rates, terminations and refusals, and contacts with eligible households
by state and territory.
➢ Tables 5A and 5B provide information on the number and percentage of landline telephone and cellular
telephone sample numbers that are eligible, ineligible, and of unknown eligibility.
➢ Table 6 provides response rates for landline telephone samples, cellular telephone samples, and
combined samples.
Page 10 of 26
Table 4A. Landline Sample.
Completions, Terminations and Refusals, Contacted Eligible Households and Total Sample by State
COIN
State
N
TERE
%
N
CONELIG
COOP
%
N
%
%
Total
Sample
AL
2,223
2.2
1,348
1.3
3,815
3.8
58.3
100,947
AK
1,387
1.4
729
0.8
2,391
2.5
58.0
96,840
AZ
2,993
2.1
1,089
0.8
4,512
3.2
66.3
142,410
AR
2,657
2.7
816
0.8
3,740
3.7
71.0
99,749
CA
2,268
1.0
1,384
0.6
4,023
1.8
56.4
226,170
CO
3,126
2.3
640
0.5
4,681
3.4
66.8
138,630
CT
4,284
4.3
1,555
1.6
6,509
6.5
65.8
99,810
DE
1,035
1.0
281
0.3
1,870
1.7
55.3
108,090
DC
1,021
1.1
517
0.5
1,658
1.7
61.6
96,000
FL
5,739
1.0
1,843
0.3
10,485
1.8
54.7
574,560
GA
2,282
0.6
771
0.2
4,240
1.1
53.8
374,460
HI
1,831
2.0
676
0.7
3,171
3.5
57.7
91,830
ID
1,410
1.0
1,722
1.2
3,193
2.3
44.2
140,281
IL
1,064
1.7
381
0.6
1,713
2.8
62.1
62,280
IN
3,558
1.7
1,923
0.9
6,122
2.9
58.1
214,140
IA
2,505
3.7
896
1.3
3,752
5.5
66.8
67,890
KS
3,587
3.0
1,345
1.1
5,315
4.5
67.5
118,980
KY
2,637
1.7
376
0.2
3,079
2.0
85.6
152,550
LA
949
1.5
661
1.0
1,750
2.7
54.2
65,430
ME
6,957
2.3
1,283
0.4
8,561
2.8
81.3
308,289
MD
9,003
2.1
4,467
1.0
15,493
3.6
58.1
425,940
MA
2,677
1.5
544
0.3
3,409
1.9
78.5
181,422
MI
3,541
1.7
958
0.5
5,375
2.5
65.9
212,880
MN
2,958
1.8
628
0.4
4,770
2.9
62.0
165,870
MS
1,438
2.4
621
1.0
2,178
3.6
66.0
60,089
MO
2,920
3.0
865
0.9
4,406
4.5
66.3
97,443
11 of 26
Table 4A. Landline Sample.
Completions, Terminations and Refusals, Contacted Eligible Households and Total Sample by State
COIN
State
N
TERE
%
N
CONELIG
COOP
%
N
%
%
Total
Sample
MT
2,712
2.9
862
0.9
3,869
4.1
70.1
94,469
NE
5,875
3.2
2,079
1.1
9,250
5.0
63.5
184,499
NV
664
1.7
186
0.5
933
2.4
71.2
39,180
NH
3,051
3.3
1,183
1.3
4,666
5.0
65.4
92,850
*
*
NJ
*
*
*
*
*
*
NM
2,009
2.2
896
1.0
3,329
3.7
60.3
89,460
NY
6,436
1.6
4,381
1.1
12,365
3.1
52.1
398,310
NC
824
2.8
692
2.3
1,636
5.5
50.4
29,700
ND
3,094
2.9
888
0.8
4,356
4.1
71.0
105,267
OH
6,708
1.1
1,722
0.3
11,405
1.9
58.8
598,050
OK
1,986
2.6
853
1.1
3,290
4.3
60.4
75,716
OR
1,069
3.3
135
0.4
1,235
3.9
86.6
32,011
PA
1,518
2.2
592
0.9
2,372
3.5
64.0
68,580
RI
2,522
2.0
1,576
1.3
4,460
3.6
56.5
125,220
SC
2,667
2.5
723
0.7
3,541
3.3
75.3
107,262
SD
3,021
2.5
280
0.2
3,409
2.8
88.6
121,944
TN
1,749
1.8
984
1.0
2,924
3.0
59.8
97,079
TX
4,432
1.3
2,089
0.6
7,393
2.2
59.9
340,260
UT
2,635
3.2
707
0.8
3,976
4.8
66.3
83,520
VT
3,200
2.6
1,538
1.3
5,153
4.2
62.1
122,040
VA
3,891
1.7
875
0.4
6,599
2.9
59.0
224,460
WA
4,021
1.8
2,010
0.9
6,661
2.9
60.4
226,901
WV
2,830
6.7
901
2.1
4,087
9.7
69.2
41,940
WI
2,096
3.7
740
1.3
3,197
5.6
65.6
56,669
WY
2,770
2.7
481
0.5
4,066
3.9
68.1
103,620
GU
1,141
2.3
620
1.3
2,405
4.9
47.4
48,794
12 of 26
Table 4A. Landline Sample.
Completions, Terminations and Refusals, Contacted Eligible Households and Total Sample by State
COIN
State
N
TERE
%
N
CONELIG
COOP
%
N
%
%
Total
Sample
PR
971
2.3
220
0.5
1,510
3.5
64.3
43,019
Minimum
664
0.6
135
0.2
933
1.1
44.2
29,700
Maximum
9,003
6.7
4,467
2.3
15,493
9.7
88.6
598,050
Mean
2,884
2.3
1,087
0.8
4,583
3.5
63.8
153,342
Median
2,662
2.2
864
0.8
3,923
3.4
62.8
104,444
*New Jersey was unable to collect enough BRFSS data in 2019 to meet the minimum requirements for inclusion
in the 2019 BRFSS public-use data set.
13 of 26
Table 4B. Cell Phone Sample.
Completions, Terminations and Refusals, Contacted Eligible Households and Total Sample by State
COIN
TERE
State
N
%
AL
4,747
4.2
N
CONELIG
COOP
Total
Sample
%
N
%
%
1,115
1.0
5,901
5.2
80.4
114,301
AK
1,639
3.0
246
0.5
1,917
3.6
85.5
53,760
AZ
5,530
3.8
1,011
0.7
6,772
4.6
81.7
146,160
AR
2,674
5.0
390
0.7
3,193
5.9
83.7
53,850
CA
9,021
4.4
2,649
1.3
12,025
5.9
75.0
203,360
CO
6,236
7.1
648
0.7
6,972
8.0
89.4
87,342
CT
5,228
3.5
1,359
0.9
6,808
4.6
76.8
149,250
DE
3,005
2.3
516
0.4
3,763
2.9
79.9
128,160
DC
1,435
1.0
531
0.4
2,005
1.4
71.6
139,169
FL
10,362
2.9
2,240
0.6
13,607
3.8
76.2
360,600
GA
4,770
1.8
1,242
0.5
6,497
2.5
73.4
259,650
HI
5,755
5.1
1,019
0.9
6,958
6.1
82.7
113,189
ID
3,945
3.6
395
0.4
4,360
4.0
90.5
108,304
IL
4,430
3.8
656
0.6
5,217
4.5
84.9
116,310
IN
5,323
4.8
906
0.8
6,532
5.8
81.5
111,780
IA
7,511
7.1
782
0.7
8,380
8.0
89.6
105,357
KS
8,373
3.7
1,085
0.5
9,554
4.2
87.6
229,284
KY
5,411
3.1
593
0.3
6,041
3.5
89.6
172,050
LA
3,789
2.9
1,120
0.9
4,976
3.8
76.1
129,390
ME
4,477
2.8
448
0.3
4,948
3.1
90.5
161,208
MD
9,004
3.6
1,987
0.8
11,386
4.5
79.1
252,599
MA
4,928
2.0
481
0.2
5,476
2.2
90.0
248,418
MI
7,258
3.8
1,021
0.5
9,081
4.7
79.9
193,531
MN
12,166
3.7
1,382
0.4
14,458
4.4
84.1
326,190
MS
3,742
5.6
543
0.8
4,315
6.4
86.7
66,956
MO
4,177
6.5
329
0.5
4,603
7.2
90.7
63,827
14 of 26
Table 4B. Cell Phone Sample.
Completions, Terminations and Refusals, Contacted Eligible Households and Total Sample by State
COIN
TERE
CONELIG
COOP
State
N
%
N
%
N
%
%
Total
Sample
MT
4,053
5.7
328
0.5
4,405
6.2
92.0
71,599
NE
10,515
5.9
1,284
0.7
12,016
6.8
87.5
176,910
NV
2,078
5.5
216
0.6
2,312
6.2
89.9
37,483
NH
3,012
4.9
542
0.9
3,630
5.9
83.0
61,304
NJ
*
*
*
*
*
*
*
*
NM
4,216
6.7
839
1.3
5,127
8.2
82.2
62,850
NY
7,985
3.5
2,154
0.9
10,954
4.8
72.9
227,460
NC
3,010
5.0
421
0.7
3,490
5.8
86.2
59,760
ND
2,757
3.3
273
0.3
3,086
3.7
89.3
83,215
OH
6,831
3.0
998
0.4
8,431
3.8
81.0
224,070
OK
4,348
4.9
871
1.0
5,319
6.0
81.7
88,292
OR
4,878
3.7
319
0.2
5,221
3.9
93.4
132,866
PA
4,998
4.0
752
0.6
5,848
4.7
85.5
125,580
RI
3,816
3.5
910
0.8
4,894
4.5
78.0
108,300
SC
4,427
4.4
671
0.7
5,162
5.1
85.8
100,846
SD
3,671
2.5
179
0.1
3,897
2.7
94.2
146,560
TN
4,389
2.9
1,360
0.9
5,823
3.8
75.4
153,540
TX
6,913
4.1
1,519
0.9
9,103
5.4
75.9
168,450
UT
9,494
8.2
738
0.6
10,571
9.1
89.8
115,890
VT
3,208
4.1
474
0.6
3,760
4.8
85.3
78,480
VA
5,898
3.3
791
0.4
7,246
4.0
81.4
179,910
WA
9,101
5.7
1,650
1.0
11,003
6.9
82.7
158,760
WV
2,550
5.5
384
0.8
2,965
6.4
86.0
46,500
WI
2,888
5.8
419
0.8
3,366
6.8
85.8
49,530
WY
1,958
3.6
174
0.3
2,199
4.0
89.0
54,810
GU
1,286
2.8
276
0.6
1,629
3.5
78.9
46,650
15 of 26
Table 4B. Cell Phone Sample.
Completions, Terminations and Refusals, Contacted Eligible Households and Total Sample by State
COIN
State
N
TERE
%
N
CONELIG
%
N
COOP
%
%
Total
Sample
PR
5,303
14.3
273
0.7
5,737
15.4
92.4
37,170
Minimum
1,286
1.0
174
0.1
1,629
1.4
71.6
37,170
Maximum
12,166
14.3
2,649
1.3
14,458
15.4
94.2
360,600
Mean
5,164
4.4
837
0.7
6,210
5.2
83.9
132,515
Median
4,612
3.8
705
0.7
5,398
4.7
84.5
116,100
*New Jersey was unable to collect enough BRFSS data in 2019 to meet the minimum requirements for inclusion
in the 2019 BRFSS public-use data set.
16 of 26
Table 5A. Landline Sample.
Categories of Eligibility by State (Landline Only).
ELIG
State
N
INELIG
%
N
UNKELIG
%
N
%
AL
3,815
3.8
81,841
81.1
15,291
15.1
AK
2,391
2.5
87,515
90.4
6,934
7.2
AZ
4,512
3.2
119,037
83.6
18,861
13.2
AR
3,740
3.7
84,480
84.7
11,529
11.6
CA
4,023
1.8
180,695
79.9
41,452
18.3
CO
4,681
3.4
112,815
81.4
21,134
15.2
CT
6,509
6.5
74,059
74.2
19,242
19.3
DE
1,870
1.7
75,191
69.6
31,029
28.7
DC
1,658
1.7
76,201
79.4
18,141
18.9
FL
10,485
1.8
462,577
80.5
101,498
17.7
GA
4,240
1.1
303,745
81.1
66,475
17.8
HI
3,171
3.5
74,501
81.1
14,158
15.4
ID
3,193
2.3
118,735
84.6
18,353
13.1
IL
1,713
2.8
49,398
79.3
11,169
17.9
IN
6,122
2.9
173,729
81.1
34,289
16.0
IA
3,752
5.5
54,387
80.1
9,751
14.4
KS
5,315
4.5
98,647
82.9
15,018
12.6
KY
3,079
2.0
127,356
83.5
22,115
14.5
LA
1,750
2.7
53,667
82.0
10,013
15.3
ME
8,561
2.8
238,467
77.4
61,261
19.9
MD
15,493
3.6
319,543
75.0
90,904
21.3
MA
3,409
1.9
128,083
70.6
49,930
27.5
MI
5,375
2.5
174,175
81.8
33,330
15.7
MN
4,770
2.9
132,310
79.8
28,790
17.4
MS
2,178
3.6
51,494
85.7
6,417
10.7
MO
4,406
4.5
79,478
81.6
13,559
13.9
MT
3,869
4.1
74,781
79.2
15,819
16.7
17 of 26
Table 5A. Landline Sample.
Categories of Eligibility by State (Landline Only).
ELIG
State
N
INELIG
%
N
UNKELIG
%
N
%
NE
9,250
5.0
150,532
81.6
24,717
13.4
NV
933
2.4
31,775
81.1
6,472
16.5
NH
4,666
5.0
68,731
74.0
19,453
21.0
NJ
*
NM
3,329
3.7
75,984
84.9
10,147
11.3
NY
12,365
3.1
289,976
72.8
95,969
24.1
NC
1,636
5.5
22,620
76.2
5,444
18.3
ND
4,356
4.1
87,660
83.3
13,251
12.6
OH
11,405
1.9
476,487
79.7
110,158
18.4
OK
3,290
4.3
64,125
84.7
8,301
11.0
OR
1,235
3.9
26,577
83.0
4,199
13.1
PA
2,372
3.5
51,140
74.6
15,068
22.0
RI
4,460
3.6
97,423
77.8
23,337
18.6
SC
3,541
3.3
88,653
82.7
15,067
14.0
SD
3,409
2.8
103,899
85.2
14,636
12.0
TN
2,924
3.0
78,304
80.7
15,851
16.3
TX
7,393
2.2
279,946
82.3
52,921
15.6
UT
3,976
4.8
69,036
82.7
10,508
12.6
VT
5,153
4.2
93,581
76.7
23,306
19.1
VA
6,599
2.9
168,660
75.1
49,201
21.9
WA
6,661
2.9
183,472
80.9
36,768
16.2
WV
4,087
9.7
29,737
70.9
8,116
19.4
WI
3,197
5.6
44,062
77.8
9,410
16.6
WY
4,066
3.9
83,493
80.6
16,061
15.5
GU
2,405
4.9
39,842
81.7
6,547
13.4
PR
1,510
3.5
35,889
83.4
5,620
13.1
933
1.1
22,620
69.6
4,199
7.2
Minimum
*
*
*
*
*
18 of 26
Table 5A. Landline Sample.
Categories of Eligibility by State (Landline Only).
ELIG
State
INELIG
UNKELIG
N
%
N
%
N
%
Maximum
15,493
9.7
476,487
90.4
110,158
28.7
Mean
4,583
3.5
122,087
80.1
26,673
16.4
Median
3,923
3.4
85,998
81.1
15,956
15.8
*New Jersey was unable to collect enough BRFSS data in 2019 to meet the minimum requirements for inclusion
in the 2019 BRFSS public-use data set.
19 of 26
Table 5B. Cell Phone Sample.
Categories of Eligibility by State (Cell Phone Only).
ELIG
State
N
INELIG
%
N
UNKELIG
%
N
%
AL
5,901
5.2
54,758
47.9
53,642
46.9
AK
1,917
3.6
45,301
84.3
6,542
12.2
AZ
6,772
4.6
86,310
59.1
53,078
36.3
AR
3,193
5.9
30,541
56.7
20,116
37.4
CA
12,025
5.9
79,232
39.0
112,103
55.1
CO
6,972
8.0
39,040
44.7
41,330
47.3
CT
6,808
4.6
68,478
45.9
73,964
49.6
DE
3,763
2.9
55,797
43.5
68,600
53.5
DC
2,005
1.4
64,394
46.3
72,770
52.3
FL
13,607
3.8
189,774
52.6
157,219
43.6
GA
6,497
2.5
132,499
51.0
120,654
46.5
HI
6,958
6.1
44,939
39.7
61,292
54.2
ID
4,360
4.0
51,566
47.6
52,378
48.4
IL
5,217
4.5
38,477
33.1
72,616
62.4
IN
6,532
5.8
50,043
44.8
55,205
49.4
IA
8,380
8.0
61,769
58.6
35,208
33.4
KS
9,554
4.2
139,825
61.0
79,905
34.8
KY
6,041
3.5
87,936
51.1
78,073
45.4
LA
4,976
3.8
63,758
49.3
60,656
46.9
ME
4,948
3.1
84,811
52.6
71,449
44.3
MD
11,386
4.5
128,252
50.8
112,961
44.7
MA
5,476
2.2
121,247
48.8
121,695
49.0
MI
9,081
4.7
104,616
54.1
79,834
41.3
MN
14,458
4.4
163,107
50.0
148,625
45.6
MS
4,315
6.4
38,873
58.1
23,768
35.5
MO
4,603
7.2
33,997
53.3
25,227
39.5
MT
4,405
6.2
37,428
52.3
29,766
41.6
20 of 26
Table 5B. Cell Phone Sample.
Categories of Eligibility by State (Cell Phone Only).
ELIG
INELIG
UNKELIG
State
N
%
N
%
N
%
NE
12,016
6.8
108,526
61.3
56,368
31.9
NV
2,312
6.2
15,994
42.7
19,177
51.2
NH
3,630
5.9
30,363
49.5
27,311
44.6
NJ
*
*
*
*
*
*
NM
5,127
8.2
33,363
53.1
24,360
38.8
NY
10,954
4.8
93,703
41.2
122,803
54.0
NC
3,490
5.8
24,652
41.3
31,618
52.9
ND
3,086
3.7
52,051
62.6
28,078
33.7
OH
8,431
3.8
108,116
48.3
107,523
48.0
OK
5,319
6.0
55,207
62.5
27,766
31.4
OR
5,221
3.9
51,012
38.4
76,633
57.7
PA
5,848
4.7
60,043
47.8
59,689
47.5
RI
4,894
4.5
51,905
47.9
51,501
47.6
SC
5,162
5.1
46,331
45.9
49,353
48.9
SD
3,897
2.7
103,408
70.6
39,255
26.8
TN
5,823
3.8
69,342
45.2
78,375
51.0
TX
9,103
5.4
74,203
44.1
85,144
50.5
UT
10,571
9.1
57,254
49.4
48,065
41.5
VT
3,760
4.8
40,343
51.4
34,377
43.8
VA
7,246
4.0
81,629
45.4
91,035
50.6
WA
11,003
6.9
69,834
44.0
77,923
49.1
WV
2,965
6.4
20,792
44.7
22,743
48.9
WI
3,366
6.8
26,549
53.6
19,615
39.6
WY
2,199
4.0
38,997
71.1
13,614
24.8
GU
1,629
3.5
31,993
68.6
13,028
27.9
PR
5,737
15.4
12,901
34.7
18,532
49.9
Minimum
1,629
1.4
12,901
33.1
6,542
12.2
21 of 26
Table 5B. Cell Phone Sample.
Categories of Eligibility by State (Cell Phone Only).
ELIG
State
N
INELIG
%
N
UNKELIG
%
N
%
Maximum
14,458
15.4
189,774
84.3
157,219
62.4
Mean
6,210
5.2
66,448
50.8
59,857
44.0
Median
5,398
4.7
55,502
49.3
54,424
46.7
*New Jersey was unable to collect enough BRFSS data in 2019 to meet the minimum requirements for inclusion
in the 2019 BRFSS public-use data set.
22 of 26
Table 6. Response Rates for Landline and Cell Phone Samples
State
Landline Response
Rate
Cell Phone
Response Rate
Combined Response
Rate
AL
49.4
42.7
45.9
AK
53.9
75.1
61.4
AZ
57.5
52.0
54.7
AR
62.8
52.5
59.2
CA
46.0
33.7
40.2
CO
56.6
47.1
52.9
CT
53.1
38.7
44.5
DE
39.5
37.1
38.2
DC
49.9
34.1
40.6
FL
45.1
43.0
44.3
GA
44.3
39.3
42.2
HI
48.8
37.9
42.8
ID
38.4
46.7
42.0
IL
51.0
31.9
38.6
IN
48.8
41.2
46.2
IA
57.2
59.7
58.7
KS
59.0
57.1
57.7
KY
73.2
48.9
60.3
LA
45.9
40.4
42.3
ME
65.1
50.4
60.1
MD
45.7
43.7
45.0
MA
56.9
45.9
50.6
MI
55.6
47.0
51.5
MN
51.2
45.8
47.6
MS
59.0
55.9
57.4
MO
57.1
54.9
56.2
MT
58.4
53.8
56.4
NE
55.0
59.6
57.3
23 of 26
Table 6. Response Rates for Landline and Cell Phone Samples
State
Landline Response
Rate
Cell Phone
Response Rate
Combined Response
Rate
NV
59.4
43.9
51.8
NH
51.7
46.0
49.4
NJ
*
*
*
NM
53.5
50.4
52.2
NY
39.5
33.5
37.3
NC
41.1
40.6
40.8
ND
62.1
59.2
60.8
OH
48.0
42.1
46.4
OK
53.7
56.0
55.0
OR
75.2
39.5
46.5
PA
49.9
44.8
46.6
RI
46.0
40.9
43.6
SC
64.7
43.8
54.6
SD
78.0
69.0
73.1
TN
50.0
36.9
42.0
TX
50.6
37.6
46.3
UT
57.9
52.6
54.8
VT
50.2
47.9
49.3
VA
46.0
40.2
43.4
WA
50.6
42.1
47.1
WV
55.8
43.9
49.6
WI
54.7
51.8
53.3
WY
57.6
66.9
60.8
GU
41.1
56.9
48.8
PR
55.9
46.3
51.5
Minimum
38.4
31.9
37.3
Maximum
78.0
75.1
73.1
24 of 26
Table 6. Response Rates for Landline and Cell Phone Samples
State
Landline Response
Rate
Cell Phone
Response Rate
Combined Response
Rate
Mean
53.4
47.1
50.0
Median
53.3
45.9
49.4
*New Jersey was unable to collect enough BRFSS data in 2019 to meet the minimum requirements for inclusion
in the 2019 BRFSS public-use data set.
25 of 26
References
1. Pierannunzi C, Town M, Garvin W, Shaw F, Balluz L. Methodologic changes in the Behavioral Risk
Factor Surveillance System in 2011 and potential effects on prevalence estimates. MMWR.2012;
61(22):410-413. http://www.cdc.gov/mmwr/preview/mmwrhtml/mm6122a3.htm. Accessed August 4,
2020.
2. The American Association for Public Opinion Research. Standard Definitions: Final Dispositions of
Case Codes and Outcome Rates for Surveys website
http://www.aapor.org/AAPOR_Main/media/publications/Standard-Definitions20169theditionfinal.pdf.
Accessed August 4, 2020.
3. The Council of American Survey Research Organizations. 2013. Code of Standards and Ethics for
Market, Opinion, and Social Research website. https://www.insightsassociation.org/issuespolicies/casro-code-standards-and-ethics. Accessed August 4, 2020.
4. Czajka JL, Beyler A. Declining Response Rates in Federal Surveys: Trends and Implications (2016).
https://aspe.hhs.gov/system/files/pdf/255531/Decliningresponserates.pdf. Accessed August 4, 2020.
5. The Pew Research Center for People and the Press. 2012. Assessing the Representativeness of Public
Opinion Surveys website. https://www.pewresearch.org/politics/2012/05/15/assessing-therepresentativeness-of-public-opinion-surveys/ . Accessed August 4, 2020.
6. Lindemann, N. What’s the Average Survey Response Rate [2019 Benchmark].
https://surveyanyplace.com/average-survey-response-rate/ Accessed August 4, 2020.
7. Groves, RM. Nonresponse rates and nonresponse bias in household surveys. Public Opinion
Quarterly. 2006; 70 (5) :646-675. https://academic.oup.com/poq/article/70/5/646/4084443.
Accessed August 4, 2020.
26 of 26
| File Type | application/pdf |
| File Title | Behavioral Risk Factor Surveillance System 2019 Summary Data Quality Report |
| Subject | Behavioral Risk Factor Surveillance System, 2019 Summary Data Quality Report |
| Author | CDC |
| File Modified | 2020-08-13 |
| File Created | 2020-08-12 |