2015 BRFSS Summary Data Quality Report

Behavioral Risk Factor Surveillance System
2015 Summary Data Quality Report
July 29, 2015

BRFSS 2015 Summary Data Quality Report

1 of 27

Table of Contents
Introduction ..................................................................................................................................................3
Interpretation of BRFSS Response Rates ....................................................................................................4
BRFSS 2015 Call Outcome Measures and Response Rate Formulae .........................................................6
Tables of Outcomes and Rates by State .....................................................................................................11
References ..................................................................................................................................................23

BRFSS 2015 Summary Data Quality Report

2 of 27

Introduction
The Behavioral Risk Factor Surveillance System (BRFSS) is a state-based, CDC-assisted healthdata collection project and partnership of state health departments, CDC’s Division of Population
Health, and other CDC programs and offices. It comprises telephone surveys conducted by the
health departments of all 50 states, the District of Columbia, Puerto Rico, and Guam.
This Summary Data Quality Report presents detailed descriptions of the 2015 BRFSS calling
outcomes and call summary information for each of the states and territories that participated in
the 2015 BRFSS. All BRFSS public-use data are collected by landline telephone and cellular
telephone to produce a single data set aggregated from the 2015 BRFSS territorial- and statelevel data sets. The variables and outcomes provided in this document are applicable to a
combined data set of responses from participants using landline telephones and cellular
telephones within each of the states and territories.
The inclusion of data from cellular telephone interviews in the BRFSS public release data set has
been standard protocol since 2011. In many respects, 2011 was a year of change—both in
BRFSS approach and methodology. As the results of cellular telephone interviews were added in
2011, so were new weighting procedures that could accommodate the inclusion of new
weighting variables. Data users should note that new weighting procedures are likely to affect
trend lines when comparing BRFSS data collected before and after 2011. Because of these
changes, users are advised NOT to make direct comparisons with pre-2011 data, and instead,
should begin new trend lines with that year. Details of changes beginning with the 2011 BRFSS
are provided in the Morbidity and Mortality Weekly Report (MMWR), which highlights
weighting and coverage effects on trend lines.1
The measures presented in this document are designed to summarize the quality of the 2015
BRFSS survey data. Response rates, cooperation rates, and refusal rates for BRFSS are
calculated using standards set by the American Association of Public Opinion Research
(AAPOR).2 The BRFSS has calculated 2015 response rates using AAPOR Response Rate #4,
which is in keeping with rates provided by BRFSS before 2011 using rates from the Council of
American Survey Research Organizations (CASRO).3
On the basis of the AAPOR guidelines, response rate calculations include assumptions of
eligibility among potential respondents or households that are not interviewed. Changes in the
geographic distribution of cellular telephone numbers by telephone companies and the portability
of landline telephone numbers are likely to make it more difficult than in the past to ascertain
which telephone numbers are out-of-sample and which telephone numbers represent likely
households. The BRFSS calculates likely households using the proportions of eligible

BRFSS 2015 Summary Data Quality Report

3 of 27

households among all phone numbers where eligibility has been determined. This eligibility
factor appears in calculations of response-, cooperation-, resolution-, and refusal rates.

Interpretation of BRFSS Response Rates
Because this report reflects the initial inclusion of BRFSS cellular telephone interviews,
contextual information on cellular telephone response rates is provided below. Although cellular
telephone response rates are generally lower than landline telephone response rates across most
surveys, the BRFSS has achieved a cellular telephone response rate that compares favorably with
other similar surveys (Table 1).
Table 1.
Examples of Cellular Telephone and Landline Survey Response Rates
Response Rates

Year(s)

Landline

Cell
Phone

2011–2012

17.0%

18.3%

The Commonwealth Fund 2010 Biennial Health
Insurance Surveyb

2012

29.0%

25.0%

National Immunization Survey (NIS)a c

2014

62.6% a

33.5%

Pew Internet and American Life Projectd

2012

30.0%

20.0%

PSRAI Omnibus Surveye

2015

5.0%

4.0%

2012-2013

47.2%

36.3%

2015

48.2%

47.2%

Survey
California Health Interview Survey (CHIS)

National Adult Tobacco Survey (NATS)f
BRFSSg
c

Unlike the BRFSS, the NIS does not include household sampling in the landline portion of the study but interviews the adult
who self-identifies as the most knowledgeable about household immunization information.
ahttp://healthpolicy.ucla.edu/chis/design/Documents/chis2011-2012-method-2_2014-02-21.pdf
bhttp://www.commonwealthfund.org/interactives-and-data/surveys/2011/mar/2010-biennial-health-insurance-survey
chttp://www.cdc.gov/mmwr/preview/mmwrhtml/mm6433a1.htm
dhttp://www.people-press.org/2006/05/15/the-cell-phone-challenge-to-survey-research/
ehttp://www.pewinternet.org/2015/04/01/appendix-a-about-the-december-week-1-and-week-3-omnibus-survey/
fhttp://www.cdc.gov/tobacco/data_statistics/surveys/nats/pdfs/2012-2013-nats-methodology-final.pdf
gBRFSS

response rates are presented here as median rates for all states and territories.

BRFSS 2015 Summary Data Quality Report

4 of 27

Research by the Pew Research Center indicates that response rates for all telephone-based surveys have
declined in recent years.4 Despite lower response rates, this research supports previous findings5 that
weighting to demographic characteristics of respondents ensures accurate estimates for most measures.
The following tables present landline telephone and cellular telephone calling outcomes and rates. The
BRFSS cellular telephone survey was collected in a manner similar to that of the BRFSS landline
telephone survey. One important difference, however, is that interviews conducted by landline
telephones include random selection among adults within households, while cellular telephone
interviews are conducted with adults who are contacted on personal (nonbusiness) cellular telephones.
The report presents data on three general types of measure by state:
1. Call outcome measures, including response rates, which are based on landline telephone
disposition codes.
2. Call outcome measures, including response rates, which are based on cellular telephone
disposition codes.
3. A weighted response rate, based on a combination of the landline telephone response rate with
the cellular telephone response rate proportional to the total sample used to collect the data for a
state.

For clarity, the BRFSS recommends that authors and researchers referencing BRFSS data quality
include the following language, below. Note the places where authors should include information
specific to their projects.
Response rates for BRFSS are calculated using standards set by the American Association of Public Opinion Research
(AAPOR) Response Rate Formula #4 (http://www.aapor.org/AAPOR_Main/media/publications/StandardDefinitions20169theditionfinal.pdf). The response rate is the number of respondents who completed the survey as a
proportion of all eligible and likely-eligible people. The median survey response rate for all states, territories and
Washington, DC, in 2015 was 47.2, and ranged from 33.9 to 61.1.a Response rates for states and territories included in this
analysis had a median of [provide median] and ranged from [provide range],b For detailed information see the BRFSS
Summary Data Quality Report.c
a

Response rates and ranges should reflect the year(s) included in the analyses.
Response rates for states selected for analysis should be included here. This sentence may be omitted if all states are used
in the analysis.
c
See the Summary Data Quality Report for the year(s) included in the analyses.
b

BRFSS 2015 Summary Data Quality Report

5 of 27

BRFSS 2015 Call Outcome Measures and Response Rate Formulae
The calculations of calling-outcome rates are based on final disposition codes that are assigned after all
calling attempts have been exhausted. The BRFSS may make up to 15 attempts to reach a respondent
before assigning a final disposition code. In 2015, the BRFSS used a single set of disposition codes for
both landline and cell phones, adapted from standardized AAPOR disposition codes for telephone
surveys. A few disposition codes apply only to landline telephone or cellular telephone sample numbers.
For example, answering-device messages may confirm household eligibility for landline telephone
numbers but are not used to determine eligibility of cellular telephone numbers. Disposition codes
reflect whether interviewers have completed or partially completed an interview (1000 level codes),
determined that the household was eligible without completing an interview (2000 level codes),
determined that a household or respondent was ineligible (4000 level codes), or was unable to determine
the eligibility of a household or respondent (3000 level codes). The table below illustrates the codes
used by the BRFSS in 2015, and it notes the instances where codes are used only for landline telephone
or cellular telephone sample numbers.
The Disposition Code Table below uses a number of terms to define and categorize outcomes. These
include the following:
•
•

•
•
•

•

Respondent: A person who is contacted by an interviewer and who may be eligible for interview.
Private residence: Persons residing in private residences or college housing are eligible. Persons
living in group homes, military barracks or other living arrangements are not eligible. Persons
living in vacation homes for 30 days or more are eligible. Eligibility is ascertained by asking
each potential respondent whether they live in a private residence. If the respondent is unsure
whether their residence qualifies, additional definitions of residences are provided.
Landline telephone: A telephone that is used within a specific location, including traditional
household telephones, Voice Over Internet Protocol (VOIP), and Internet phones connected to
computers in a household.
Cellular telephone: A mobile device that is not tied to a specific location for use.
Selected respondent: A person who is eligible for interview. For the cellular telephone sample, a
selected respondent is an adult associated with the phone number who lives in a private residence
or college housing within the United States or territories covered by the BRFSS. For the landline
telephone sample, a selected respondent is the person chosen for interview during the household
enumeration section of the screening questions.
Personal cellular telephone: A cellular telephone that is used for personal calls. Cellular
telephones that are used for both personal and business calls may be categorized as personal
telephones and persons contacted on these phones are eligible for interview. Persons using
telephones that are exclusively for business use are not eligible for interview.

BRFSS 2015 Summary Data Quality Report

6 of 27

Table 2.
2015 Landline Telephone and Cellular Telephone BRFSS Disposition Codes
Category

Interviewed
(1000 level codes)

Eligible, Non-Interview
(2000 level codes)

Unknown Eligibility

BRFSS 2015 Summary Data Quality Report

Code

Description

1100

Completed interview

1200

Partially completed interview

2111

Household level refusal (used for landline only)

2112

Selected respondent refusal

2120

Break off/termination within questionnaire

2210

Selected respondent never available

2220

Household (nonbusiness) answering device
(used for landline only)

2320

Selected respondent physically or mentally unable to
complete interview

2330

Language barrier of selected respondent

3100

Unknown if housing unit

3130

No answer

3140

Answering device, unknown whether eligible

3150

Telecommunication barrier (i.e. call blocking)

3200

Household, not known if respondent eligible

3322

Physical or mental impairment (household level)

3330

Language barrier (household level)

3700

On never-call list

7 of 27

Table 2.
2015 Landline Telephone and Cellular Telephone BRFSS Disposition Codes
Category

Not Eligible

Code

Description

4100

Out of sample

4200

Fax/data/modem

4300

Nonworking/disconnected number

4400

Technological barrier
(i.e., fast busy, phone circuit barriers)

4430

Call forwarding/pager

4450

Cellular telephone number
(used for landline telephone only)

4460

Landline telephone number
(used for cellular telephone only)

4500

Non-residence/business

4510

Group home

4700

Household, no eligible respondent
(teen phone/minor child cellular telephone)

4900

Miscellaneous, non-eligible

Factors affecting the distribution of disposition codes by state include differences in telephone systems,
sample designs, surveyed populations, and data collection processes. Table 3 defines the categories of
disposition codes used to calculate outcome and response rates illustrated in Tables 4A through 6.
Table 3.
Categories of 2015 Landline and Cellular Telephone Disposition Codes

Category

Disposition Code
Definitions

Formulae
Abbreviation

Completed
Interviews

1100+1200

COIN

Eligible

1100+1200+2111+2112+2120+2210+2220+2320+2330

ELIG

Contacted Eligible

1100+1200+2111+2112+2120+2210+2320+2330

CONELIG

BRFSS 2015 Summary Data Quality Report

8 of 27

Table 3.
Categories of 2015 Landline and Cellular Telephone Disposition Codes

Category

Disposition Code
Definitions

Formulae
Abbreviation

Terminations and
Refusals

2111+2112+2120

TERE

Ineligible Phone
Numbers

All 4000 level disposition codes

INELIG

Unknown Whether
Eligible

All 3000 level disposition codes

UNKELIG

Eligibility Factor

ELIG/(ELIG + INELIG)

E

The disposition codes are categorized according to the groups illustrated in Table 3 to produce rates of
resolution, cooperation, completion, refusal and response. In accordance with population surveillance
standards, the proportions of people who may have been eligible for interview, but who were not able to
be interviewed, are accounted for in the formulae.
Eligibility Factor
E = ELIG/ (ELIG + INELIG)
The Eligibility Factor is the proportion of eligible phone numbers from among all sample numbers for
which eligibility has been determined. The eligibility factor, therefore, provides a measure of eligibility
that can be applied to sample numbers with unknown eligibility. The purpose of the eligibility factor is
to estimate the proportion of the sample that is likely to be eligible. The eligibility factor is used in the
calculations of refusal and response rates. Separate eligibility factors are calculated for landline
telephones and cellular telephone samples for each state and territory.

Resolution Rate
((ELIG + INELIG) / (ELIG+INELIG+UNKELIG))*100
The Resolution Rate is the percentage of numbers in the total sample for which eligibility has been
determined. The total number of eligible and ineligible sample phone numbers is divided by the total
number of phone numbers in the entire sample. The result is multiplied by 100 to calculate the
percentage of the sample for which eligibility is determined. Separate resolution rates are calculated for
landline telephone and cellular telephone samples for each state and territory.

BRFSS 2015 Summary Data Quality Report

9 of 27

Interview Completion Rate
(COIN / (COIN + TERE)) * 100
The Interview Completion Rate is the rate of completed interviews among all respondents who have
been determined to be eligible and selected for interviewing. The numerator is the number of complete
and partially completed interviews. This number is divided by the number of completed interviews,
partially completed interviews, and all break offs, refusals, and terminations. The result is multiplied by
100 to provide the percentage of completed interviews among eligible respondents who are contacted by
interviewers. Separate interview completion rates are calculated for landline telephone and cellular
telephone samples for each state and territory.

Cooperation Rate
(COIN / CONELIG) *100
The AAPOR Cooperation Rate is the number of complete and partial complete interviews divided by the
number of contacted and eligible respondents. The BRFSS Cooperation Rate follows the guidelines of
AAPOR Cooperation Rate #2. Separate cooperation rates are calculated for landline telephone and
cellular telephone samples for each state and territory.

Refusal Rate
(TERE / (ELIG + (E * UNKELIG))) * 100
The BRFSS Refusal Rate is the proportion of all eligible respondents who refused to complete an
interview or terminated an interview prior to the threshold required to be considered a partial interview.
Refusals and terminations (TERE) are in the numerator, and the denominator includes all eligible
numbers and a proportion of the numbers with unknown eligibility. The proportion of numbers with
unknown eligibility is determined by the eligibility factor (E as described above). The result is then
multiplied by 100 to provide a percentage of refusals among all eligible and likely to be eligible
numbers in the sample. Separate refusal rates are calculated for landline telephone and cellular telephone
samples for each state and territory.
Response Rate
(COIN / ((ELIG + (E * UNKELIG))) * 100
A Response Rate is an outcome rate with the number of complete and partial interviews in the
numerator and an estimate of the number of eligible units in the sample in the denominator. The BRFSS
Response Rate calculation assumes that the unresolved numbers contain the same percentage of eligible
households or eligible personal cell phones as the records whose eligibility or ineligibility are
determined. The BRFSS Response Rate follows the guidelines for AAPOR Response Rate #4. It also is
similar to the BRFSS CASRO Rates reported prior to 2011. Separate eligibility factors are calculated for
landline telephone and cellular telephone samples for each state and territory and a combined Response
BRFSS 2015 Summary Data Quality Report

10 of 27

Rate for landline telephone and cellular telephone also is calculated. The combined landline telephone
and cellular telephone response rate is generated by weighting to the respective size of the two samples.
The total sample equals the landline telephone sample plus cellular telephone sample. The proportion of
each sample is calculated using the total sample as the denominator. The formulae for the proportions of
the sample are found below:
P1 = TOTAL LANDLINE SAMPLE /
(TOTAL LANDLINE SAMPLE + TOTAL CELL PHONE SAMPLE);
P2 = TOTAL CELL PHONE SAMPLE /
(TOTAL LANDLINE SAMPLE + TOTAL CELL PHONE SAMPLE);
The formula for the Combined Landline Telephone and Cellular Telephone Weighted Response Rate,
therefore, is described below:
COMBINED RESPONSE RATE=
(P1 * LANDLINE RESPONSE RATE) + (P2 * CELL PHONE RESPONSE RATE).

Tables of Outcomes and Rates by State
The tables on the following pages illustrate calling outcomes in categories of eligibility, rates of
cooperation, refusal, resolution, and response by landline telephone and cellular telephone samples.
 Tables 4A and 4B provide information on the size of the sample and the numbers and
percentages of completed interviews, cooperation rates, terminations and refusals, and contacts
with eligible households by state and territory.
 Tables 5A and 5B provide information on the number and percentage of landline telephone and
cellular telephone sample numbers that are eligible, ineligible, and of unknown eligibility.
 Table 6 provides response rates for landline telephone samples, cellular telephone samples, and
combined samples.

BRFSS 2015 Summary Data Quality Report

11 of 27

Table 4A. Landline Sample Completions, Terminations and Refusals, Contacted Eligible Households and
Total Sample by State
COIN
State

TERE

N

%

AL

3,453

3.6

AK

2,523

AZ

COOP
Total
Sample

%

N

%

%

2,200

2.3

6,754

7.1

51.1

95,436

1.8

1,005

0.7

4,119

2.9

61.3

143,910

5,270

3.7

2,081

1.5

8,457

6.0

62.3

140,550

AR

4,008

4.0

1,698

1.7

6,633

6.7

60.4

99,270

CA

3,558

2.7

2,578

1.9

6,928

5.2

51.4

134,011

CO

7,424

5.3

1,716

1.2

10,658

7.6

69.7

139,619

CT

7,858

3.6

2,866

1.3

13,100

6.1

60.0

216,048

DE

2,191

2.7

490

0.6

3,345

4.2

65.5

80,310

DC

3,409

2.2

1,310

0.8

5,581

3.6

61.1

155,310

FL

6,087

2.8

3,113

1.4

10,793

5.0

56.4

214,890

GA

3,030

2.2

727

0.5

4,751

3.4

63.8

139,410

HI

2,795

2.2

874

0.7

4,675

3.7

59.8

126,100

ID

3,298

3.6

1,393

1.5

5,306

5.8

62.2

91,260

IL

2,856

3.7

1,062

1.4

4,744

6.2

60.2

77,040

IN

3,941

3.9

1,828

1.8

6,667

6.6

59.1

100,584

IA

3,510

5.5

1,183

1.9

5,325

8.4

65.9

63,270

KS

11,356

5.6

3,929

1.9

16,642

8.2

68.2

203,310

KY

5,345

3.2

1,133

0.7

6,827

4.1

78.3

168,480

LA

2,789

2.9

1,503

1.6

4,909

5.1

56.8

96,403

ME

6,397

6.6

1,725

1.8

9,044

9.3

70.7

97,350

MD

11,007

4.7

4,711

2.0

18,458

7.8

59.6

235,666

MA

4,276

2.6

1,970

1.2

6,527

4.0

65.5

163,001

MI

4,215

4.1

1,166

1.1

6,421

6.2

65.6

104,010

MN

8,007

5.3

1,010

0.7

10,887

7.3

73.5

150,090

MS

3,703

4.0

1,636

1.8

5,989

6.5

61.8

91,713

BRFSS 2015 Summary Data Quality Report

N

CONELIG

12 of 27

Table 4A. Landline Sample Completions, Terminations and Refusals, Contacted Eligible Households and
Total Sample by State
COIN

TERE

State

N

%

MO

4,216

5.6

MT

4,048

NE

COOP
Total
Sample

%

N

%

%

1,227

1.6

6,392

8.5

66.0

75,350

5.8

967

1.4

5,824

8.3

69.5

69,786

8,750

6.6

3,009

2.3

13,449

10.1

65.1

133,470

NV

1,826

4.4

563

1.4

2,689

6.5

67.9

41,580

NH

4,997

5.5

1,630

1.8

7,592

8.4

65.8

90,690

NJ

7,811

3.5

1,855

0.8

11,969

5.4

65.3

222,150

NM

3,958

4.5

1,690

1.9

6,491

7.5

61.0

87,120

NY

8,433

3.3

4,567

1.8

15,756

6.1

53.5

259,230

NC

2,447

5.5

946

2.1

3,853

8.7

63.5

44,490

ND

2,978

4.6

752

1.1

4,189

6.4

71.1

65,430

OH

8,706

3.8

1,490

0.7

12,228

5.4

71.2

226,350

OK

4,510

5.1

2,175

2.5

7,888

8.9

57.2

88,360

OR

2,525

3.7

1,000

1.4

3,662

5.3

69.0

69,113

PA

2,787

6.1

1,204

2.6

4,535

10.0

61.5

45,506

RI

4,001

6.3

1,459

2.3

6,347

10.0

63.0

63,390

SC

6,075

6.6

1,578

1.7

9,108

9.9

66.7

91,875

SD

4,297

3.3

980

0.8

6,105

4.7

70.4

129,240

TN

4,090

3.9

1,912

1.8

6,680

6.4

61.2

104,744

TX

9,260

3.0

4,493

1.4

16,163

5.2

57.3

312,870

UT

4,174

5.5

780

1.0

5,871

7.7

71.1

75,766

VT

3,205

7.7

689

1.7

4,380

10.6

73.2

41,400

VA

4,954

5.8

767

0.9

6,927

8.1

71.5

85,650

WA

10,162

4.1

4,256

1.7

16,507

6.7

61.6

247,770

WV

2,969

11.1

661

2.5

4,021

15.0

73.8

26,760

WI

3,176

5.1

1,650

2.7

5,400

8.8

58.8

61,710

BRFSS 2015 Summary Data Quality Report

N

CONELIG

13 of 27

Table 4A. Landline Sample Completions, Terminations and Refusals, Contacted Eligible Households and
Total Sample by State
COIN

TERE

State

N

%

WY

4,142

3.2

GU

1,259

PR

COOP
Total
Sample

%

N

%

%

1,532

1.2

6,420

5.0

64.5

128,250

6.1

244

1.2

2,290

11.1

55.0

20,690

2,598

4.9

398

0.8

3,957

7.5

65.7

52,918

Minimum

1,259

1.8

244

0.5

2,290

2.9

51.1

20,690

Maximum

11,356

11.1

4,711

2.7

18,458

15.0

78.3

312,870

Mean

4,805

4.5

1,686

1.5

7,552

7.0

64.0

118,655

Median

4,048

4.1

1,490

1.5

6,421

6.6

63.8

97,350

BRFSS 2015 Summary Data Quality Report

N

CONELIG

14 of 27

Table 4B. Cell Phone Sample Completions, Terminations and Refusals, Contacted Eligible Households and
Total Sample by State
COIN
State

TERE

N

%

AL

4,512

6.4

AK

1,134

AZ

COOP
Total
Sample

%

N

%

%

1,155

1.6

5,808

8.2

77.7

70,453

5.1

126

0.6

1,317

5.9

86.1

22,350

2,328

7.7

477

1.6

2,986

9.9

78.0

30,300

AR

1,204

7.9

227

1.5

1,530

10.0

78.7

15,300

CA

8,817

7.6

2,946

2.5

12,362

10.7

71.3

115,717

CO

6,161

10.5

683

1.2

6,986

11.9

88.2

58,680

CT

4,477

6.4

1,013

1.4

5,870

8.3

76.3

70,427

DE

2,036

5.0

292

0.7

2,520

6.2

80.8

40,350

DC

502

3.3

106

0.7

658

4.3

76.3

15,360

FL

2,925

7.1

644

1.6

3,619

8.8

80.8

41,308

GA

1,403

4.9

314

1.1

1,840

6.5

76.3

28,410

HI

4,861

9.6

828

1.6

5,881

11.6

82.7

50,759

ID

2,546

13.8

490

2.7

3,149

17.1

80.9

18,450

IL

2,314

7.4

323

1.0

2,705

8.6

85.5

31,290

IN

2,068

7.9

461

1.8

2,627

10.0

78.7

26,249

IA

2,639

13.2

218

1.1

2,935

14.7

89.9

20,010

KS

12,628

6.8

2,028

1.1

14,871

8.0

84.9

186,569

KY

3,590

4.9

405

0.6

4,067

5.6

88.3

72,930

LA

1,905

5.8

416

1.3

2,370

7.3

80.4

32,582

ME

2,761

10.2

461

1.7

3,321

12.3

83.1

27,060

MD

1,474

7.2

330

1.6

1,928

9.5

76.5

20,400

MA

5,085

3.9

1,933

1.5

7,147

5.5

71.1

129,879

MI

4,896

8.3

1,056

1.8

6,727

11.4

72.8

58,770

MN

8,362

8.3

550

0.5

9,495

9.4

88.1

100,740

MS

2,315

9.4

387

1.6

2,762

11.2

83.8

24,652

MO

2,856

9.4

281

0.9

3,242

10.6

88.1

30,478

BRFSS 2015 Summary Data Quality Report

N

CONELIG

15 of 27

Table 4B. Cell Phone Sample Completions, Terminations and Refusals, Contacted Eligible Households and
Total Sample by State
COIN

TERE

State

N

%

MT

2,060

9.3

NE

9,315

NV

%

N

136

0.6

10.8

1,089

988

8.9

NH

2,172

NJ

COOP
Total
Sample

%

%

2,251

10.1

91.5

22,226

1.3

10,890

12.7

85.5

85,890

114

1.0

1,119

10.1

88.3

11,070

7.9

512

1.9

2,860

10.4

75.9

27,510

3,781

5.8

551

0.8

4,779

7.4

79.1

64,860

NM

2,902

10.2

602

2.1

3,570

12.6

81.3

28,380

NY

3,856

6.1

1,107

1.7

5,331

8.4

72.3

63,330

NC

4,063

9.5

595

1.4

4,796

11.2

84.7

42,660

ND

2,212

7.5

330

1.1

2,648

9.0

83.5

29,460

OH

3,116

6.0

288

0.6

3,627

7.0

85.9

51,810

OK

2,312

7.1

1,079

3.3

3,627

11.1

63.7

32,646

OR

2,690

5.5

319

0.6

3,098

6.3

86.8

49,111

PA

2,798

8.9

366

1.2

3,257

10.3

85.9

31,482

RI

2,360

7.3

508

1.6

3,126

9.6

75.5

32,550

SC

5,639

10.7

697

1.3

6,493

12.4

86.8

52,541

SD

3,036

7.3

215

0.5

3,285

7.9

92.4

41,467

TN

1,746

6.8

356

1.4

2,138

8.4

81.7

25,558

TX

4,727

6.6

1,768

2.5

6,784

9.5

69.7

71,700

UT

7,553

16.0

504

1.1

8,516

18.1

88.7

47,100

VT

3,224

8.7

485

1.3

3,886

10.5

83.0

37,170

VA

3,575

8.1

405

0.9

4,280

9.7

83.5

44,250

WA

5,931

7.3

1,925

2.4

8,550

10.6

69.4

80,700

WV

3,177

10.7

281

0.9

3,512

11.8

90.5

29,728

WI

3,130

10.6

769

2.6

4,031

13.7

77.6

29,520

WY

1,378

5.9

194

0.8

1,638

7.0

84.1

23,400

GU

414

6.3

86

1.3

553

8.4

74.9

6,570

BRFSS 2015 Summary Data Quality Report

N

CONELIG

16 of 27

Table 4B. Cell Phone Sample Completions, Terminations and Refusals, Contacted Eligible Households and
Total Sample by State
COIN
State

PR

N

TERE
%

N

CONELIG
%

N

COOP

%

%

Total
Sample

2,878

15.8

248

1.4

3,306

18.2

87.1

18,194

Minimum

414

3.3

86

0.5

553

4.3

63.7

6,570

Maximum

12,628

16.0

2,946

3.3

14,871

18.2

92.4

186,569

Mean

3,523

8.1

635

1.4

4,350

9.9

81.4

45,625

Median

2,878

7.6

461

1.3

3,321

9.9

82.7

32,582

BRFSS 2015 Summary Data Quality Report

17 of 27

Table 5A. Landline Sample Categories of Eligibility by State (Landline Only)

ELIG
State

N

INELIG
%

N

UNKELIG
%

N

%

AL

9,918

10.4

76,716

80.4

8,802

9.2

AK

4,545

3.2

129,811

90.2

9,554

6.6

AZ

10,406

7.4

111,768

79.5

18,376

13.1

AR

7,220

7.3

80,374

81.0

11,676

11.8

CA

8,340

6.2

98,613

73.6

27,058

20.2

CO

11,363

8.1

110,724

79.3

17,532

12.6

CT

16,368

7.6

161,939

75.0

37,741

17.5

DE

3,555

4.4

55,474

69.1

21,281

26.5

DC

7,589

4.9

120,975

77.9

26,746

17.2

FL

13,444

6.3

162,293

75.5

39,153

18.2

GA

5,072

3.6

108,302

77.7

26,036

18.7

HI

5,770

4.6

108,195

85.8

12,135

9.6

ID

5,801

6.4

75,414

82.6

10,045

11.0

IL

7,459

9.7

60,214

78.2

9,367

12.2

IN

7,785

7.7

77,303

76.9

15,496

15.4

IA

5,595

8.8

50,592

80.0

7,083

11.2

KS

17,812

8.8

161,353

79.4

24,145

11.9

KY

7,224

4.3

134,259

79.7

26,997

16.0

LA

6,195

6.4

77,489

80.4

12,719

13.2

ME

9,802

10.1

72,418

74.4

15,130

15.5

MD

21,889

9.3

164,056

69.6

49,721

21.1

MA

7,589

4.7

110,435

67.8

44,977

27.6

MI

7,008

6.7

82,311

79.1

14,691

14.1

MN

11,176

7.4

114,287

76.1

24,627

16.4

MS

8,052

8.8

74,513

81.2

9,148

10.0

MO

6,806

9.0

58,033

77.0

10,511

13.9

MT

6,542

9.4

55,945

80.2

7,299

10.5

BRFSS 2015 Summary Data Quality Report

18 of 27

Table 5A. Landline Sample Categories of Eligibility by State (Landline Only)

ELIG
State

N

INELIG
%

N

UNKELIG
%

N

%

NE

14,778

11.1

105,613

79.1

13,079

9.8

NV

3,215

7.7

30,813

74.1

7,552

18.2

NH

9,620

10.6

65,601

72.3

15,469

17.1

NJ

12,287

5.5

156,120

70.3

53,743

24.2

NM

6,744

7.7

71,057

81.6

9,319

10.7

NY

18,893

7.3

179,534

69.3

60,803

23.5

NC

5,723

12.9

33,214

74.7

5,553

12.5

ND

4,446

6.8

54,995

84.1

5,989

9.2

OH

12,486

5.5

180,897

79.9

32,967

14.6

OK

8,232

9.3

69,618

78.8

10,510

11.9

OR

3,662

5.3

54,642

79.1

10,809

15.6

PA

4,991

11.0

31,355

68.9

9,160

20.1

RI

7,396

11.7

36,714

57.9

19,280

30.4

SC

10,129

11.0

69,666

75.8

12,080

13.1

SD

6,704

5.2

113,557

87.9

8,979

6.9

TN

9,453

9.0

81,219

77.5

14,072

13.4

TX

23,560

7.5

238,323

76.2

50,987

16.3

UT

5,934

7.8

62,700

82.8

7,132

9.4

VT

4,987

12.0

30,375

73.4

6,038

14.6

VA

7,008

8.2

62,171

72.6

16,471

19.2

WA

24,082

9.7

191,211

77.2

32,477

13.1

WV

4,489

16.8

15,984

59.7

6,287

23.5

WI

6,439

10.4

46,987

76.1

8,284

13.4

WY

9,342

7.3

103,627

80.8

15,281

11.9

GU

2,430

11.7

16,225

78.4

2,035

9.8

PR

4,188

7.9

42,948

81.2

5,782

10.9

Minimum

2,430

3.2

15,984

57.9

2,035

6.6

BRFSS 2015 Summary Data Quality Report

19 of 27

Table 5A. Landline Sample Categories of Eligibility by State (Landline Only)

ELIG
State

N

INELIG
%

N

UNKELIG
%

N

%

Maximum

24,082

16.8

238,323

90.2

60,803

30.4

Mean

8,897

8.1

91,301

76.9

18,456

15.0

Median

7,396

7.7

77,303

77.9

13,079

13.4

BRFSS 2015 Summary Data Quality Report

20 of 27

Table 5B. Cell Phone Sample Categories of Eligibility by State (Cell Phone Only)

ELIG
State

N

INELIG
%

N

UNKELIG
%

N

%

AL

5,808

8.2

38,289

54.3

26,356

37.4

AK

1,317

5.9

16,761

75.0

4,272

19.1

AZ

2,986

9.9

13,406

44.2

13,908

45.9

AR

1,530

10.0

7,804

51.0

5,966

39.0

CA

12,362

10.7

42,194

36.5

61,161

52.9

CO

6,986

11.9

26,710

45.5

24,984

42.6

CT

5,870

8.3

26,903

38.2

37,654

53.5

DE

2,520

6.2

17,619

43.7

20,211

50.1

DC

658

4.3

8,375

54.5

6,327

41.2

FL

3,619

8.8

15,328

37.1

22,361

54.1

GA

1,840

6.5

13,838

48.7

12,732

44.8

HI

5,881

11.6

17,730

34.9

27,148

53.5

ID

3,149

17.1

7,624

41.3

7,677

41.6

IL

2,705

8.6

14,968

47.8

13,617

43.5

IN

2,627

10.0

11,220

42.7

12,402

47.2

IA

2,935

14.7

9,302

46.5

7,773

38.8

KS

14,871

8.0

108,824

58.3

62,874

33.7

KY

4,067

5.6

38,581

52.9

30,282

41.5

LA

2,370

7.3

16,935

52.0

13,277

40.7

ME

3,321

12.3

11,343

41.9

12,396

45.8

MD

1,928

9.5

8,672

42.5

9,800

48.0

MA

7,147

5.5

63,248

48.7

59,484

45.8

MI

6,727

11.4

31,130

53.0

20,913

35.6

MN

9,495

9.4

49,818

49.5

41,427

41.1

MS

2,762

11.2

12,874

52.2

9,016

36.6

MO

3,242

10.6

14,676

48.2

12,560

41.2

BRFSS 2015 Summary Data Quality Report

21 of 27

Table 5B. Cell Phone Sample Categories of Eligibility by State (Cell Phone Only)

ELIG
State

N

INELIG
%

N

UNKELIG
%

N

%

MT

2,251

10.1

12,573

56.6

7,402

33.3

NE

10,890

12.7

50,105

58.3

24,895

29.0

NV

1,119

10.1

4,424

40.0

5,527

49.9

NH

2,860

10.4

12,106

44.0

12,544

45.6

NJ

4,779

7.4

29,042

44.8

31,039

47.9

NM

3,570

12.6

14,898

52.5

9,912

34.9

NY

5,331

8.4

26,059

41.1

31,940

50.4

NC

4,796

11.2

19,648

46.1

18,216

42.7

ND

2,648

9.0

16,639

56.5

10,173

34.5

OH

3,627

7.0

25,955

50.1

22,228

42.9

OK

3,627

11.1

19,082

58.5

9,937

30.4

OR

3,098

6.3

18,746

38.2

27,267

55.5

PA

3,257

10.3

12,974

41.2

15,251

48.4

RI

3,126

9.6

13,737

42.2

15,687

48.2

SC

6,493

12.4

23,979

45.6

22,069

42.0

SD

3,285

7.9

24,857

59.9

13,325

32.1

TN

2,138

8.4

11,441

44.8

11,979

46.9

TX

6,784

9.5

35,367

49.3

29,549

41.2

UT

8,516

18.1

21,749

46.2

16,835

35.7

VT

3,886

10.5

16,298

43.8

16,986

45.7

VA

4,280

9.7

20,701

46.8

19,269

43.5

WA

8,550

10.6

31,098

38.5

41,052

50.9

WV

3,512

11.8

12,035

40.5

14,181

47.7

WI

4,031

13.7

14,850

50.3

10,639

36.0

WY

1,638

7.0

15,567

66.5

6,195

26.5

GU

553

8.4

4,330

65.9

1,687

25.7

PR

3,306

18.2

6,000

33.0

8,888

48.9

BRFSS 2015 Summary Data Quality Report

22 of 27

Table 5B. Cell Phone Sample Categories of Eligibility by State (Cell Phone Only)

ELIG
State

N

INELIG
%

N

UNKELIG
%

N

%

Minimum

553

4.3

4,330

33.0

1,687

19.1

Maximum

14,871

18.2

108,824

75.0

62,874

55.5

Mean

4,350

9.9

21,836

48.0

19,439

42.1

Median

3,321

9.9

16,639

46.5

14,181

42.7

BRFSS 2015 Summary Data Quality Report

23 of 27

Table 6. Response Rates for Landline Telephone, Cellular Telephone, and Combined Samples

State

Landline Response
Rate

Cell Phone
Response Rate

Combined Response
Rate

AL

31.6

48.6

38.8

AK

51.8

69.6

54.2

AZ

44.0

42.2

43.7

AR

49.0

48.0

48.9

CA

34.0

33.6

33.9

CO

57.1

50.6

55.2

CT

39.6

35.5

38.6

DE

45.3

40.3

43.6

DC

37.2

44.9

37.9

FL

37.0

37.1

37.0

GA

48.6

42.1

47.6

HI

43.8

38.4

42.2

ID

50.6

47.2

50.0

IL

33.6

48.3

37.9

IN

42.8

41.5

42.6

IA

55.7

55.0

55.5

KS

56.2

56.3

56.2

KY

62.1

51.6

59.0

LA

39.1

47.6

41.2

ME

55.1

45.1

52.9

MD

39.7

39.7

39.7

MA

40.8

38.6

39.8

MI

51.7

46.9

49.9

MN

59.9

51.9

56.7

MS

41.4

53.2

43.9

MO

53.3

51.8

52.9

MT

55.4

61.0

56.8

BRFSS 2015 Summary Data Quality Report

21 of 27

Table 6. Response Rates for Landline Telephone, Cellular Telephone, and Combined Samples

State

Landline Response
Rate

Cell Phone
Response Rate

Combined Response
Rate

NE

53.4

60.7

56.3

NV

46.5

44.2

46.0

NH

43.1

41.3

42.7

NJ

48.2

41.3

46.6

NM

52.4

52.9

52.5

NY

34.2

35.9

34.5

NC

37.4

48.5

42.9

ND

60.9

54.7

58.9

OH

59.6

49.1

57.6

OK

48.3

44.3

47.2

OR

58.2

38.6

50.0

PA

44.6

44.3

44.5

RI

37.6

39.1

38.1

SC

52.1

50.4

51.5

SD

59.6

62.7

60.4

TN

37.5

43.4

38.6

TX

32.9

41.0

34.4

UT

63.7

57.0

61.1

VT

54.9

45.1

50.2

VA

57.1

47.2

53.7

WA

36.7

34.1

36.0

WV

50.6

47.3

48.9

WI

42.7

49.7

45.0

WY

39.1

61.9

42.6

GU

46.7

55.6

48.9

PR

55.3

44.5

52.5

Minimum

31.6

33.6

33.9

BRFSS 2015 Summary Data Quality Report

22 of 27

Table 6. Response Rates for Landline Telephone, Cellular Telephone, and Combined Samples

State

Landline Response
Rate

Cell Phone
Response Rate

Combined Response
Rate

Maximum

63.7

69.6

61.1

Mean

47.4

47.2

47.1

Median

48.2

47.2

47.2

BRFSS 2015 Summary Data Quality Report

23 of 27

References
1. Pierannunzi C, Town M, Garvin W, Shaw F, Balluz L. Methodologic changes in the Behavioral Risk
Factor Surveillance System in 2011 and potential effects on prevalence estimates.
MMWR.2012;61(22):410-413. http://www.cdc.gov/mmwr/preview/mmwrhtml/mm6122a3.htm .
Accessed September 5, 2015.
2. The American Association for Public Opinion Research. Standard Definitions: Final Dispositions of
Case Codes and Outcome Rates for Surveys website
http://www.aapor.org/AAPOR_Main/media/publications/Standard-Definitions20169theditionfinal.pdf
Accessed June 14, 2016.
3. The Council of American Survey Research Organizations. 2013. Code of Standards and Ethics for
Market, Opinion, and Social Research website.
www.casro.org/resource/resmgr/code/september_2013_revised_code.pdf?hhSearchTerms=%22casro+an
d+response+and+rate%22. Accessed September 5, 2015.
4. The Pew Research Center for People and the Press. 2012. Assessing the Representativeness of Public
Opinion Surveys website. http://www.people-press.org/files/legacypdf/Assessing%20the%20Representativeness%20of%20Public%20Opinion%20Surveys.pdf . Accessed
September 5, 2015.
5. Groves, RM. Nonresponse rates and nonresponse bias in household surveys. Public Opinion
Quarterly. 2006;70(5):646-675.

BRFSS 2015 Summary Data Quality Report

23 of 27