Attachment X - Differences in Responses Rates in the Consumer Expenditure Survey

Attachment X - Differences in Response Rates in the Consumer Expenditure Survey - Brian - May 2018.docx

Consumer Expenditure Surveys: Quarterly Interview and Diary

Attachment X - Differences in Responses Rates in the Consumer Expenditure Survey

OMB: 1220-0050

Document [docx]
Download: docx | pdf

U .S. Department of Labor Bureau of Labor Statistics

2 Massachusetts Ave. N.E.

Washington, D.C. 20212




May 25, 2018


MEMORANDUM FOR: David Swanson, Branch Chief

Branch of Consumer Expenditure Survey

Price Statistical Methods Division

Office of Prices and Living Conditions


MEMORANDUM FROM: Brian Nix, Mathematical Statistician

Branch of Consumer Expenditure Survey

Price Statistical Methods Division

Office of Prices and Living Conditions


SUBJECT: Differences in Response Rates in the Consumer Expenditure Survey



1. Introduction


This memo documents the codes and formulae used to calculate the response rates in the Consumer Expenditure Survey (CE). It updates a similar memo written by Sally Reyes-Morales in 2011 showing more recent data as well as changes made in the processing of response rate codes in the intervening years.


The CE consists of two separate surveys, a quarterly Interview survey (CEQ) and a two-week Diary survey (CED). Response rates are computed separately for each survey, because each survey has its own questionnaire and its own sample of households.1 The CEQ and CED are collected and processed separately, due to the differences in format and design. The United States Census Bureau (Census) carries out data collection for both surveys under contract with the Bureau of Labor Statistics (BLS).


Response rates are calculated for both surveys at different stages of data processing by Census and BLS. Differences between these response rates are the result of changes made during data processing at both agencies, and also differences in the ways interviews and noninterviews are defined at the two agencies. Response rates are calculated by two groups at Census (the Field Division, and the Demographic Statistical Methods Division, or DSMD)2, and by one group at BLS (the Statistical Methods Division, or CESMD), hence response rates are calculated by three groups for two surveys. This makes six response rates – three response rates for the Interview survey, and three response rates for the Diary survey.


The same formula is used for all six response rate computations:



where the denominator is the number of “eligible” housing units in the sample, and the numerator is the number of eligible housing units that completed an interview. The basic concept is straightforward, but the three offices have different definitions of “eligible” housing units and “completed interviews,” which cause their response rates to be different.3


2. Basic Concepts


A few basic concepts need to be discussed at this point: eligible/ineligible, in-scope/out-of-scope, completed interviews, and Type A/B/C noninterviews. But before discussing these concepts, the more basic concept of a “case” needs to be defined.


A “case” is the most basic element of a field representative’s (FR’s) workload. Every month FRs are given a list of cases to contact, from which they are expected to get interviews. Their initial “caseloads” are a list of addresses, since that is the basic sample unit that Census draws from its sampling frame. However, after visiting the addresses, some turn out to be bad addresses – some turn out to be vacant housing units, nonexistent addresses, or nonresidential buildings, while others turn out to be housing units whose occupants are not in the CE’s target population, such as nursing homes, military barracks, or prisons. These addresses are removed from the FR’s caseloads upon their discovery. Furthermore, separate records are created for each consumer unit (CU) in a housing unit, so that ultimately each CU becomes a separate case. Thus, the FR’s initial caseloads are addresses, but in subsequent waves of the survey they become housing units, which then become consumer units. For the purpose of clarity, this mixture of addresses, housing units, and consumer units are called “cases4.”


Now that cases are defined, the next basic concepts that needs to be discussed are the different ways that outcomes of the cases are categorized. The different outcomes of a case for either the Interview or Diary survey are categorized in three main ways: in-scope/out-of-scope, eligible/ineligible, and by type (completed interview, or Type A/B/C noninterview).


A case is considered “in-scope” for the survey if it has a residential housing unit; otherwise it is considered “out-of-scope.” Examples of in-scope cases include houses, condominiums, and apartment buildings. Examples of out-of-scope cases include addresses that could not be found or do not exist, addresses where the housing unit was demolished or converted to a nonresidential use, and addresses located on military bases.


A case is considered “eligible” to participate in the survey if it has a residential housing unit that is occupied by its usual residents; otherwise it is “ineligible.” Examples of ineligible cases include housing units which are vacant or under construction, and those which are occupied by people whose usual residence is elsewhere.


Cases are also categorized into four other types of outcomes: completed interviews, Type A noninterviews, Type B noninterviews, and Type C noninterviews. A “completed interview” is where the sample address has an occupied housing unit whose members are in CE’s target population (the civilian noninstitutional population), and who have successfully completed an interview. A “Type A” noninterview is where the sample address has an occupied housing unit whose members are in CE’s target population, but who did not complete an interview. A “Type B” noninterview is where the sample address has an unoccupied housing unit, or an occupied unit whose occupants are not in CE’s target population. Finally, a “Type C” noninterview is where the sample address does not have a housing unit.


Both completed interviews and Type A nonrespondents are considered eligible. Type A cases are most often refusals5, as well as occupied households which cannot be contacted despite repeated contact attempts.


Thus, Type A cases are eligible, and thus part of the denominator of the response rate, which is also used as a success rate for field representatives. Type B cases are not eligible, but are part of the denominator of a calculated eligibility rate, out of all residential addresses. Type C cases are considered out-of-scope.


Once a case is determined to be Type C, it is never visited again; it is presumed that a Type C case by definition cannot “improve” to a good interview, or to a Type A or B noninterview. If the first interview attempt is classified as a Type C case, then the system at BLS will automatically generate a Type C case for the second and subsequent interview numbers, but these subsequent Type C cases are not included in the totals shown in our internal memos, because the purpose of BLS response rates is to measure the success of the interview process, and the address has not been revisited. However, the purpose of Type C totals in the Census response rates is to measure the quality of the sampling frame. Therefore, Census includes subsequent noninterviews of Type C cases in their totals, even though the address is not revisited, in order to keep the total number of addresses in the sampling frame consistent.


The response rate (Good Interviews / Eligible Cases) is one of the three rates that can be determined from the total numbers in each category of case. The other two are the eligibility rate (Eligible Cases / In-Scope Cases), useful for measuring the quality of the sample, and the in-scope rate (In-Scope Cases / All Cases), useful for measuring the quality of the sampling frame. The categories of cases by Census and BLS can be seen in this diagram:


Categories of Cases Used by Census and BLS


3. Monthly Reports from Census


Every month, Census sends BLS two progress reports on the collection process, one showing response rates for CEQ and CED calculated by the Field division, and another showing the “DSMD” response rates for CEQ and CED provided by ADDP. In this memo, the response rates presented in these two reports are compared with the response rates calculated by CESMD.


The first progress report, “CE Comparison and Progress Final Report,” includes response rates (“Field RR”) calculated using the outcome codes pulled directly from the Regional Office Survey Control Operation (ROSCO) system.


The second progress report, “CE Diary Regional Office Monthly Rates and CE Quarterly Regional Office Monthly Rates,” includes the response rates (“DSMD RR”) formerly calculated by the Census DSMD office6, at both the national level and for each of the twelve regional offices. These response rates are calculated using the final Census outcome codes after data processing has finished, which is why the response rates in this progress report are slightly different from those in the first progress report.


4. Data Processing at BLS


Monthly data are received at BLS from the Census Bureau for both surveys and processed by the Initial Edit System (IES), formerly known as the Phase 2 system. The IES has several screening processes in place to ensure data quality. One of these processes is the minimal expenditure edit, a process which screens out cases with no entries or unusually low reported total expenditures, and reclassifies their response status from interviews to noninterviews. The minimal expenditure edit has been in place in the Diary survey since 2002, and in the Interview survey since 2006.


In the minimal expenditure edit in the Interview survey, households are selected by an automated process to be manually reviewed on an individual basis before being reclassified. However, in the Diary survey, the minimal expenditure edit is an entirely automated process, and none of the selected diaries are manually reviewed before being reclassified as noninterviews; these are what are called “reclassified cases.”


The next stage of processing is the Edit and Estimation System (EES), formerly known as Phase 3. After the EES is completed, the official response rates are calculated by CESMD, using the variable OUTCOME for the CEQ and the variable PICKCODE for the CED.


These outcome codes are generated at Census, but not all codes generated by Census are applicable at BLS. For example, Census treats the two weeks of the Diary Survey as one case, but BLS treats the two weeks as separate cases, each with their own outcome code7. Some outcome codes used by Census such as 206 (“Week 1 Type A, Week 2 interview”) are not used at BLS; instead the two weeks are given two separate outcome codes, in this example given a Type A code and a completed interview response code respectively.


(For complete OUTCOME and PICKCODE definitions, see Attachment A.)


5. CEQ Response Rates


Here is the formula used for all three response rate calculations (BLS, “DSMD,” and “Field”) for CEQ:


Since all three offices use the same formula it is natural to think all three response rates would be the same. However, relatively small differences exist in spite of using the same formula.


There are several reasons Field, “DSMD,” and CESMD have different response rates in spite of using the same formula. The first reason is that, prior to the 2010-Census based revision of the CE Survey, which took effect in 2015, the CEQ consisted of five interviews; Census used all five interviews in its response rate calculations, but BLS used only the last four interviews. The first interview was a “bounding interview,” which provided baseline data and was designed to remove out-of-scope expenditures that respondents tended to report through a flawed cognitive process called “telescoping.”8 Expenditure data was collected in that interview, but it was not used in CE’s final published expenditure estimates. Census included the bounding interview in its response rate calculations, because it was collected. BLS did not include the bounding interview in its response rate calculations, because it was not used in the published expenditure estimates. However, the bounding interview no longer exists, so the discrepancy no longer exists.

This illustrates a second reason for different response rates between agencies, which is that Census and BLS are really measuring different concepts. Census is measuring the performance of its field representatives over all their assigned cases, while BLS is measuring the field representatives’ ability to collect usable data for the final expenditure estimates. Despite the end of the bounding interview, there are still small differences in the numbers used in the response rate formulae, such as whether incomplete (partial) interviews should be counted as “good” interviews. The differences of these estimates can be observed in Graph 1 and Graph 2 below. The monthly response rates from Census have historically been slightly higher than those from BLS, but this difference has decreased in the past two years, with a difference between “DSMD” and BLS of 0.045 percent in 2015 and 0.022 percent in 2016. (See complete list in Attachment B.) Notice on the graphs that the Field and “DSMD” data points are mostly the same, and the BLS data points are as well for 2015 and 2016. (Also visible on the graphs is the result in lost productivity from the October 2013 federal government shutdown.)


While BLS response rates are different from Census response rates, monthly and annual response rates calculated at Census by Field and “DSMD” differ very little. The differences of the reported monthly response rates between the two sources has been consistently less than one percentage point since 2005, and the rates have been identical for the majority of the months over that time. The largest difference between the two was 0.63 in September 2016. (See complete list in Attachment B.)


6. CED Response Rates


Although the response rate formula shown in the introduction is used to calculate all the response rates discussed in this study, differences in the definition of a good interview and the eligibility of a noninterview can result in big differences among them. The classification variable used in the CED is called PICKCODE.


Based on the classification of PICKCODE values, here are the current CED response rate formulae:


The BLS version of the formula is as follows:


The two Census response rates (“Field” and “DSMD”) are now calculated the same as each other:


However, prior to 2017, the Census response rates used these formulae:


old Field CED Response Rate:

old “DSMD” CED Response Rate:

Because Census considers the two weeks of the Diary Survey to be a single record, there are several outcome codes for Diary which are non-applicable for us at BLS.9 For example, the codes 203 and 206 consist of, from the BLS point of view, a good interview one week and a Type A non-interview the other week. However, Census considers these codes to be good interviews for the two weeks combined, thus partially explaining the higher response rates reported by Census.


Monthly and annual Field and “DSMD” response rates calculated at Census differed prior to 2016 largely because the cases coded as PICKCODE = 217 (“Temporarily Absent”) were classified as Type A noninterviews for the DSMD response rate calculations, but Type B noninterviews for the Field response rate calculations. However, in 2016, Field began classifying Temporarily Absent cases as Type A’s as well.

Temporarily Absent cases are those where the CU is not at home during the interview period due to a vacation or business trip for less than six months. Therefore, they have no expenditures at home during that week, and as such, BLS considers them complete diaries with a valid total expenditure of $0, although this has recently been of some concern at BLS.


The formulae above show the “reclassified cases,” which are considered complete interviews by Census, but as Type A noninterviews by BLS. As discussed above, “reclassified cases” are those interviews automatically selected by the minimal expenditure edit to be considered Type A cases.


The only other change in PICKCODE values, as shown in the formulae above, occurred in the last year at Census. Prior to 2017, an outcome of 326 was classified as a Type B noninterview for both Field and “DSMD,” but in 2017 BLS directed Census to change the classification of the 326 outcome code to a Type A. Consequently, all three response rate calculations are now made using 326 as a Type A, but this was another difference in the past between Census and BLS response rates for CED.


As shown above, the list of PICKCODE values classified as Type C noninterviews has always been constant across Census and BLS.


While the differences among the three types of response rates are very small for the CEQ survey, the differences for the CED survey can be quite big. These differences are the result of a combination of factors that include changes made during data processing at Census, changes made during the IES reclassification process at BLS, and differences in the classification of interviews and noninterview cases between Field, “DSMD,” and BLS (especially prior to the changes made in the past two years.)


Graph 3 and Graph 4 show the differences between the response rates, and the complete list of response rates is shown in Attachment C. (Visible in the graph is the sharp drop in productivity caused by the October 2013 federal government shutdown, which affected the weekly CED survey much more than the quarterly CEQ survey.)


For reference, I have appended the e-mails I received from Census. (Attachment D).


7. Further Reading


Bass, S. (2008). “Reclassifying Low-Expenditure Consumer Units in the Consumer Expenditure Interview Survey.” Consumer Expenditure Survey Anthology, 2008. (https://www.bls.gov/cex/anthology08/csxanth6.pdf)


Elkin, I. (2013). “Recommendation Regarding the Use of a CE Bounding Interview.” BLS research paper. (https://www.bls.gov/cex/research_papers/pdf/Recommendation-Regarding-the-Use-of-a-CE-Bounding-Interview.pdf)


Hackett, C. (2011). “Consumer Expenditure Quarterly Interview Survey: Effectiveness of the Bounding Interview.” BLS internal report.


Neter, J. and Waksberg, J. (1964). “A study of response errors in expenditure data from household interviews.” Journal of the American Statistical Association, 59, 18-55.


Reyes-Morales, S. (2011). “Consumer Expenditure Survey: Differences in Response Rates.” BLS internal report.


Tseng, N., and Tan, L. (2009). “An exploration of the effectiveness of the Bounding Interview in the Consumer Expenditure Interview Survey.” BLS internal report.





OPLC/SMD/Nix/05252018/A1801.5


cc: D. Swanson

S. King

S. Krieger

S. Reyes-Morales

B. Steinberg




Graph 1: CEQ Monthly Response Rates (2010-2017)




Graph 2: CEQ Annual Response Rates (2005-2017)



Graph 3: CED Monthly Response Rates (2010-2017)




Graph 4: CED Annual Response Rates (2004-2017)



ATTACHMENT A: Complete (Current) Response Code Classifications



Selected variable is OUTCOME for CEQ, PICKCODE for CED.

( = Good Interview, A = Type A Noninterview, B = Type B Noninterview, C = Type C Noninterview)



CEQ

CED

Code

Description in Data Dictionary

Notes

201

Completed Interview



203

CEQ: Transmit, no more follow-up possible (Through Section 20 complete)

CED: Week 1 Interview, Week 2 Type A

CED: Two-week code not used by BLS; Census considers complete interview.



204

Week 1 Interview, Week 2 Type B/C

CED: Two-week code not used by BLS; Census considers complete interview. Not used after 2016.



206

Week 1 Type A, Week 2 Interview

CED: Two-week code not used by BLS; Census considers complete interview.



207

Week 1 Type B/C, Week 2 Interview

CED: Two-week code not used by BLS; Census considers complete interview.

Not used after 2016.



210

Type A one week, Type B/C the other week

CED: Two-week code not used by BLS; Census considered to be Type A.

Not used after 2016.

A

217

Temporarily Absent

CED: Type A for Census

A

-

215

Insufficient Partial


A

A

216

No one home, unable to contact


A

A

219

Other Type A – Specify

CED: Includes reclassified diaries (reclassified from code 201)


A

320

Week 2 Diary Picked Up Too Early

CED: Not used before 2017. Not a final outcome code for Census.

A

A

321

Refused, Hostile respondent


A

A

322

Refused, Time-related excuses


A

A

323

Refused, Language problems


A

A

324

Refused, Other – Specify


-

A

325

Diary placed Too Late


-

A

326

Blank Diary, majority of items recalled without receipts

CED: Type B at Census before 2017.

B

B

224

All persons under 16


B

B

225

Occupied by persons with usual residence elsewhere


B

B

226

Vacant for rent


B

B

228

Unfit, to be demolished


B

B

229

Under construction


B

B

231

Unoccupied tent/trailer site


B

B

232

Permit granted, construction not started


B

B

233

Other Type B – Specify


B

B

331

Vacant for sale


B

B

332

Vacant other – Specify


C

C

240

Demolished


C

C

241

House or trailer moved


C

C

243

Converted to permanent nonresidential use


C

C

244

Merged units within the same structure


C

C

245

Condemned


C

C

246

Built after April 1, 1990

Not used after 2015.

C

C

247

Unused serial number on listing sheet


C

C

248

Other Type C – Specify


C

C

252

Located on military base or post


C

C

256

Removed during subsampling

Not used after 2015.

C

C

257

Unit already had a chance of selection

Not used after 2015.

C

C

258

Unlocated sample address

Not used before 2015.

C

C

259

Unit does not exist or is out-of-scope

Not used before 2015.

C

-

290

Spawned in error


C

C

341

Household replaced


C

C

342

CU merged with other CE CU within same address




As mentioned in the main text, there have been two changes in the classification of response codes, both affecting only Diary (CED) at Census:

217 (“Temporarily Absent”) changed for Field in 2016 from Type B to Type A, but was always Type A for “DSMD.”

326 (“Blank Diary, majority of items recalled without receipts”) changed for Census (both Field and “DSMD”) from Type B to Type A (now matching BLS) in 2017.



ATTACHMENT B: Consumer Expenditure Survey Response Rates: Quarterly Interview Survey (CEQ)




FIELD

DSMD

Field -

BLS

DSMD -

Month

RR

RR

DSMD

RR

BLS

201001

75.55

75.55

0.00

75.26

0.30

201002

74.09

74.09

0.00

73.84

0.25

201003

75.50

75.49

0.01

74.74

0.75

201004

74.85

74.85

0.00

74.72

0.14

201005

73.48

73.48

0.00

73.10

0.38

201006

74.74

74.74

0.00

74.43

0.31

201007

72.54

72.54

0.00

71.43

1.10

201008

74.14

74.14

0.00

73.87

0.27

201009

74.00

74.00

0.00

74.09

-0.09

201010

72.63

72.63

0.00

71.99

0.65

201011

73.15

73.15

0.00

72.87

0.28

201012

71.29

71.29

0.00

70.81

0.48

201101

71.75

71.75

0.00

71.31

0.43

201102

71.70

71.70

0.00

70.49

1.21

201103

72.34

72.34

0.00

70.69

1.65

201104

71.26

71.26

0.00

70.55

0.71

201105

72.22

72.22

0.00

71.22

1.00

201106

70.96

70.96

0.00

69.40

1.56

201107

70.74

70.74

0.00

69.32

1.43

201108

72.02

72.02

0.00

70.82

1.20

201109

70.89

70.89

0.00

69.41

1.48

201110

71.82

71.82

0.00

70.50

1.31

201111

71.52

71.52

0.00

70.64

0.89

201112

70.68

70.68

0.00

70.19

0.48

201201

71.29

71.29

0.00

70.45

0.84

201202

71.83

71.83

0.00

71.45

0.38

201203

70.55

70.55

0.00

69.98

0.57

201204

71.44

71.44

0.00

71.03

0.40

201205

70.78

70.78

0.00

70.51

0.27

201206

68.62

68.62

0.00

67.90

0.72

201207

69.20

69.20

0.00

69.28

-0.08

201208

70.23

70.23

0.00

69.84

0.39

201209

68.01

68.01

0.00

67.67

0.35

201210

71.30

71.30

0.00

70.57

0.73

201211

70.72

70.72

0.00

69.57

1.15

201212

66.34

66.34

0.00

65.79

0.55

201301

69.86

69.86

0.00

69.43

0.43

201302

69.77

69.77

0.00

69.86

-0.09

201303

67.60

67.60

0.00

66.94

0.66

201304

69.46

69.46

0.00

69.63

-0.17

201305

70.52

70.52

0.00

70.03

0.50

201306

67.92

67.92

0.00

67.31

0.61

201307

69.43

69.43

0.00

68.20

1.24

201308

68.53

68.53

0.00

67.57

0.96

201309

66.09

66.09

0.00

65.72

0.37

201310

56.36

56.36

0.00

56.27

0.09

201311

65.66

65.66

0.00

65.68

-0.02

201312

64.01

64.01

0.00

63.86

0.15

201401

67.00

67.00

0.00

66.01

0.98

201402

66.97

66.97

0.00

66.00

0.97

201403

67.34

67.34

0.00

66.83

0.51

201404

68.16

68.16

0.00

67.06

1.10

201405

68.93

68.93

0.00

68.10

0.83

201406

67.18

67.18

0.00

66.01

1.17

201407

68.24

68.24

0.00

67.12

1.12

201408

68.64

68.64

0.00

67.55

1.09

201409

66.45

66.45

0.00

65.73

0.72

201410

68.76

68.76

0.00

67.91

0.85

201411

66.36

66.36

0.00

66.15

0.22

201412

63.28

63.28

0.00

62.71

0.57

201501

65.63

65.63

0.00

65.60

0.03

201502

63.80

63.80

0.00

63.76

0.04

201503

64.29

64.29

0.00

64.25

0.04

201504

67.93

67.91

0.02

67.88

0.03

201505

65.14

65.14

0.00

65.06

0.09

201506

63.69

63.69

0.00

63.69

0.00

201507

65.91

65.75

0.16

65.69

0.06

201508

65.73

65.73

0.00

65.64

0.09

201509

62.99

63.01

-0.02

62.95

0.06

201510

63.15

63.15

0.00

63.12

0.03

201511

63.76

63.76

0.00

63.70

0.06

201512

61.78

61.78

0.00

61.78

0.00

201601

63.74

63.74

0.00

63.74

0.00

201602

63.75

63.75

0.00

63.69

0.06

201603

63.08

63.04

0.04

63.00

0.04

201604

63.95

63.89

0.07

63.83

0.06

201605

61.86

61.86

0.00

61.90

-0.03

201606

62.60

62.60

0.00

62.62

-0.02

201607

63.26

63.26

0.00

63.26

0.00

201608

64.19

64.19

0.00

64.23

-0.05

201609

62.48

63.11

-0.63

62.98

0.13

201610

62.37

62.37

0.00

62.34

0.03

201611

63.04

63.04

0.00

63.04

0.00

201612

61.58

61.58

0.00

61.52

0.05

201701

61.40

61.40

0.00

61.37

0.03

201702

60.68

60.68

0.00

60.69

-0.02

201703

62.09

62.09

0.00

62.09

0.00

201704

64.28

64.28

0.00

62.97

1.31

201705

61.63

61.63

0.00

61.43

0.20

201706

61.42

61.42

0.00

61.14

0.28

201707

61.53

61.53

0.00

61.34

0.19

201708

62.58

62.58

0.00

62.36

0.22

201709

60.05

60.05

0.00

59.88

0.17

201710

60.39

60.39

0.00

60.19

0.20

201711

60.24

60.24

0.00

60.05

0.19

201712

57.47

57.47

0.00

57.44

0.03






FIELD

DSMD

Field -

BLS

DSMD -

Year

RR

RR

DSMD

RR

BLS

2005

75.60

75.56

0.04

74.53

1.03

2006

77.02

77.02

0.00

76.55

0.47

2007

74.22

74.22

0.00

73.86

0.36

2008

74.34

74.34

0.00

73.84

0.50

2009

75.06

75.06

0.00

74.50

0.56

2010

73.83

73.83

0.00

73.43

0.40

2011

71.49

71.49

0.00

70.38

1.11

2012

70.03

70.03

0.00

69.51

0.52

2013

67.09

67.09

0.00

66.70

0.39

2014

67.27

67.27

0.00

66.43

0.85

2015

64.46

64.45

0.01

64.40

0.04

2016

62.99

63.03

-0.04

63.01

0.02

2017

61.14

61.14

0.00

60.90

0.24



ATTACHMENT C: Consumer Expenditure Survey Response Rates: Diary Survey (CED)



FIELD

DSMD

Field -

BLS

DSMD -

Month

RR

RR

DSMD

RR

BLS

201001

79.17

74.62

4.55

73.93

0.69

201002

76.91

72.03

4.88

73.70

-1.66

201003

80.02

75.77

4.26

73.66

2.11

201004

78.07

75.20

2.87

73.59

1.61

201005

75.21

70.93

4.29

70.92

0.01

201006

74.37

69.45

4.92

69.29

0.16

201007

75.53

70.58

4.95

70.21

0.37

201008

75.84

69.76

6.08

72.07

-2.31

201009

77.42

73.87

3.55

71.43

2.44

201010

77.63

73.70

3.93

70.92

2.79

201011

76.60

72.66

3.94

71.33

1.33

201012

71.53

66.81

4.72

67.15

-0.34

201101

76.63

72.80

3.83

72.00

0.80

201102

76.99

73.40

3.58

72.51

0.89

201103

76.51

72.20

4.31

71.02

1.18

201104

77.85

72.29

5.55

73.68

-1.38

201105

77.53

73.00

4.53

70.09

2.91

201106

76.68

71.78

4.90

69.95

1.83

201107

77.19

71.20

6.00

70.96

0.24

201108

76.89

70.93

5.96

70.63

0.30

201109

72.19

67.19

5.00

67.36

-0.17

201110

73.93

69.85

4.08

70.38

-0.53

201111

71.27

67.28

3.99

66.56

0.72

201112

72.18

66.86

5.33

68.14

-1.28

201201

74.73

70.26

4.47

70.40

-0.14

201202

73.02

69.31

3.71

68.87

0.43

201203

74.78

71.03

3.75

70.81

0.22

201204

75.23

71.20

4.03

71.28

-0.08

201205

73.25

67.68

5.57

66.96

0.72

201206

74.97

70.66

4.31

70.68

-0.02

201207

71.86

66.81

5.05

68.75

-1.94

201208

70.53

66.42

4.11

66.27

0.15

201209

71.21

67.64

3.57

65.86

1.78

201210

70.16

66.55

3.61

66.28

0.27

201211

68.58

65.59

2.99

63.69

1.90

201212

69.03

64.24

4.79

64.23

0.01

201301

69.80

66.08

3.71

64.61

1.47

201302

69.81

66.52

3.28

65.10

1.43

201303

67.99

63.65

4.35

63.78

-0.13

201304

69.11

65.84

3.27

64.36

1.48

201305

74.04

71.00

3.04

67.75

3.25

201306

71.23

66.81

4.42

67.12

-0.31

201307

73.16

69.22

3.94

69.49

-0.27

201308

71.76

66.65

5.11

65.28

1.36

201309

49.97

48.71

1.26

46.88

1.82

201310

37.64

36.82

0.83

35.73

1.09

201311

64.69

61.29

3.40

60.77

0.52

201312

65.76

61.72

4.05

61.78

-0.06

201401

68.13

66.01

2.12

63.68

2.33

201402

71.03

67.45

3.58

65.25

2.20

201403

71.69

67.61

4.09

69.02

-1.42

201404

74.54

70.83

3.72

69.19

1.64

201405

68.56

64.69

3.87

64.89

-0.20

201406

70.47

66.18

4.29

65.80

0.38

201407

71.37

67.48

3.89

66.33

1.14

201408

71.93

68.33

3.59

66.36

1.97

201409

70.05

67.72

2.33

65.20

2.52

201410

71.86

68.80

3.06

66.14

2.66

201411

62.90

61.05

1.84

57.69

3.36

201412

63.63

59.82

3.81

61.00

-1.18

201501

60.92

58.71

2.21

57.39

1.32

201502

61.00

57.73

3.26

57.36

0.37

201503

65.92

63.19

2.72

60.98

2.21

201504

63.66

61.02

2.64

59.57

1.45

201505

61.20

58.77

2.43

56.44

2.33

201506

64.18

61.25

2.93

60.16

1.10

201507

61.27

57.92

3.35

57.63

0.30

201508

63.54

60.53

3.01

60.00

0.53

201509

62.09

59.66

2.43

57.52

2.13

201510

61.45

59.32

2.13

56.99

2.33

201511

56.32

53.45

2.87

53.07

0.39

201512

58.98

55.70

3.28

55.69

0.01

201601

62.10

59.99

2.12

58.10

1.88

201602

64.90

62.14

2.76

61.67

0.47

201603

58.21

56.36

1.85

55.91

0.45

201604

62.31

60.05

2.26

58.80

1.25

201605

58.92

56.53

2.39

54.94

1.59

201606

62.76

59.53

3.24

59.19

0.34

201607

63.97

60.36

3.61

61.10

-0.74

201608

62.17

60.19

1.98

58.47

1.72

201609

56.92

55.17

1.75

54.18

1.00

201610

55.19

53.21

1.98

50.68

2.53

201611

61.56

59.09

2.48

56.60

2.49

201612

57.21

53.47

3.74

51.18

2.29

201701

60.50

59.57

0.93

56.04

3.53

201702

59.43

59.55

-0.11

57.38

2.16

201703

61.68

61.43

0.25

57.99

3.44

201704

63.50

62.89

0.61

60.61

2.28

201705

63.20

63.35

-0.15

57.82

5.53

201706

63.28

62.91

0.37

59.91

3.00

201707

62.06

62.03

0.04

56.42

5.60

201708

66.10

66.13

-0.04

62.42

3.71

201709

63.12

62.69

0.42

58.42

4.28

201710

61.64

61.71

-0.06

57.71

4.00

201711

60.12

61.19

-1.08

58.02

3.17

201712

55.40

55.92

-0.51

* 53.24

2.67

* - BLS Diary response rates for December 2017 are preliminary



FIELD

DSMD

Field -

BLS

DSMD -

Year

RR

RR

DSMD

RR

BLS

2004

72.55

70.01

2.54

68.94

1.08

2005

74.79

71.28

3.52

70.98

0.29

2006

77.54

73.90

3.64

74.22

-0.32

2007

74.67

70.98

3.69

70.16

0.82

2008

75.99

72.18

3.81

71.94

0.24

2009

76.78

72.45

4.32

73.03

-0.58

2010

76.55

72.13

4.42

71.52

0.61

2011

75.45

70.69

4.76

70.25

0.44

2012

72.23

68.07

4.16

67.79

0.28

2013

64.99

61.75

3.24

60.78

0.97

2014

69.62

66.28

3.33

64.98

1.30

2015

61.70

58.93

2.77

57.71

1.21

2016

60.58

57.94

2.64

56.66

1.28

2017

61.52

61.59

-0.07

* 57.97

3.63

* - BLS Diary response rates for 2017 are preliminary

ATTACHMENT D: E-mails from Census


Shape1

From: Michael Bagley (CENSUS/ADDP FED) [mailto:[email protected]]
Sent: Friday, August 25, 2017 10:55 AM
To: Galemore, Adam J. <[email protected]>; Nix, Brian T. - BLS <[email protected]>
Subject: Re: BLS/Census response rate definitions

Prior to 2017, an outcome of 326 was classified as a Type B (with an action code of 31 in ROSCO).  In 2017, BLS directed us to change the classification of the 326 outcome code to a Type A.  This outcome code occurs pretty infrequently.  Of course, these classifications are internal to Census.  I can't speak to how BLS interprets/translates these codes.

____________________________________

Michael Bagley
Survey Statistician
Consumer Expenditure Surveys
Associate Directorate for Demographic Programs
U.S. Census Bureau


Office 301.763.9470
[email protected]

 

Shape2

From: Adam J Galemore (CENSUS/ADDP FED)
Sent: Friday, August 25, 2017 10:29:24 AM
To: Nix, Brian T. - BLS
Cc: Michael Bagley (CENSUS/ADDP FED)
Subject: Re: BLS/Census response rate definitions

Hey Brian,


Code 326 is a type A; however, (Mike correct me if I'm wrong) I think the code is used to describe the case as a whole since that is how FLD calculates response rates.  DSMD on the other hand calculates the rate based off each diary.


In response to your other email, code 320 is a Type A as well.


Mike Bagley would be able to answer CED specific questions better than me though so you might want to forward them to him in the future.


Shape3

From: Adam J Galemore (CENSUS/ADDP FED) [mailto:[email protected]]
Sent: Wednesday, July 19, 2017 9:20 AM
To: Nix, Brian T. - BLS <[email protected]>
Cc: Patricia M. Holley <[email protected]>
Subject: Re: BLS/Census response rate definitions 


Hey Brian,

 

Yes we don't have a 215 in CEQ, it would have to be coded as a Type A.


For CED, I've attached 2 documents, 1 with the current outcome codes and the other with the codes from a few years ago.  If you have any other questions, you might be better asking Tricia Holley (she's cc'd) since she's more familiar with CED.

 

As far as the response rate calculations, below are the calculations for ROSCO (FLD) and DSMD.

 

ROSCO - FLD calculates CED response rates based on the final outcome code for the case.  The formula used to calculate the rr is:

(201s+203s+206s+207s+209s) / (201s+203s+206s+207s+209s+216s+217s+219s+321s+322s+323s+324s+325s+320s+326s)

 

DSMD - calculates the CED response rate based on the individual diary using the PICK_UP1 and PICK_UP2 variables so the workload ends up being double.  The formula they use is:

201s / (201s+216s+217s+321s+322s+323s+324s+219s+325s)

 

Lastly, the only code I know that has changed types is the 217 Temporarily Absent.  For CEQ this has always been a Type A; however, for CED this used to be considered a Type B, but last year changed to a Type A.

 

Hopefully this email helps in understanding the differences in the way CED rates are calculated.  If you need further clarification please don't hesitate to give me a call at 301-763-1290.  I'm teleworking today but leave me a message and I can call you back.  Also, as I mentioned, if you have further questions with CED rates feel free to reach out to Tricia as well.

 

Thanks


Shape4

From: Adam J Galemore (CENSUS/ADDP FED) [mailto:[email protected]]
Sent: Monday, July 10, 2017 9:02 AM
To: Nix, Brian T. - BLS <[email protected]>
Subject: Re: BLS/Census response rate definitions

 

Hey Brian,

 

I'm still waiting to hear back from FLD for how they calculate the CED response rates, I'll let you know as soon as I do.  However, DSMD and FLD calculate the CEQ rates the same way and it hasn't changed much since 6 years ago.  The only changes should be adding and removing a few Type C codes.

 

Here's a list of the final outcome codes that we use for CEQ:


201 - Completed interview
203 - Sufficient Partial


216 Type A - No one home (unable to contact)
217 Type A - Temporarily absent
219 Type A - Other - specify
321 Type A - Refused, Hostile respondent
322 Type A - Refused, Time related excuses
323 Type A - Refused, Language problems
324 Type A - Refused, Other – specify


224 Type B - All persons under 16
225 Type B - Occupied by persons with URE
226 Type B - Vacant for rent
331 Type B - Vacant for sale
332 Type B - Vacant other – specify


228 Type B - Unfit, to be demolished
229 Type B - Under construction, not ready
231 Type B - Unoccupied tent/trailer site
232 Type B - Permit granted, construction not started
233 Type B - Other – specify


240 Type C - Demolished
241 Type C - House or trailer moved
243 Type C - Converted to permanent nonresidential use
244 Type C - Merged units within same structure
245 Type C - Condemned
247 Type C - Unused serial # or listing sheet
248 Type C - Other - specify
252 Type C - Located on military base or post
258 Type C - Unlocatable sample address
259 Type C - Unit does not exist or unit is out-of-scope
290 Type C - Spawned in error
341 Type C - CU moved
342 Type C - CU merged with another CE CU within same address



1 The two surveys have independent samples of households in the sense that each year a systematic sample of 12,000 addresses is drawn from the Census Bureau’s sampling frame for the Interview survey, and another systematic sample of 12,000 addresses is independently drawn for from the sampling frame for the Diary survey. Prior to 2015, a single sample of 240,000 addresses (= 12,000 addresses per year x 10 years x 2 surveys) was drawn for the entire decade for both surveys, with addresses alternatingly assigned to the Interview survey and the Diary survey. Thus prior to 2015 the two samples were not independent of each other, but starting in 2015 the two samples are independent of each other.

2 DSMD no longer calculates what we, for historical reasons, continue to call the DSMD response rates; however, BLS still receives them monthly from the Demographic Surveys Division (DSD) and Associate Directorate for Demographic Programs (ADDP) at Census. Since 2016, DSMD has begun calculating a weighted response rate for the Quarterly Interview Survey, in response to an internal program review, and began calculating a weighted response rate for the Diary Survey in 2017. However, we do not use these weighted response rates at this time at BLS.

3 Should a housing unit whose residents are away from home for six months be included in the denominator? Should an interview that was only partially completed be included in the numerator? Issues such as these are the reason the three offices have different response rates.

4 It should be noted that for the Diary survey Census and BLS use the term “case” differently. Census considers the entire two-week period to be a single case, whereas BLS considers each week to be a separate case. As a result, it is important to know which audience is being addressed when using the word “case.” To avoid confusion, it is a good habit to routinely refer to the Census concept as “bi-weekly cases” and the BLS concept as “weekly cases.”

5 Refusals are usually clarified further in the system, such as “Refused, Hostile Respondent” or “Refused, Time-Related Excuses.” One specific type of refusal is a Congressional Refusal, which occurs when a Congressional Office contacts the regional office or Census headquarters on behalf of a survey respondent or nonrespondent who wishes to be permanently removed from the sample. If the Congressional office instructs Census to cease all contact efforts, then Census will assign a Type A Refusal outcome code for the current interview, but the household is then removed from the sample address file so that no more contacts will be attempted in the future. This happens rarely enough that it is not given its own outcome code, however – there was only one reported instance of a Congressional Refusal in 2017.

6 DSMD began producing weighted response rates in 2016. We are not currently using these in our comparisons, but instead continue to use the Census-supplied response rates, formerly produced by DSMD, which we still refer to as “DSMD Response Rates” for the sake of continuity.

7 Census sends BLS three outcome codes for each biweekly case – one outcome code for week 1, one outcome code for week 2, and a third outcome code for both weeks combined, which we do not use.

8 Telescoping is the flawed cognitive process in which a respondent incorrectly recalls expenditures made outside the recall period to have been made inside the recall period. In 1964, John Neter and Joseph Waksberg found that telescoping was a significant problem in expenditure surveys, and they developed the concept of the bounding interview to reduce its effect. However, research in the past decade by various people in the CE program (Ian Elkin, Catherine Hackett, Neil Tseng, et. al.) found that it was no longer a significant problem in our survey, paving the way for the removal of the bounding interview from the CE survey in 2015.

9 These include the codes 203 (“Week 1 interview, Week 2 Type A,” 204 (“Week 1 interview, Week 2 Type B/C”), 206 (“Week 1 Type A, Week 2 interview”), 207 (“Week 1 Type B/C, Week 2 interview”), and 210 (“Type A one week, Type B/C the other week”). (Codes 204, 207, and 210 were removed from the system with changes made at Census in 2017.)

File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
AuthorNix, Brian T. - BLS
File Modified0000-00-00
File Created2021-01-13

© 2024 OMB.report | Privacy Policy