Attachment O -- Non-response Bias Analysis

Attachment O -- Non-Response Bias Analysis of the 2014 MEPS-IC.docx

Medical Expenditure Panel Survey - Insurance Component (MEPS-IC)

Attachment O -- Non-response Bias Analysis

OMB: 0935-0110

Document [docx]
Download: docx | pdf

Attachment P -- Non-response Bias Analysis of Private Establishments from the 2014 Medical Expenditures Panel Survey – Insurance Component (MEPS-IC)

Introduction:

When an expected unit response rate is below 80 percent, OMB Standards & Guidelines for Statistical
Surveys
recommends conducting a nonresponse bias analysis. Of the 42,055 sample units selected for the 2014 MEPS-IC, 27,226 (64.7%) responded, 11,776 (28.0%) did not respond, and 3,053 (7.3%) were out of sample or out of business. Removing the out of sample and out of business units from the response rate calculation results in an unweighted response rate of 69.8 percent. As shown in the formula below, nonresponse bias is a function of both the nonresponse rate and the difference between the respondent mean and the nonrespondent mean on the variable of interest:

Shape1

OR

Respondent Mean = Full Sample Mean + (Nonresponse Rate)*(Respondent Mean –Nonrespondent Mean)

In the MEPS-IC we are most concerned about nonresponse bias in our key estimates- the percent of establishments offering health insurance, the percent of employees offered health insurance and the percent of employees enrolled in health insurance, among other important estimates. Unfortunately, since we do not have these estimates for the nonresponding establishments, we cannot directly measure the potential nonresponse bias in these estimates. However, from the sampling frame we have data for both responding and nonresponding establishments that are correlated with, or vary by, many of our key estimates. These variables include the size of the firm the establishment is in (number of employees), the industry group the establishment belongs to and the region of the country where the establishment is located (Census division). This analysis will compare the responding establishments to the nonresponding establishments on these sampling frame variables, using both a chi-square test of independence and a t-test to test differences in means and percentages.

The rest of this memo includes three sections where the differences between responding and nonresponding establishments will be tested and discussed, followed by a discussion of the weighting adjustments for nonresponse bias and a conclusion section.

Firm Size:

Firm size is highly correlated with at least one of our key measures, the percentage of establishments that offer health insurance. In 2014, 25.7 percent of private sector establishments in firms with less than 10 employees offered health insurance and this percentage increased to establishments in firms with 1,000 or more employees where 99.2 percent offered insurance. Table 1 presents the results of a chi-square test of the relationship between firm size and response. The test shows that response to the MEPS-IC is not independent of firm size and this may be a source of nonresponse bias.

To identify which firms size categories are possibly the source of this bias, table 2 shows the percent distribution of responding and nonresponding establishments across the firm size categories and the results of testing the difference in these percentages. The results show those establishments in firms with less than 10 employees, and those with 25 to 99 employees may be a source of nonresponse bias.

Industry Group:

Most of the MEPS-IC key estimates vary by industry group. For example, in 2014 the percent of establishments that offered health insurance to their employees ranged from 23.6 percent for establishments in agriculture, fishing and forestry to 61.8 percent for those in mining and manufacturing. Table 3 presents the results of a chi-square test of the relationship between industry group and response. The test shows that response to the MEPS-IC is not independent of industry category and this may be a source of nonresponse bias.

To identify which industry category is possibly the source of this bias, table 4 shows the percent distribution of responding and nonresponding establishments across the industry categories and the results of testing the difference in these percentages. The results show those establishments in agriculture, fishing and forestry, mining and manufacturing, construction, and professional services may be a source of nonresponse bias.

Census Division:

Many of the MEPS-IC key estimates vary by Census division. For example, in 2013 the percent of employees in establishments that offered health insurance ranged from 82.0 percent for establishments located in West South Central to 88.6 percent for those located in New England. Table 5 presents the results of a chi-square test of the relationship between Census region and response. The test shows that response to the MEPS-IC is not independent of Census division and this may be a source of nonresponse bias.

To identify which Census division is possibly the source of this bias, table 6 shows the percent distribution of responding and nonresponding establishments across divisions and the results of testing the difference in these percentages. The results show those establishments located in 6 of the 9 divisions may be a source of nonresponse bias.

Weighting adjustments for nonresponse bias:

The base sampling weights of the respondents to the MEPS-IC are adjusted so that the respondents also represent the nonrespondents while minimizing the bias associated with nonresponse. The adjustment is made by controlling firm size, establishment size, industry group, type of firm, and state. Thus, a nonresponding establishment is represented by a responding establishment with characteristics similar to the extent possible in terms of these variables. A raking procedure is applied to adjust the weights of the respondents to represent all eligible establishments on the frame (i.e., both respondents and nonrespondents) while controlling for the marginal distributions of all these variables. The raking adjustment is expected to reduce any bias due to nonresponse to the extent the MEPS-IC estimates are associated with the characteristics used in the raking procedure. Since the MEPS-IC estimates are generally highly correlated with these characteristics, the weighting adjustment is expected to minimize the nonresponse bias to a large extent.



Conclusion:

The results of this analysis show that there is the potential for nonresponse bias in the MEPS-IC. Although we never really know the extent of any bias in the survey estimates, since the distributions of responding and nonresponding establishments are close, and since the weighting adjustment takes into account the important variables by which MEPS-IC estimates mostly vary, we can be fairly confident that, to the extent possible, nonresponse bias has been addressed in the MEPS-IC.






Table 1. Chi-Square of Response by Firm Size, 2014 MEPS-IC




Firm Size

Responding (N)

Nonresponding (N)

Total



Frequency

Less than 10

2653849

1263299

3917148



Expected

 

2685550

1231598

 



Percent

 

39.96

19.02

58.98



Row Pct

 

67.75

32.25

 



Col Pct

 

58.29

60.5

 




10 to 24

550138

240744

790882




 

542219

248663

 




 

8.28

3.63

11.91




 

69.56

30.44

 




 

12.08

11.53

 




25 to 99

377294

146313

523607




 

358979

164628

 




 

5.68

2.2

7.88




 

72.06

27.94

 




 

8.29

7.01

 




100 to 999

298031

138730

436762




 

299439

137323

 




 

4.49

2.09

6.58




 

68.24

31.76

 




 

6.55

6.64

 




1,000 or more

673818

298987

972805




 

666944

305861

 




 

10.15

4.5

14.65




 

69.27

30.73

 




 

14.8

14.32

 




Total

4553130

2088073

6641204




 

68.56

31.44

100




Statistic

DF

Value

Prob




Chi-Square

4

4776.6482

<.0001




Cramer's V

 

0.0268

 



Table 2. T-test of Response by Firm Size, 2014 MEPS_IC


Firm Size

Responding (%)

Nonresponding (%)

DF

t Value

Pr > |t|


Less than 10

58.31

60.47

22377

-4

<.0001


10 to 24

12.1

11.48

39000

1.76

0.0776


25 to 99

8.28

7.01

23839

4.4

<.0001


100 to 999

6.54

6.65

27276

-0.4

0.6883


1,000 or more

14.77

14.39

22425

0.98

0.328


Total

100

100









Table 3. Chi-Square of Response by Industry, 2014 MEPS-IC


Industry

Responding (N)

Nonresponding (N)

Total

Frequency

Agriculture,

118102

63347.5

181450

Expected

Fishing, and Forestry

124400

57050

 

Percent

 

1.78

0.95

2.73

Row Pct

 

65.09

34.91

 

Col Pct

 

2.59

3.03

 


Mining and

206529

83937.3

290466


Manufacturing

199140

91326

 


 

3.11

1.26

4.37


 

71.1

28.9

 


 

4.54

4.02

 


Construction

362858

201415

564273


 

386859

177414

 


 

5.46

3.03

8.5


 

64.31

35.69

 


 

7.97

9.65

 


Utilities and

137421

61984

199405


Transportation

136710

62695

 


 

2.07

0.93

3


 

68.92

31.08

 


 

3.02

2.97

 


Wholesale Trade

241713

114359

356072


 

244119

111953

 


 

3.64

1.72

5.36


 

67.88

32.12

 


 

5.31

5.48

 


Financial Services

528676

237594

766270


and Real Estate

525346

240924

 


 

7.96

3.58

11.54


 

68.99

31.01

 


 

11.61

11.38

 


Retail Trade

643184

286023

929207


 

637053

292154

 


 

9.68

4.31

13.99


 

69.22

30.78

 


 

14.13

13.7

 


Professional Services

1178993

507821

1686814


 

1156460

530354

 


 

17.75

7.65

25.4


 

69.89

30.11

 


 

25.89

24.32

 


Other

1135655

531592

1667246


 

1143044

524202

 


 

17.1

8

25.1


 

68.12

31.88

 


 

24.94

25.46

 


Total

4553130

2088073

6641204


 

68.56

31.44

100


Statistic

DF

Value

Prob


Chi-Square

8

8512.2193

<.0001


Cramer's V

 

0.0358

 



Table 4. T-test of Response by Industry, 2014 MEPS_IC

Industry

Responding (%)

Nonresponding (%)

DF

t Value

Pr > |t|

Agriculture, Fishing, and Forestry

2.59

3.04

20723

-2.44

0.0146

Mining and Manufacturing

4.54

4.01

23455

2.4

0.0165

Construction

7.99

9.61

20628

-5.11

<.0001

Utilities and Transportation

3.01

2.98

22297

0.15

0.8811

Wholesale Trade

5.31

5.48

21899

-0.7

0.4823

Financial Real Estate

11.59

11.42

22334

0.48

0.6341

Retail Trade

14.11

13.74

22434

0.98

0.3284

Professional Services

25.92

24.25

39000

3.51

0.0005

Other

24.94

25.47

22070

-1.07

0.2863

Total

100

100















Table 5. Chi-Square of Response by Division, 2014 MEPS-IC


Division

Responding (Y)

Nonresponding (N)

Total

Frequency

New

240544

102128

342672

Expected

 England

234932

107740

 

Percent

 

3.62

1.54

5.16

Row Pct

 

70.2

29.8

 

Col Pct

 

5.28

4.89

 


Middle

587908

335436

923344


 Atlantic

633034

290310

 


 

8.85

5.05

13.9


 

63.67

36.33

 


 

12.91

16.06

 


East

670924

289825

960749


 North

658678

302071

 


 Central

10.1

4.36

14.47


 

69.83

30.17

 


 

14.74

13.88

 


West

388453

137034

525487


 North

360268

165219

 


 Central

5.85

2.06

7.91


 

73.92

26.08

 


 

8.53

6.56

 


South

861378

416445

1277823


 Atlantic

876060

401763

 


 

12.97

6.27

19.24


 

67.41

32.59

 


 

18.92

19.94

 


East

242474

92219.5

334693


South 

229462

105231

 


Central 

3.65

1.39

5.04


 

72.45

27.55

 


 

5.33

4.42

 


West

480372

227581

707952


 South

485364

222589

 


 Central

7.23

3.43

10.66


 

67.85

32.15

 


 

10.55

10.9

 


Mountain

355633

140220

495853


 

339951

155902

 


 

5.35

2.11

7.47


 

71.72

28.28

 


 

7.81

6.72

 


Pacific

725446

347185

1072630


 

735383

337248

 


 

10.92

5.23

16.15


 

67.63

32.37

 


 

15.93

16.63

 


Total

4553130

2088073

6641204


 

68.56

31.44

100


Statistic

DF

Value

Prob


Chi-Square

8

24415.1746

<.0001


Cramer's V

 

0.0606

 



Table 6. T-test of Response by Geographic Division, 2014 MEPS_IC

Division

Responding (%)

Nonresponding (%)

DF

t Value

Pr > |t|

New England

5.3

4.85

39000

1.84

0.066

Middle Atlantic

12.92

16.07

20505

-7.97

<.0001

East North Central

14.74

13.87

39000

2.27

0.0231

West North Central

8.52

6.59

24796

6.77

<.0001

South Atlantic

18.92

19.93

21814

-2.3

0.0216

East South Central

5.33

4.41

24142

3.94

<.0001

West South Central

10.54

10.92

21901

-1.09

0.2777

Mountain

7.8

6.74

23623

3.72

0.0002

Pacific

15.93

16.62

21868

-1.68

0.0939

Total

100

100






File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
AuthorWindows User
File Modified0000-00-00
File Created2021-01-22

© 2024 OMB.report | Privacy Policy