Non Response Bias Study Results

United States
Department of
Agriculture

National
Agricultural
Statistics
Service
Research and
Development Division
Washington DC 20250
RDD Research Report
RDD-08-06

Assessing the Effect of
Calibration on
Nonresponse Bias in the
2006 ARMS Phase III
Sample Using Census
2002 Data
Morgan S. Earp
Jaki S. McCarthy
Nick D. Schauer
Phillip S. Kott

June 2008

This paper was prepared for limited distribution to the research community outside the United
States Department of Agriculture. The views expressed herein are not necessarily those of the
National Agricultural Statistics Service or of the United States Department of Agriculture.

EXECUTIVE SUMMARY
Phase III of the Agricultural Resource Management Survey (ARMS) is one of the most
complex and detailed sample survey data collections conducted by the United States
Department of Agriculture’s (USDA) National Agricultural Statistics Service (NASS).
For this survey, NASS collects calendar year economic data from agricultural producers
nationwide.
In September 2006 the Executive Office of the President released the Office of
Management and Budget Standards and Guidelines for Statistical Surveys. The Office of
Management and Budget’s (OMB) new standards and guidelines for statistical surveys
addressed a number of federal statistical agency issues. Standard 3.2 specifically
addressed the issue of low response rates and analysis of unit nonresponse; Guideline
3.2.9 required that surveys failing to meet an 80 percent response rate be subject to
nonresponse bias analysis.
The 2005 Agricultural Resource Management Survey (ARMS) Phase III Survey
Administration Analysis (Hopper, 2006) reported a response rate of 70.5 percent and the
2006 Agricultural Resource Management Survey (ARMS) Phase III Survey
Administration Analysis (Hopper, 2007) reported an even lower response rate of 67.6
percent, both requiring NASS to conduct analyses of nonresponse bias. This latter
analysis uses the same methodology used in the former study which is described in the
NASS Research and Development Division (RDD) report, Assessing the Effect of
Calibration on Nonresponse Bias in the 2005 ARMS Phase III Sample Using Census
2002 Data (Earp, McCarthy, Schauer, & Kott, 2008).
Records sampled for the 2006 ARMS Phase III were matched against records from the
long-form sample of the 2002 Census of Agriculture. Nonresponse bias in the ARMS
respondent sample was assessed using census data. Three weighted means of census data
were computed and compared across 20 regions: 1) the mean for all the matching records
computed using base sampling weights, 2) the mean for the matching records responding
to the ARMS using the same base sampling weights, and 3) the mean for the latter group
using the sampling weights adjusted by calibration.
Relative bias of the mean was assessed for 17 “study variables” using a variation of the
formula provided by OMB in Guideline 3.2.9. Although significant biases were
exhibited in 11 of 17 variables using the 2006 ARMS Phase III base sampling weights,
the 2006 ARMS Phase III calibration weights were able to reduce the bias to statistical
insignificance (i.e. p > .05) in over 90 percent (10/11) of the study variables. For this
analysis, the calibration process varied slightly from that of the 2006 ARMS Phase III in
that egg and milk production were not measured by the 2002 Census and could not be
included in the calibration. The inability to replicate the 2006 ARMS Phase III
calibration process fully may in part account for the one variable, total sales,
demonstrating a significant level of bias even using the calibrated weights. This study
suggests, however, that the process of calibration is an effective tool in reducing
nonresponse bias levels.

i

RECOMMENDATIONS
1.

Nonresponse bias of all study variables, especially livestock purchases and
fertilizer expenses, should be reevaluated when the 2007 Census data are
available. This Census will contain equivalent calibration target variables for
egg and milk production, as well as expenditure data for all Census respondents,
allowing for an assessment that will be consistent with the calibration targets
used for the 2006 ARMS Phase III.

2.

Methods should be developed to assess biases not measured in this analysis,
especially those that may exist in only a single region.1

1

ARMS PHASE III estimation regions include the 15 leading cash receipts states (Arkansas, California,
Florida, Georgia, Illinois, Indiana, Iowa, Kansas, Minnesota, Missouri, Nebraska, North Carolina, Texas,
Washington, and Wisconsin). The remaining 33 states (Alaska and Hawaii were not sampled for ARMS)
were assigned to one of the five main production regions (Atlantic, South, Midwest, Plains, and West).

ii

Assessing the Effect of Calibration on Nonresponse Bias in the 2006 ARMS
Phase III Sample Using Census 2002 Data
Morgan Earp, Jaki McCarthy, Nick Schauer, & Phil Kott2
Abstract
The United States Department of Agriculture’s National Agricultural Statistics Service
(NASS) conducts the annual Agricultural Resource Management Survey (ARMS) in
three phases. The third phase of the ARMS collects detailed economic data which is
highly sensitive. As a consequence, this phase suffers from relatively low response rates
for a federal survey. According to the 2006 Office of Management and Budget Standards
and Guidelines for Statistical Surveys, response rates lower than 80 percent may not only
result in nonresponse bias, but they can jeopardize the future of surveys carried out by
federal agencies. NASS has been operating under the assumption that the use of
calibrated weights derived from appropriate targets addresses nonresponse bias in the
2006 Phase III ARMS. This assumption was tested using Census 2002 expendituresample data.
The results showed that calibrated weights decreased bias levels so that they were no
longer significantly different from zero at the 0.05 level for over 90 percent of the
variables evaluated.
Key Words: Nonresponse; response rate; bias; calibration weights.

2

Jaki McCarthy provided assistance with this research while the Chief Research Methodologist with the
USDA’s National Agricultural Statistics Service (NASS) – Research and Development Division (RDD).
Phil Kott provided assistance with this research while the Chief Research Statistician with the USDA’s
NASS – RDD. Morgan Earp is a survey and mathematical statistician in the RDD, located in Room 305,
3251 Old Lee Highway, Fairfax, VA 22030. Nick Schauer is a mathematical statistician in the Agency’s
Statistics Division.

1. INTRODUCTION
On September 22, 2006, the Executive Office of the President released the Office of
Management and Budget Standards and Guidelines for Statistical Surveys based on the
recommendations of the Federal Committee on Statistical Methodology’s (FCSM)
Subcommittee on Standards for Statistical Surveys. The Office of Management and
Budget’s (OMB) new standards and guidelines for statistical surveys pertain to aspects of
surveys conducted by federal statistical agencies.
Federal statistical agencies, such as the United States Department of Agriculture’s
(USDA) National Agricultural Statistics Service (NASS), are directly affected by OMB’s
new standards and guidelines for statistical surveys. One of the standards (3.2) issued by
OMB addresses response rates and analysis of nonresponse bias. According to Standard
3.2,
Agencies must appropriately measure, adjust for, report, and analyze unit and
item nonresponse to assess their effects on data quality and to inform users.
Response rates must be computed using standard formulas to measure the
proportion of the eligible sample that is represented by the responding units in
each study, as an indicator of potential nonresponse bias. (Office of Management
and Budget, 2006, p. 14)
In 2005, the Agricultural Resource Management Survey (ARMS) Phase III response rate
was 70.5 percent (n = 34,937), which fell below the OMB response rate threshold of 80
percent listed in Guideline 3.2.9; therefore, NASS was required by OMB to research the
effect of nonresponse bias (Earp, McCarthy, Schauer, & Kott, 2008). In 2006, the
Agricultural Resource Management Survey (ARMS) Phase III response rate was 67.6
percent (n = 34,192), which again fell below the 80 percent threshold, and thus NASS
was again required by OMB to research the effect of nonresponse bias. Since the Phase
II response rate of 81.3 percent exceeded OMB’s 80 percent threshold, nonresponse bias
assessment was only required for Phase III, the “problem” stage. Specifically, Guideline
3.2.9 states
Given a survey with an overall unit response rate of less than 80 percent, conduct
an analysis of nonresponse bias using unit response rates as defined above, with
an assessment of whether the data are missing completely at random. As noted
above, the degree of nonresponse bias is a function of not only the response rate
but also how much the respondents and nonrespondents differ on the survey
variables of interest. For a sample mean, an estimate of the bias of the sample
respondent mean is given by:

⎛n ⎞
B( yr ) = yr − yt = ⎜ nr ⎟( yr − ynr )
⎝ n ⎠
where,
y t = the mean based on all sample cases;

2

yr =
y nr =
n=
nnr =

the mean based only on respondent cases;
the mean based only on nonrespondent cases;
the number of cases in the sample; and
the number of nonrespondent cases.

For a multistage (or wave) survey, focus the nonresponse bias analysis on each
stage, with particular attention to the “problem” stages. A variety of methods can
be used to examine nonresponse bias, for example, make comparisons between
respondents and nonrespondents across subgroups using available sample frame
variables. In the analysis of unit nonresponse, consider a multivariate modeling
of response using respondent and nonrespondent frame variables to determine if
nonrespondent bias exists. (Office of Management & Budget, 2006, p. 16)
Currently, NASS calculates the unweighted unit response rates (RRU) for the ARMS
based on the formula provided under Guideline 3.2.2 of the Office of Management and
Budget Standards and Guidelines for Statistical Surveys:

RRU =

C
C + R + NC + O + e(U )

where,
C = the number of completed cases or sufficient partials;
R = the number of refused cases;
NC = the number of noncontacted sample units known to be eligible;
O = the number of eligible sample units not responding for reason other than
refusal;
U = the number of sample units of unknown eligibility, not completed; and
e = the estimated proportion of sample units of unknown eligibility that are
eligible. (p. 14)
Thus, NASS sums the number of positive usables, out-of-business, and non-farms and
calculates the percentage this sum represents of the total number of reports to calculate
the response rate for ARMS Phase III.
The ARMS is conducted in three phases. Phase I screens for potential samples for Phases
II and III. Phase II collects data on cropping practices and agricultural chemical usage,
while Phase III collects detailed economic information about the agricultural operation,
as well as the operator’s household. Phase III is the only phase of the 2006 ARMS with
response rates lower than 80 percent.
Due to lower response rates with 2006 ARMS Phase III, the potential for nonresponse
bias is greater there. NASS weights the ARMS Phase III respondent sample in such a
way that estimated variable totals for a large set of items match “targets” determined
from other sources. This is done through a weighting process called “calibration.”
Calibration is the process of adjusting survey weights so that certain targets are met.

3

NASS uses official estimates of farm numbers, corn, soybean, wheat, cotton, fruit and
vegetable acres as well as cattle, milk production, hogs, broilers, eggs and turkeys as
calibration targets. For example, after calibration, the calibration-weighted sum of the
survey data will equal the NASS estimate for corn acres. In addition to reducing
confusion in the user community that might result from NASS releasing alternative
estimates for the same totals, calibration weighting produces 2006 ARMS Phase III
estimates with generally lower variances and reduces nonresponse biases. This report
describes an ongoing research effort aimed at measuring the potential for nonresponse
bias in the ARMS Phase III and the success or failure of calibration in removing it.
Nonresponse bias is very difficult to measure directly. Fortunately, an indirect measure
of nonresponse bias is available for the 2006 ARMS Phase III, hereafter called simply the
“ARMS.”
The Census of Agriculture is a mandatory collection of data from all known agricultural
operations. NASS has data from the Census on items of interest for many of the ARMS
nonrespondents; however, the Census itself is incomplete. An estimated 17.90 percent of
all farms were missing from the 2002 Census Mailing List, and 12.26 percent of farms on
the List failed to respond to the Census. Moreover, not all ARMS sampled farms could
be matched to 2002 Census records. Nevertheless, by comparing the 2002 Census
values of ARMS respondents to the full sample of ARMS respondents as a whole, we can
measure the difference between the average ARMS respondent and the average of the full
sample without any nonresponse adjustment. Additionally, this analysis intends to
measure the reduction of that difference from using a calibration-weighting process
similar to the one used for the 2006 ARMS.
Although the 2002 Census data do not perfectly match the 2006 ARMS Phase III data,
they are highly correlated (see Appendix A). The present evaluation will effectively
compare 2006 ARMS Phase III survey respondents to nonrespondents using their 2002
Census data. The 2002 Census expenditure data, which were required for all Census
reports considered usable in this research, were available for 43 percent of 2006 ARMS
Phase III reports.3
2. METHODS
Our analytical data set consists of census values for farms sampled for the ARMS that
also provided 2002 expenditure-sample data on the Census. In the 2002 Census only a
sample of farms received the long version of the questionnaire which asked the
expenditure questions.
The base sampling weight for a farm in our analytical data set was its ARMS sample
weight before calibration multiplied by its Census sample weight. Each ARMS
responding farm was calibrated to produce weighted totals for the calibration variables
that were equal to the base-sampling-weighted totals computed from both respondents
3

The match rate for 2006 ARMS Phase III reports with 2002 Census expenditure data was significantly
higher for nonrespondents (47.5%) than for respondents (40.5%) (z = 12.24, p < .05).

4

and nonrespondents. The calibration variables used were inventory/acreage numbers for
cattle, corn, cotton, pigs, soybeans, wheat, fruit, vegetables, broilers, and turkeys. Each
of these target variables, plus egg and milk production, was used operationally in
calibrating the ARMS data.
As in the operational program, the ARMS respondent subset was calibrated
independently in 20 regions. These included the 15 leading cash receipts states
(Arkansas, California, Florida, Georgia, Illinois, Indiana, Iowa, Kansas, Minnesota,
Missouri, Nebraska, North Carolina, Texas, Washington, and Wisconsin). The remaining
33 states (Alaska and Hawaii are not sampled for the ARMS) were grouped using the five
production regions: 1) Atlantic, 2) South, 3) Midwest, 4) Plains, and 5) West.
Our analysis focuses on 17 specific (non-calibration) variables collected on both the
ARMS and the Census:
1. Total Acres
2. Total Sales
3. Acres Rented
4. Cropland Acres
5. Total Production Expenses
6. Crop Expenses
7. Seed Expenses
8. Fertilizer Expenses
9. Chemical Expenses
10. Livestock Purchases
11. Feed Purchases
12. Hired Labor Expenses
13. Machinery and Equipment Value
14. Government Payments
15. Operator’s Age
16. Operator’s Race
17. Farm Type.
These variables were also included in a similar analysis for ARMS PHASE III 2005
(Earp et al., 2008).
Letting yr denote the base-sample or calibrated-sample mean among the ARMS
respondent subset for a study variable, and yt denote the corresponding base-sample
mean among the entire matched sample, it is a simple matter to compute the relative bias
of the former with respect to the latter, relBias =

yr − yt
yr

. The statistical significance of

this value is much harder to assess since the samples on which yr and yt are based are
complex and overlapping.
Fortunately, we can easily test the persistence or absence of a systematic bias across the
20 regions. To this end, we compute the following measure of bias of an ARMS5

respondent mean (before or after calibration) with respect to the Census mean in every
region:

M = log( y r ) − log( y t )

⎛y
⎞
= log ⎜ r y ⎟
t⎠
⎝
⎛
y − yt ⎞
⎟
= log⎜⎜1 + r
y t ⎟⎠
⎝

≈

y r − yt
yt

This measure is conveniently symmetric, log( yt ) − log( yr ) = −[log( yr ) − log( yt )] , while
retaining the scale-invariance property of the relative bias (i.e., multiplying the reported
item value on each farm by a fixed factor does not affect the overall relative bias).
The bias measure M for a study variable in a region can be treated as an independent
random variable. The null hypothesis of no bias (again, either before or after calibration)
can be tested against an alternative hypothesis of a persistent bias (p%) across all the
regions. The conventional t test based on the 20 observations (one per region) is
asymptotically normal under both the null and alternative hypotheses. We follow the
standard practice of approximating the distribution of this test statistic with a Student’s t
having 19 degrees of freedom. This may lead to liberal inferences (the inappropriate
rejection of the null hypothesis when it is true) because the M-values for the study
variable may not be normally distributed with a common variance across regions.
Nevertheless, by taking logs we create a test statistic that is more nearly normal and
homoscedastic than absolute biases would be.
A sign and a signed-rank test of the 20 paired observations for a study variable before and
after calibration was conducted. The sign test is not as powerful as the other two tests
(i.e., it more often fails to find that M is significantly different from 0 when, in fact, there
is a persistent bias across the regions), but it assumes neither that M is normal nor
homoscedastic. The signed-rank test assumes the latter, but not the former. We include
it in our results for completeness.
3. RESULTS AND DISCUSSION
Our results are summarized in Table 1. Chemical expenses, machinery and equipment
value, government payments, acres rented, farm type, fuel and oil expenses, operator’s
age, and cropland acres (Variables 1-6) do not exhibit significant biases using either
calibrated or uncalibrated weights. Results slightly varied from those in the previous
analysis of ARMS PHASE III 2005 data (Earp et al., 2008); total acres operated no
longer exhibits significant bias using either calibrated or uncalibrated weights; on the
contrary, chemical expenses, machinery and equipment value, and fuel and oil expenses
now exhibit significant bias using the uncalibrated weights, but not the calibrated weights
(Earp et al., 2008).

6

7

Table 1: Mean Comparisons and Indicated Biases for Matching Records Using Base Sampling Weights versus Calibrated Weights

8

Table 1 (Cont.): Mean Comparisons and Indicated Biases for Matching Records Using Base Sampling Weights versus Calibrated Weights

In over 90 percent (10/11) of the study variables exhibiting persistent biases using the
base sample weights (i.e., variables 7-17), calibration weighting is able to reduce the bias
so that it was no longer significantly different from zero using a t-test with p < .05. The
rate of bias elimination remained consistent from 2005 to 2006, although the rate of bias
and the rank of variable bias varied (Earp et al., 2008). All of these variables show a
significant reduction in bias levels using a paired t-test. After calibration, only one study
variable, total sales, has a significant bias. This result varied from 2005, where fertilizer
expense was the only variable exhibiting significant bias. Using the 2006 data, fertilizer
expense no longer exhibits significant bias using the calibrated weights, but total sales
which did not exhibit significant bias using the calibrated weights does (Earp et al., 2008.
As in 2005, the estimated bias of livestock purchases remains the largest among the study
variables. Using only the base-sampling weights, this bias was highly significant using
all three test statistics. After calibration, although still large in magnitude, the estimated
bias was reduced to statistical insignificance using all the tests. For this variable,
calibration continues to reduce the bias significantly, if not completely.
4. CONCLUSION
ARMS data are used by farm organizations, commodity groups, agribusiness, Congress,
State Departments of Agriculture, and the USDA. The USDA uses ARMS data to
evaluate the financial performance of farms and ranches, which influence agricultural
policy decisions. The Department also uses Phase III data for objective evaluation of
critical issues related to agriculture and the rural economy; therefore, it is essential that
measures be taken to minimize the effect of nonresponse bias in ARMS, specifically
Phase III.
In assessing the adjustment for nonresponse bias in the 2006 ARMS Phase III, the 2002
Census mean estimates of total production expenses, livestock purchases, hired labor
expenses, feed purchases, fuel and oil expenses, chemical expenses, machinery and
equipment value, seed expenses, cropland expenses, and fertilizer expenses demonstrated
significant bias using just the base sample weights. Although the magnitude of the
relative bias of the mean estimate remained high for livestock purchases using the
calibrated weights, calibration reduced the magnitude of this bias to statistical
insignificance (see Table 1). For this analysis, the calibration process varied slightly
from that of the 2006 ARMS Phase III. Egg and milk production were not included as
calibration targets, because these data items were not collected for the 2002 Census. This
may help to explain why the magnitude of the estimated relative bias of the mean for
livestock in Table 1 remained high even after the data were calibrated. Although it was
not possible to use these as calibration targets in this analysis, their use in the ARMS
PHASE III survey may reduce the bias for livestock purchases in published ARMS data.
According to Guideline 3.2.13 of the Office of Management and Budget Standards and
Guidelines for Statistical Surveys, NASS should:

9

Base decisions regarding whether or not to adjust or impute data for item
nonresponse on how the data will be used, the assessment of nonresponse bias
that is likely to be encountered in the review of collections, prior experience with
this collection, and the nonresponse analysis discussed in this section. When
used, imputation and adjustment procedures should be internally consistent,
sampled on theoretical and empirical considerations, appropriate for the analysis,
and make use of the most relevant data available. If multivariate analysis is
anticipated, care should be taken to use imputations that minimize the attenuation
of underlying relationships.
Due to the broadness of the ARMS Phase III data user community and the survey’s
impact on agricultural policy, it is crucial that the calibration process effectively adjusts
for nonresponse bias. Assuming that the adjustment process is even more effective than
demonstrated here (particularly for livestock purchases and total sales) when all
calibration targets (including egg and milk production) are available and used, it appears
that NASS is appropriately addressing the issue of nonresponse bias in ARMS Phase III
through the calibration process.
Limitations of this analysis include: 1) Inability to replicate the 2006 ARMS Phase III
calibration process exactly without egg and milk items; 2) Inability to assess farms not
covered or responding to the Census of Agriculture; and 3) Inability to recognize
localized biases in the ARMS data (tests were limited to persistent biases across regions).
Knowing that the analyzed data come from the 2002 Census and not from the 2005
ARMS Phase III survey does not limit, but strengthens the analysis. It allows us to focus
entirely on the impact of the nonresponse per se.
5. Recommendations
Based on the results of the present study, the following recommendations are offered:

10

1.

Nonresponse bias of all study variables, especially livestock purchases and
fertilizer expenses should be reevaluated when the 2007 Census data are
available. This Census will contain equivalent calibration target variables for
egg and milk production, as well as expenditure data for all Census respondents,
allowing for an assessment that will be consistent with the calibration targets
used for the 2006 ARMS Phase III.

2.

Methods should be developed to assess biases not measured in this analysis,
especially those that may exist in only a single region.

6. References
Earp, M.S., McCarthy, J.S., Shauer, N.D., & Kott, P.S. (2008), Assessing the Effect of
Calibration on Nonresponse Bias in the 2005 ARMS Phase III Sample Using Census
2002 Data. Research and Development Division Staff Report RDD-08-01, United
States Department of Agriculture, National Agricultural Statistics Service.
Kott, P.S. (2005), “Using Calibration Weighting to Adjust for Nonresponse and Coverage
Errors” Survey Methodology 32: 133-142.
Hopper, R. (2007). 2006 Agricultural Resource Management Survey (ARMS) Phase III
Survey Administration Analysis. Census and Survey Division Staff Report SAB-0714, United States Department of Agriculture, National Agricultural Statistics Service.
United States. Department of Agriculture. (2002). 2002 Census of Agriculture, Vol. 1,
Appendix C. Washington, DC: U.S. Department of Agriculture.
United States. Department of Agriculture. 2007. 2006 Agricultural Resource
Management Survey (ARMS) Phase III Survey Administration Analysis. Washington,
DC: U.S. Department of Agriculture.
United States. Department of Education. 2003. National Center for Education Statistics
Statistical Standards. Washington, DC: U.S. Department of Education.
United States. Executive Office of the President. 2006. Office of Management and Budget
Standards and Guidelines for Statistical Surveys. Washington, DC: U.S. Executive
Office of the President.

11

APPENDIX A: Census 2002 and ARMS Phase III 2006 Correlations & Scatter
Plots
Table A1: Census 2002 and ARMS Phase III 2006 Variable Correlations with Outliers
r2

.89399
( n = 9,380 )

.79922

Scatter Plots
Census

Total Acres Operated

r

Acres Rented

.63828
( n = 9,380 )

.40740

Census

ARMS PHASE III

Cropland Acres

.64025
( n = 9,380 )

.40992

Census

ARMS PHASE III

Total Production Expenses

.83655
( n = 9,380 )

.69982

Census

ARMS PHASE III

Seed Expenses

.57250
( n = 9,380 )

.32776

Census

ARMS PHASE III

Fertilizer Expenses

.71797
( n = 9,380 )

.51548

Census

ARMS PHASE III

Chemical Expenses

.79302
( n = 9,380 )

.62888

Census

ARMS PHASE III

Crop Expenses

.75064
( n = 9,380 )

.56346

Census

ARMS PHASE III

Livestock Purchases

.69380
( n = 9,380 )

.48136

Census

ARMS PHASE III

Feed Purchases

.83451
( n = 9,380 )

.69641

Census

ARMS PHASE III

Hired Labor Expenses

.74749
( n = 9,380 )

.55874

Census

ARMS PHASE III

Fuel & Oil Expenses

.56588
( n = 9,380 )

.32022

Census

ARMS PHASE III

ARMS PHASE III

12

.49977
( n = 9,380 )

.24977

Census

Machinery & Equipment

Government Payments

.47336
( n = 9,380 )

.22407

Census

ARMS PHASE III

Operator’s Age

.63618
( n = 9,380 )

.40472

Census

ARMS PHASE III

Farm Type

.84887
( n = 9,380 )

.72058

Census

ARMS PHASE III

ARMS PHASE III

1. All correlations were significant at the .05 level.
2. Correlations were only estimated for ARMS respondents.
3. Outliers were flagged using DFFITS, Cook’s D, and studentized residuals and are shown in red.

13

Table A2: Census 2002 and ARMS PHASE III 2006 Variable Correlations without
Outliers
r2

.95629
( n =9,278)

.91449

Scatter Plots
Census

Total Acres Operated

r

Acres Rented

.87279
( n = 9,295 )

.76176

Census

ARMS PHASE III

Cropland Acres

.86821
( n = 8,996 )

.88723

Census

ARMS PHASE III

Total Production Expenses

.85281
( n = 9,177 )

.72728

Census

ARMS PHASE III

Seed Expenses

.68220
( n = 9,167)

.46540

Census

ARMS PHASE III

Fertilizer Expenses

.75103
( n =9,129)

.56405

Census

ARMS PHASE III

Chemical Expenses

.81290
( n = 9,077 )

.81290

Census

ARMS PHASE III

Crop Expenses

.81961
( n = 9,114 )

.67176

Census

ARMS PHASE III

Livestock Purchases

.46853
( n = 9,289 )

.21952

Census

ARMS PHASE III

Feed Purchases

.63679
( n =9,159)

.40550

Census

ARMS PHASE III

Hired Labor Expenses

.83480
( n = 9,135)

.69689

Census

ARMS PHASE III

Fuel & Oil Expenses

.68545
( n =9,126)

.46984

Census

ARMS PHASE III

Machinery & Equipment

14

.61670
( n = 8,976 )

.38032

Census

ARMS PHASE III

ARMS PHASE III

.59126
( n =8,972)

.34959

Census

Government Payments

Operator’s Age

.89151
( n = 8,725)

.79479

Census

ARMS PHASE III

Farm Type

.95712
( n =8,842)

.91608

Census

ARMS PHASE III

ARMS PHASE III

1. All correlations were significant at the .05 level (n = 19,483).
2. Correlations were only estimated for ARMS respondents.

15

Figures A1-A2: Census 2002 versus ARMS PHASE III 2006 Total Acres Operated

Census 2002 – Total Acres Operated (Acres)

Scatter Plot of Total Acres Operated with Outliers
400000

300000

200000

100000

0
0

100000

200000

300000

400000

ARMS PHASE III 2006 – Total Acres Operated (Acres)

Census 2002 –Total Acres Operated (Acres)

Scatter Plot of Total Acres Operated without Outliers
70000
60000
50000
40000
30000
20000
10000
0
0

10000 20000 30000 40000 50000 60000 70000

ARMS PHASE III 2006 – Total Acres Operated (Acres)

16

Figures A3-A4: Census 2002 versus ARMS PHASE III 2006 Acres Rented
Scatter Plot of Acres Rented with Outliers

Census 2002 – Acres Rented (Acres)

400000

300000

200000

100000

0
0

100000

200000

ARMS PHASE III 2006 – Acres Rented (Acres)

Census 2002 – Acres Rented (Acres)

Scatter Plot of Acres Rented without Outliers
30000

20000

10000

0
0

10000

20000

30000

ARMS PHASE III 2006 – Acres Rented (Acres)

17

Figures A5-A6: Census 2002 versus ARMS PHASE III 2006 Cropland Acres
Scatter Plot of Cropland Acres with Outliers

Census 2002 – Cropland Acres (Acres)

30000

20000

10000

0
0

20000

40000

60000

80000 100000 120000

ARMS PHASE III 2006 – Cropland Acres (Acres)

Census 2002 – Cropland Acres (Acres)

Scatter Plot of Cropland Acres without Outliers
6000

5000

4000

3000

2000

1000

0
0

2000

4000

6000

8000

10000

12000

ARMS PHASE III 2006 – Cropland Acres (Acres)

18

Figures A7-A8: Census 2002 versus ARMS PHASE III 2006 Total Production
Expenses
Census 2002 – Total Production Expenses (Dollars)

Scatter Plot of Total Production Expenses with Outliers
1. 10E+08
1. 00E+08
9. 00E+07
8. 00E+07
7. 00E+07
6. 00E+07
5. 00E+07
4. 00E+07
3. 00E+07
2. 00E+07
1. 00E+07
0. 00E+00
0. 00E+00 5. 00E+07 1. 00E+08 1. 50E+08 2. 00E+08

Census 2002 – Total Production Expenses (Dollars)

ARMS PHASE III 2006 – Total Production Expenses (Dollars)

Scatter Plot of Total Production Expenses without Outliers
30000000

20000000

10000000

0
0

10000000 20000000 30000000 40000000

ARMS PHASE III 2006 – Total Production Expenses (Dollars)

19

Figures A9-A10: Census 2002 versus ARMS PHASE III 2006 Seed Expenses

Census 2002 – Seed Expenses (Dollars)

Scatter Plot of Seed Expenses with Outliers
4000000

3000000

2000000

1000000

0
0

2000000

4000000

6000000

8000000

10000000

ARMS PHASE III 2006 – Seed Expenses (Dollars)

Census 2002 – Seed Expenses (Dollars)

Scatter Plot of Seed Expenses without Outliers
600000

500000

400000

300000

200000

100000

0
0

1000000

2000000

ARMS PHASE III 2006 – Seed Expenses (Dollars)

20

Figures A11-A12: Census 2002 versus ARMS PHASE III 2006 Fertilizer Expenses

Census 2002 – Fertilizer Expenses (Dollars)

Scatter Plot of Fertilizer Expenses with Outliers
3000000

2000000

1000000

0
0

2000000

4000000

ARMS PHASE III 2006 – Fertilizer Expenses (Dollars)

Census 2002 – Fertilizer Expenses (Dollars)

Scatter Plot of Fertilizer Expenses without Outliers
600000

500000

400000

300000

200000

100000

0
0

500000

1000000

1500000

ARMS PHASE III 2006 – Fertilizer Expenses (Dollars)

21

Figures A13-A14: Census 2002 versus ARMS PHASE III 2006 Chemical Expenses

Census 2002 – Chemical Expenses (Dollars)

Scatter Plot of Chemical Expenses with Outliers
5000000

4000000

3000000

2000000

1000000

0
0

2000000

4000000

6000000

ARMS PHASE III 2006 – Chemical Expenses (Dollars)

Census 2002 – Chemical Expenses (Dollars)

Scatter Plot of Chemical Expenses without Outliers
700000
600000
500000
400000
300000
200000
100000
0
0

200000

400000

600000

800000

1000000

ARMS PHASE III 2006 – Chemical Expenses (Dollars)

22

Figures A15-A16: Census 2002 versus ARMS PHASE III 2006 Crop Expenses

Census 2002 – Crop Expenses (Dollars)

Scatter Plot of Crop Expenses with Outliers
6000000

5000000

4000000

3000000

2000000

1000000

0
0

5000000

10000000

15000000

20000000

ARMS PHASE III 2006 – Crop Expenses (Dollars)

Census 2002 – Crop Expenses (Dollars)

Scatter Plot of Crop Expenses without Outliers
1100000
1000000
900000
800000
700000
600000
500000
400000
300000
200000
100000
0
0

1000000

2000000

3000000

ARMS PHASE III 2006 – Crop Expenses (Dollars)

23

Figures A17-A18: Census 2002 versus ARMS PHASE III 2006 Livestock Purchases

Census 2002 – Livestock Purchases (Dollars)

Scatter Plot of Livestock Purchases with Outliers
70000000
60000000
50000000
40000000
30000000
20000000
10000000
0
0

20000000

40000000

60000000

80000000

ARMS PHASE III 2006 – Livestock Purchases (Dollars)

Census 2002 –Livestock Purchases (Dollars)

Scatter Plot of Livestock Purchases without Outliers
3000000

2000000

1000000

0
0

1000000 2000000 3000000 4000000 5000000

ARMS PHASE III 2006 – Livestock Purchases (Dollars)
24

Figures A19-A20: Census 2002 versus ARMS PHASE III 2006 Feed Purchases

Census 2002 – Feed Purchases (Dollars)

Scatter Plot of Feed Purchases with Outliers
50000000

40000000

30000000

20000000

10000000

0
0

20000000

40000000

60000000

80000000

ARMS PHASE III 2006 – Feed Purchases (Dollars)

Census 2002 –Feed Purchases (Dollars)

Scatter Plot of Feed Purchases without Outliers
6000000

5000000

4000000

3000000

2000000

1000000

0
0

2000000

4000000

6000000

8000000

ARMS PHASE III 2006 – Feed Purchases (Dollars)

25

Figures A21-A22: Census 2002 versus ARMS PHASE III 2006 Hired Labor Expenses

Census 2002 – Hired Labor Expenses (Dollars)

Scatter Plot of Hired Labor Expenses with Outliers
13000000
12000000
11000000
10000000
9000000
8000000
7000000
6000000
5000000
4000000
3000000
2000000
1000000
0
0

10000000

20000000

30000000

ARMS PHASE III 2006 – Hired Labor Expenses (Dollars)

Census 2002 –Hired Labor Expenses (Dollars)

Scatter Plot of Hired Labor Expenses without Outliers
3000000

2000000

1000000

0
0

2000000

4000000

6000000

8000000

ARMS PHASE III 2006 – Hired Labor Expenses (Dollars)

26

Figures A23-A24: Census 2002 versus ARMS PHASE III 2006 Fuel & Oil Expenses

Census 2002 – Fuel & Oil Expenses (Dollars)

Scatter Plot of Fuel & Oil Expenses with Outliers
3000000

2000000

1000000

0
0

1000000

2000000

3000000

ARMS PHASE III 2006 – Fuel & Oil Expenses (Dollars)

Census 2002 – Fuel & Oil Expenses (Dollars)

Scatter Plot of Fuel & Oil Expenses without Outliers
190000
180000
170000
160000
150000
140000
130000
120000
110000
100000
90000
80000
70000
60000
50000
40000
30000
20000
10000
0
0

100000 200000 300000 400000 500000 600000

ARMS PHASE III 2006 – Fuel & Oil Expenses (Dollars)

27

Census 2002 – Machinery & Equipment Value (Dollars)

Census 2002 – Machinery & Equipment Value (Dollars)

Figures A25-A26: Census 2002 versus ARMS PHASE III 2006 Machinery &
Equipment Value
Scatter Plot of Machinery & Equipment Value with Outliers
7000000
6000000
5000000
4000000
3000000
2000000
1000000
0
0

5000000

10000000

15000000

ARMS PHASE III 2006 – Machinery & Equipment Value (Dollars)

Scatter Plot of Machinery & Equipment Value without Outliers
1600000
1500000
1400000
1300000
1200000
1100000
1000000
900000
800000
700000
600000
500000
400000
300000
200000
100000
0
0

1000000 2000000 3000000 4000000

ARMS PHASE III 2006 – Machinery & Equipment Value (Dollars)

28

Figures A27-A28: Census 2002 versus ARMS PHASE III 2006 Government Payments

Census 2002 – Government Payments (Dollars)

Scatter Plot of Government Payments with Outliers
1300000
1200000
1100000
1000000
900000
800000
700000
600000
500000
400000
300000
200000
100000
0
0

500000

1000000

1500000

ARMS PHASE III 2006 – Government Payments (Dollars)

Census 2002 – Government Payments (Dollars)

Scatter Plot of Government Payments without Outliers
150000
140000
130000
120000
110000
100000
90000
80000
70000
60000
50000
40000
30000
20000
10000
0
0

100000

200000

300000

400000

500000

ARMS PHASE III 2006 – Government Payments (Dollars)
29

Figures A29-A30: Census 2002 versus ARMS PHASE III 2006 Operator’s Age
Scatter Plot of Operator’s Age with Outliers

Census 2002 – Operator’s Age (Years)

100
90
80
70
60
50
40
30
20
10
- 10

0

10

20

30

40

50

60

70

80

90

100

ARMS PHASE III 2006 – Operator’s Age (Years)

Census 2002 – Operator’s Age (Years)

Scatter Plot of Operator’s Age without Outliers
90
80
70
60
50
40
30
20
0

10

20

30

40

50

60

70

80

90

ARMS PHASE III 2006 – Operator’s Age (Years)

30

Figures A31-A32: Census 2002 versus ARMS PHASE III 2006 Farm Type
Scatter Plot of Farm Type with Outliers

Census 2002 – Farm Type (Nominal)

16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
1

2

3

4

5

6

7

8

9

10

11 12

13 14

15

16

ARMS PHASE III 2006 – Farm Type (Nominal)
Scatter Plot of Farm Type without Outliers

Census 2002 – Farm Type (Nominal)

16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
1

2

3

4

5

6

7

8

9

10

11 12

13 14

15

16

ARMS PHASE III 2006 – Farm Type (Nominal)

31

32