Farm Production Expenditures Methodology and Quality Measures

Farm Production Expenditures
Methodology and Quality Measures
Released August 2, 2011, by the National Agricultural Statistics Service (NASS), Agricultural Statistics Board, United States Department of
Agriculture (USDA).

Agricultural Resource Management Survey Methodology Farm Production Expenditures
Scope and Purpose: Estimates of farm production expenditures are based on data collected in the Agricultural Resource
Management Survey (ARMS). The ARMS provides an annual snapshot of the financial health of the farm sector and farm
household finances. The ARMS is the only source of information available for objective evaluation of many critical
policy issues related to agriculture and the rural economy. The ARMS also collects data on production practices and costs
of production for two or three crop and livestock commodities each year, selected on a rotational basis. The cost of
production surveys are conducted in the top producing states while the farm production expenditures survey is conducted
in all states, except Alaska and Hawaii. In effect, the ARMS is several surveys conducted using a single data collection.
The ARMS is cosponsored by the USDA’s Economic Research Service (ERS).
The ARMS is conducted in three phases. The initial phase (ARMS I) screens a large sample of farms and ranches to
determine if a farm qualifies for the ARMS. Samples of qualifying farms are selected for the other two phases. The
second phase (ARMS II) collects data on agricultural production practices, chemical and other resource use, and variable
costs of production for each crop for which a cost of production survey is conducted. ERS is responsible for estimating
cost of production for major commodities and determines the commodity rotation.
The third phase (ARMS III) collects whole farm finance and operator characteristics information for a calendar year.
Respondents from the second phase are included in the third phase to obtain financial, resource use, and cost of
production data for the entire operation. It is vital that both the ARMS II and ARMS III questionnaires be completed for
these operations. Data from both phases provide the link between agricultural resource use and farm financial conditions
and allows for economic impact analysis of regulation and policy. This is a cornerstone of the ARMS design.
Farm production expenditures are estimated for five regions, which include the fifteen leading cash receipt states and the
other states within each region to account for all states except Alaska and Hawaii. Farm production expenditures are also
estimated for eight economic sales classes and two farm type categories.
Survey Timeline: Data collection and analysis for the ARMS I are conducted from May through July. The ARMS II data
collection begins in September and runs through December. The ARMS III data collection begins in January and concludes
in April with further analysis and review continuing until the results are published in early August.
Sampling: The target population for the ARMS is all agricultural establishments with more than $1,000 in agricultural
sales (or potential sales). NASS uses a dual frame approach, consisting of list frame and area frame components, to
provide complete coverage of this target population.
NASS maintains a list of farm and ranch operators. NASS is constantly seeking new operations from outside list sources
confirmed to be qualifying farms before being added to the list. A profile, known as control data, of each operation is
maintained which indicates what the farm has historically produced and a general indication of size. This information
allows NASS to define sampling populations that are specific to each survey and employ advanced and more efficient
sample designs.

The ARMS list sample is selected based on a calculated Farm Value of Sales (FVS). All farms on the list frame with an
estimated FVS of $1,000 or more are eligible. The value of sales control data need not be exact as it is used to stratify
similar list operations into homogeneous groups.
NASS utilizes the Sequential Interval Poisson (SIP) sampling method to select the ARMS I sample. In a SIP sample
design, each operation is assigned a Permanent Random Number (PRN) between 0 and 1 from a uniform distribution. A
sample can be defined as all operations falling within a specified range of PRNs. This provides a mechanism to control
overlap between samples across surveys. In this case, SIP is used to minimize overlap between the previous year’s ARMS
sample and the current year Agricultural Survey sample, which is a large scale quarterly acreage and grain stocks survey.
Calculating a probability of selection for each operation in the ARMS population is a complicated task. First, there are
multiple “surveys” in the ARMS. Each Production Practices and Costs Reports (PPCR) commodity requires a separate
sample and is considered a separate “survey” for sampling purposes. The farm production expenditures sample is divided
into two samples which use different questionnaires. The Costs and Returns Report (CRR) is used to obtain detailed
expense data and the Core questionnaire is an abbreviated version used for mail data collection. Each operation in the
population must be assigned a probability of selection for each survey and the sum of those probabilities must not be
greater than 1.
For the CRR and Core samples, target samples sizes by Farm Value of Sales (FVS) strata as well as Type of Farm
determine the probability of selection for each operation. For the cost of production surveys, probabilities of selection are
computed based on probability proportional to size (PPS) using crop acres or livestock inventory for the target
commodities.
The SIP sampling procedure is flexible and allows the use of different sample designs for each “survey” in the ARMS and
ensures that each operation is selected for one and only one survey of the ARMS. Once the probabilities of selection are
assigned the PRN determines which if any of the ARMS surveys will be assigned to that operation.
The area frame contains all land in the United States (except Alaska) and is therefore complete for the ARMS program.
The land is stratified according to intensity of agriculture using satellite imagery. Land in each stratum is divided into
segments of roughly one square mile. Segments are optimally allocated and sampled to effectively measure crops and
livestock. The area sample is completely enumerated in June. All farms and ranches found operating in these segments are
checked to see if they are included in the list frame ARMS population. Farms and ranches that are not on the list frame are
sampled for the ARMS so that the target population is completely represented.
The U.S. list frame sample size for the ARMS III is approximately 35,000. The area frame sample size is approximately
2,000. Each list frame and area frame sampling unit is assigned a sampling weight which is used to create the survey
estimates.
Data Collection: For consistency across modes, the paper version is considered the master questionnaire and the web and
Computer Assisted Telephone Interview (CATI) instruments are built to model the paper instrument. Questionnaire
content and format are evaluated annually by NASS and ERS through a specifications process where requests for changes
are evaluated and approved or disapproved. Input may vary from question wording or formatting to a program change
involving the deletion or modification of current questions or addition of new ones. If there are significant changes to
either the content or format proposed, a NASS survey methodologist will pre-test the changes for usability. Prior to the
start of data collection, all modes of instruments are reviewed and web and CATI instruments are thoroughly tested. ERS
also plays a significant role in the development of questionnaires.
All federal data collections require approval by the Office of Management and Budget (OMB). NASS must document the
public need for the data, show the design applies sound statistical practice, ensure the data do not already exist elsewhere,
and show that the public is not excessively burdened. The ARMS questionnaires must display an active OMB number that
gives NASS the authority to conduct the survey, a statement of the purpose of the survey and the use of the data being
collected, a response burden statement that gives an estimate of the time required to complete the form, a confidentiality
statement that the respondent’s information will be protected from disclosure, and a statement saying that response to the

2

Farm Production Expenditures Methodology and Quality Measures (August 2011)
USDA, National Agricultural Statistics Service

survey is voluntary and not required by law. For the ARMS, respondents must be informed that their data will be shared
with the cosponsor.
In addition to asking the specific economic and cost of production questions, all survey instruments collect information to
verify the sampled unit, determine any changes in the name or address, identify any partners to detect possible
duplication, verify the farm still qualifies for the target population, and identify any additional operations operated by the
sampled operator.
Sampled farms and ranches receive a presurvey postcard explaining the survey and that they will be contacted for survey
purposes only. Mail, web, telephone and face-to-face interview modes of data collection are utilized for the ARMS
survey. The ARMS III utilizes multiple questionnaire versions. A Core version of the questionnaire is used to obtain
global farm level expense, income, and household data. This version is used for mail out/mail back for a portion of the
sample and for web-based data collection in all states. The Costs and Returns Report asks for the same data as the Core
questionnaire in greater detail in all states and is collected by face-to-face interviews. A separate Costs and Returns Report
questionnaire is developed for each of the cost of production commodities with additional questions relating to the current
year’s targeted crop and/or livestock commodities.
Survey Edit: As survey data are collected and captured, they are edited for consistency and reasonableness using
automated systems. Reported data are typically first edited as a “batch” of data when first captured. The edit logic ensures
administrative coding follows the methodological rules associated with the survey design. Relationships between data
items on the current survey are verified and in certain situations, items are compared to data from earlier surveys to make
sure certain relationships are logical. The edit determines the status of each record to be either “dirty” or “clean”. Dirty
records must be updated and reedited, or certified by an analyst to be accurate. Corrected data are reedited interactively.
Only clean records are eligible for analysis tools and summary.
Analysis Tools: Edited economic and cost of production data are processed through an interactive analysis tool which
displays data for all reports by item. The tool provides various scatter plots, tables, charts, and special tabulations that
allow the analyst to compare an individual record to other similar records within their state. Outliers and unusual data
relationships become evident and Field Office and Headquarters staff review them to determine if they are correct.
Suspect data found to be in error are corrected, while data found to be correct are kept.
Nonsampling Errors: Nonsampling errors are present in any survey process. These errors include reporting, recording,
editing, and imputation errors. Steps are taken to minimize the impact of these errors, such as questionnaire testing,
comprehensive interviewer training, validation and verification of processing systems, detailed computer edits, and the
analysis tool.
Nonresponse Adjustment: Response to the ARMS is voluntary. Some producers refuse to participate in the survey,
others cannot be located during the data collection period, and some submit incomplete reports. These nonrespondents
must be accounted for if accurate estimates of total farm expenditures are to be made. For this survey, item level
nonresponse is accounted for by imputing data where there are missing values. Imputed values are calculated through an
automated imputation system that calculates an unweighted mean for an imputation group based on locality, farm type,
and value of sales class. These groups of homogeneous farms exclude extreme high and low end outliers so that the
imputed values are not biased as a result of a few large or unique operations. An imputation group must have a minimum
of ten or more positive responses. When a group lacks a sufficient number of responses, groups are collapsed by value of
sales class, locality, and farm type according to a defined hierarchy preserving as much of the homogeneity as possible.
About 75 percent of the necessary imputations are completed at the first level of imputation groups. Records with imputed
data are re-edited to ensure the returned value is acceptable. The imputation algorithm fails to deliver an acceptable value
less than one-half of one percent of the time and Field Office statisticians are required to manually impute for those items.
Calibration: Calibration is a weighting technique used in survey sampling to adjust the survey weights for sampled
elements so that the weighted sum of a set of benchmark variables equals a pre-determined set of values for the
population. The input to the calibration algorithm used for ARMS III is the weights generated from the sampling
procedures. Sampling weights are calculated based on numerous factors so that the sample allocation can be
representative of the entire population of farms at the state level for the fifteen leading cash receipts states and the five
Farm Production Expenditures Methodology and Quality Measures (August 2011)
USDA, National Agricultural Statistics Service

3

regions for all other states. Due to survey nonresponse and the possibility of disproportionate responses across different
farm types and economic sales classes, weights are adjusted through a calibration algorithm. Calibration adjusts the
sampling weights so that the expanded data will match several known commodity, livestock and farm number published
totals. This ensures that the expense data collected will accurately represent the expense breakdowns for all farm types
and farm sizes as well as cover the expenses for the entire target population.
Estimators: The ARMS utilizes direct expansions for all survey indications. For both the list and area frame respondents,
direct expansions are calculated by summing the reported or imputed economic and expenditure values weighted by the
calibration weights. Variance estimates are computed for all expansions. The dual frame direct expansion and variance are
the sum of the estimates from the list frame and the portion of the area frame that contains operations not included on our
list frame population for the ARMS.
Outliers: NASS conducts a formal review of outliers found in the expanded data. Outliers may be caused by aging
control data resulting in misstratification, data errors, or the nonresponse and calibration adjustments to the sampling
weight. A preliminary calibration and summary are run and any individual record accounting for 0.5 percent of the
national expansion for total expenses or 2.5 percent of a regional expansion for total expenses is tagged as an outlier.
After verifying the data have not been misrecorded or mishandled, background information on these outliers is compiled
and presented to a National Outlier Board. A team of NASS and ERS analysts meet to discuss the national outliers and
form a consensus on a course of action. Most outliers trace back to unique situations that do not exist in the target
population as often as a large calibrated sample weight indicates. The Board looks at other reports of the same type and
sales class as the reported data on the outlier. The Board examines the weights of the comparable reports and most often
overrides the outlier’s weight with the median weight of the comparable reports. After the most extreme outliers have
been addressed, the Board reviews the national totals by expense category following the same methodology and, when
necessary, overrides the outlier’s weight with the median weight of the comparable reports. Finally, Headquarters staff
examines outliers found at the state level for the published expense categories. A determination is made as to whether a
weight adjustment is justified. Adjustments are not made to all outliers, but they are reviewed closely for accuracy. Once
all adjustments are made, the calibration program is executed again to create the final set of weights for summary
purposes.
Estimation: When all samples are accounted for, all responses fully edited, and the analysis material is reviewed,
Headquarters staff executes a summary that generates state level totals for the fifteen leading cash receipt states and
regional totals for the remaining states. Since all states conduct identical surveys, the samples can be pooled and national
survey results computed. The summary results provide point estimates and their standard errors for each data series being
estimated. It also provides information used to assess the performance of the current survey and evaluate the quality of the
survey estimates, such as expansions by farm types and economic sales classes, response rates, and the effectiveness of
calibration.
The ARMS III supports the annual estimates of total farm expenditures and the total expenses related to the following
categories: Livestock, Feed, Farm Services, Rent, Agricultural Chemicals, Fertilizer, Interest, Taxes, Labor Costs, Fuels,
Farm Supplies and Repairs, Farm Improvements and Construction, Tractors and Self-Propelled Farm Machinery, Other
Farm Machinery, Seeds, Trucks and Autos, and Miscellaneous Capital Expenses.
Field Offices are responsible for performing a detailed review and providing comments that justify survey results. Any
irregularities revealed by the summary must be investigated and, if necessary, resolved. Field Offices see their state’s
survey results only and do not have access to other states’ results.
For the national estimates, NASS assembles another joint panel of NASS and ERS analysts to serve as the Agricultural
Statistics Board which reviews the national level survey results and establishes the national estimates. Since larger sample
sizes yield more precise results, NASS employs the “top-down” approach by determining the national estimates first and
reconciling the region and state estimates to the national number for all estimates. The Board has the advantage of being
able to examine results across states, review the state comments and justifications, and utilize external administrative data
available only at the U.S. level to corroborate survey results. The same estimators used in the state summaries are utilized
by the national summary. Upon Board consensus, national level summary indications are adopted as official national
estimates except in cases where strong justification supports deviating from survey totals. In a separate process, a team of
4

Farm Production Expenditures Methodology and Quality Measures (August 2011)
USDA, National Agricultural Statistics Service

Field Office and Headquarters staff must re-examine the region and state results and adjust state estimates to sum to the
national estimate. This process occurs both when indications are adopted as official estimates and when estimates deviate
from survey totals.
Estimates are open to revision on a predetermined schedule only if new information becomes available. In general,
revisions to the expenditure estimates may be considered one year later if there were revisions made to any of the
calibration targets or other information became available that significantly impacts the previous year estimates. External
information (administrative data) is also utilized in this process. In order to be considered, these data must be deemed to
be reliable and come from unbiased sources. Census of Agriculture farm production expenditure estimates are available
every five years and are used to assess the accuracy of the ARMS III results.

Farm Production Expenditures Methodology and Quality Measures (August 2011)
USDA, National Agricultural Statistics Service

5

Quality Metrics for Farm Production Expenditures
Purpose and Definitions: Under the guidance of the Statistical Policy Office of the Office of Management and Budget
(OMB), the United States Department of Agriculture’s National Agricultural Statistics Service (NASS) provides data users
with quality metrics for its published data series. The metrics tables below describe the performance data for the survey
contributing to the publication. The accuracy of data products may be evaluated through sampling and non-sampling error.
The measurement of error due to sampling in the current period is evaluated by the coefficient of variation for each estimated
item. Non-sampling error is evaluated by response rates and the percent of the estimate from respondents.
Sample Size is the number of observations selected from the population that are used to be representative of the
entire population.
Response rates measure the proportion of the sample that is represented by the responding units in the survey.
Coefficient of Variation provides a measure of the size for the standard error relative to the point estimate and is
used to measure the precision of the results of a survey estimator.
Percent of estimate from respondents is a ratio of survey data expanded by the sampling weight compared to
survey data expanded by the calibrated weight that adjusts for survey nonresponse and other non-sampling errors. In
other words, it is the percent of the estimate represented by the actual survey respondents.
Farm Production Expenditures Sample Size and Response Rates – United States: 2009 and 2010
State and region

Sample size
2009

Response rate
2010

(number)

2009

(number)

2010

(percent)

(percent)

Atlantic ............................................................
North Carolina .................................................
Other States ....................................................

3,923
1,605
2,318

4,910
1,386
3,524

75.1
83.5
69.4

72.2
86.8
66.4

South ...............................................................
Arkansas .........................................................
Florida .............................................................
Georgia ...........................................................
Other States ....................................................

5,030
1,454
1,319
1,333
924

5,158
1,282
1,420
1,447
1,009

74.4
75.2
75.3
67.8
81.1

69.9
73.9
69.1
60.8
79.3

Midwest ...........................................................
Illinois ..............................................................
Indiana ............................................................
Iowa ................................................................
Minnesota .......................................................
Missouri ..........................................................
Wisconsin .......................................................
Other States ....................................................

11,039
1,722
1,508
1,886
1,695
1,638
1,435
1,155

12,216
1,745
1,606
1,921
1,791
1,696
1,991
1,466

66.9
66.8
68.5
63.7
71.8
67.3
68.9
59.6

63.2
62.3
60.6
64.1
65.7
66.6
64.0
57.8

Plains ...............................................................
Kansas ............................................................
Nebraska ........................................................
Texas ..............................................................
Other States ....................................................

7,161
1,616
1,782
1,950
2,382

6,940
1,677
1,686
2,310
2,375

64.3
57.4
60.3
78.5
69.2

63.7
53.1
60.7
75.7
69.1

West .................................................................
California .........................................................
Washington .....................................................
Other States ....................................................

6,195
2,238
1,552
2,405

6,207
2,566
1,615
2,026

65.7
64.0
70.4
64.2

64.1
63.3
68.2
61.9

United States ......................................................

33,348

35,431

68.2

65.7

6

Farm Production Expenditures Methodology and Quality Measures (August 2011)
USDA, National Agricultural Statistics Service

Quality Metrics for Farm Production Expenditures – Region, Selected States, Farm Type, and
Economic Class: 2009 and 2010
Total

Coefficients of variation
2009

Percent of estimate from respondents

2010

(number)

2009

(number)

2010

(percent)

(percent)

Total farm production expenditures ...........................
Livestock, poultry and related expenses ...................
Feed .........................................................................
Farm services ...........................................................
Rent ..........................................................................
Agricultural chemicals ...............................................
Fertilizer, lime and soil conditioners ..........................
Interest .....................................................................
Taxes (real estate and property) ...............................
Labor ........................................................................

1.1
4.7
2.6
1.2
1.3
1.4
0.9
1.5
0.9
2.3

1.3
5.8
3.1
1.3
1.4
2.0
1.5
1.9
1.0
3.1

58.4
51.5
57.2
61.1
57.2
59.0
58.6
58.5
58.6
61.8

60.5
58.7
60.2
60.6
61.9
62.5
60.8
57.3
58.5
60.4

Fuels ........................................................................
Farm supplies and repairs ........................................
Farm improvements and construction .......................
Tractors and self-propelled farm machinery ..............
Other farm machinery ...............................................
Seeds and plants ......................................................
Trucks and autos ......................................................
Miscellaneous capital expenses ................................

1.0
1.6
2.3
2.1
2.4
1.4
2.6
6.3

1.4
1.5
2.2
2.8
2.6
1.7
2.8
8.8

60.4
59.9
58.2
60.1
60.9
57.2
59.1
78.4

60.8
59.7
60.1
62.9
63.2
60.4
62.4
69.4

Atlantic ..................................................................
North Carolina ......................................................
Other States .........................................................
South .....................................................................
Arkansas ..............................................................
Florida ..................................................................
Georgia .................................................................
Other States .........................................................

2.5
1.4
3.2
2.5
1.7
2.9
4.4
5.6

3.1
2.6
3.9
2.6
2.1
4.5
1.8
5.8

66.6
80.4
62.1
62.5
62.0
49.3
54.8
72.2

69.9
83.6
65.8
64.5
59.0
50.0
49.2
81.2

Midwest .................................................................
Illinois ...................................................................
Indiana ..................................................................
Iowa ......................................................................
Minnesota .............................................................
Missouri ................................................................
Wisconsin .............................................................
Other States .........................................................

1.0
2.1
1.3
2.3
2.5
0.8
1.5
4.2

1.3
1.3
2.0
3.8
2.2
1.7
1.4
5.2

59.9
62.8
57.6
53.8
65.2
59.7
60.4
62.7

56.3
55.9
50.7
57.8
58.6
63.1
55.1
52.4

Plains .....................................................................
Kansas .................................................................
Nebraska ..............................................................
Texas ....................................................................
Other States .........................................................
West .......................................................................
California ..............................................................
Washington ...........................................................
Other States .........................................................

3.0
14.2
2.3
3.6
3.8
2.5
3.5
2.3
4.4

3.6
2.6
8.8
6.6
7.1
3.3
5.8
3.8
4.1

51.3
41.9
42.9
60.9
53.7
57.5
58.0
65.3
55.2

62.5
39.7
44.9
93.5
58.6
56.4
54.7
67.6
55.6

1.3
2.1

1.6
2.3

59.5
57.2

62.5
58.0

2.2
2.6
2.2
1.7
1.7
1.3
1.5
5.0

2.2
2.2
2.5
2.4
2.0
1.8
2.1
5.8

55.6
72.7
47.0
60.6
58.9
52.1
66.5
51.5

56.1
67.1
54.6
59.5
55.5
54.3
69.5
57.3

Regional - total farm expense

Farm type - total farm expense
Crop .....................................................................
Livestock ..............................................................
Economic class - total farm expense
Less than $10,000 ................................................
$10,000-$49,999 ...................................................
$50,000-$99,999 ...................................................
$100,000-$249,999 ...............................................
$250,000-$499,999 ...............................................
$500,000-$999,999 ...............................................
$1,000,000-$4,999,999 .........................................
$5,000,000 and over .............................................

Farm Production Expenditures Methodology and Quality Measures (August 2011)
USDA, National Agricultural Statistics Service

7

Information Contacts
Process

Unit

Estimation ....................................... Environmental, Economics, and
Demographics Branch ........................................
Data Collection ............................... Program Administration Branch ..........................
Questionnaires and Editing ............. Editing and Questionnaire Branch ......................
Sampling ......................................... Survey Sampling Branch ....................................
Analysis and Estimators .................. Statistical Methods Branch .................................
Dissemination and Webmaster ........ Data Dissemination Section ...............................
Media Contact ................................. Public Affairs Section .........................................

8

Telephone
(202) 720-6146
(202) 690-8747
(202) 720-6201
(202) 720-3895
(202) 720-4008
(202) 720-7017
(202) 720-7017

Email
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]

Farm Production Expenditures Methodology and Quality Measures (August 2011)
USDA, National Agricultural Statistics Service