Farm Production Expenditures Methodology and Quality Measures

Farm Production Expenditures
Methodology and Quality Measures
ISSN: 2166-966X

Released July 29, 2022, by the National Agricultural Statistics Service (NASS), Agricultural Statistics Board, United States Department of
Agriculture (USDA).

Agricultural Resource Management Survey Methodology for Farm Production Expenditures
Scope and Purpose: Estimates of farm production expenditures are based on data collected in the Agricultural Resource
Management Survey (ARMS). The ARMS provides an annual snapshot of the financial health of the farm sector and farm
household finances. The ARMS is the only source of information available for objective evaluation of many critical
policy issues related to agriculture and the rural economy. In addition to obtaining farm production expenditures, the
ARMS also collects data on production practices and costs of production for designated crop and livestock commo dities
each year, selected on a rotational basis. The production practices and cost of production surveys for these designated
commodities are conducted in the top producing states for the selected commodity, while the farm production
expenditures survey is conducted in all states, except Alaska and Hawaii. The ARMS is cosponsored by the USDA’s
Economic Research Service (ERS).
The ARMS is conducted in three phases. The initial phase (ARMS I) screens a large sample of farms and ranches to
determine which farms qualify for subsequent phases of ARMS. Subsamples of qualifying farms are selected for the other
two phases. The second phase (ARMS II) collects data on agricultural production practices, chemical use, and costs of
production for each designated crop commodity. ERS is responsible for estimating the cost of production for major
commodities and determines the commodity rotation.
The third phase (ARMS III) collects whole farm finance and operator characteristics information for a calendar year.
Respondents from the second phase are included in the third phase to obtain financial and farm production expenditure
data for the entire operation. It is vital that both the ARMS II and ARMS III be completed for these designated crop
commodity operations. Data from both phases provide the link between agricultural resource use and farm financial
conditions and allows for economic impact analysis of regulation and policy. This is a cornerstone of the ARMS design.
In addition, production practices, costs of production, and farm production-expenditure data for designated livestock
commodities are collected in one interview during the third phase (ARMS III).
Farm production expenditures are estimated for five regions, which include the fifteen leading cash receipt states and the
other states within each region to account for all states except Alaska and Hawaii. Farm production expenditures are also
estimated for eight economic sales classes and two farm type categories.
Survey Timeline: Data collection and analysis for the ARMS I are conducted from May through July. The ARMS II data
collection begins in September and runs through December. The ARMS III data collection begins in January and concludes
in April with further analysis and review continuing until the results are published in late July.
Sampling: The target population for the ARMS is all agricultural establishments with more than $1,000 in agricultural
sales (or potential sales). NASS uses a dual frame approach, consisting of list frame and area frame components, to
provide complete coverage of this target population.
NASS maintains a list of farm and ranch operators, known as the list frame. NASS is constantly seeking new operations
from outside list sources confirmed to be qualifying farms before being added to the list. A profile, known as control data,
of each operation is maintained which indicates what the farm has historically produced and a general indication of size.
This information allows NASS to define list-frame sampling populations that are specific to each survey and employ
advanced and more efficient sample designs.

The ARMS list sample is selected based on a calculated Farm Value of Sales (FVS). All farms on the list frame with an
estimated FVS of $1,000 or more are eligible. The value of sales control data need not be exact as it is used to stratify
similar list operations into homogeneous groups.
NASS utilizes the Sequential Interval Poisson (SIP) sampling method to select the ARMS I sample. In a SIP sample
design, each operation is assigned a Permanent Random Number (PRN) between 0 and 1 from a uniform distribution. A
sample can be defined as all operations falling within a specified range of PRNs. This provides a mechanism to control
overlap across multiple surveys. In this case, SIP is used to minimize overlap between the previous year’s ARMS sample
and the current year Agricultural Survey sample, which is a large-scale quarterly acreage and grain stocks survey
program.
Calculating a probability of selection for each operation in the ARMS population is a complicated task. There are multiple
questionnaire versions in the ARMS III. Each designated commodity for the production practices versions requires a
separate sample.
For the two farm production expenditure subsamples, target sample sizes by Farm Value of Sales (FVS) strata determine
the probability of selection for each operation. For designated-commodity samples in the production practices versions,
probabilities of selection are computed based on probability proportional to size (PPS) using crop acres or livestock
inventory for the designated commodities.
The SIP sampling procedure is flexible and allows the use of different sample designs for each version in the ARMS III. It
also ensures that each operation is selected for one and only one version. Once the probabilities of selection are assigned,
the PRN determines which, if any, of the ARMS III versions will be assigned to that operation.
The area frame contains all land in the United States (except Hawaii and Alaska) and is therefore complete for the ARMS
III program. The land is stratified according to intensity of agriculture using satellite imagery. Land in each stratum is
divided into segments of roughly one square mile. Segments are optimally allocated and sampled to effectively measure
crops and livestock. Annually, NASS conducts the June Area Survey and conducts face-to-face interviews of every
individual who operates or owns land within a sampled area segment. All farms and ranches found operating in these
segments are checked to see if they are included in the list-frame ARMS III population. Farms and ranches that are not on
the list frame are sampled for the ARMS III so that the target population is completely represented.
The United States list-frame sample size for the ARMS III is approximately 36,000. The area-frame sample size is
approximately 2,000. Each list-frame and area-frame sampling unit is assigned a sampling weight which is used to create
the survey estimates.
Data Collection: For consistency across modes, the paper version is considered the master questionnaire and Computer
Assisted Web Interview (CAWI) and Computer Assisted Personal Interview (CAPI) instruments are built to model the
paper instrument. ERS plays a significant role in the development of questionnaires. Questionnaire content and format are
evaluated annually by NASS and ERS through a specifications process, where requests for changes are evaluated and
approved or disapproved. Input may vary from question wording or formatting to a program change involving the deletion
or modification of current questions or addition of new ones. If significant changes to either the content or format are
proposed, a NASS survey methodologist will pre-test the changes for usability. Prior to the start of data collection, all
modes of instruments are reviewed and the CAWI instrument is thoroughly tested.
All federal data collections require approval by the Office of Management and Budget (OMB). NASS must document the
public need for the data, show the design applies sound statistical practice, ensure the data do not already exist elsewhere,
and show that the public is not excessively burdened. The ARMS questionnaires must display an active OMB number that
gives NASS the authority to conduct the survey, as well as a statement of the purpose of the survey and the use of the data
being collected. The questionnaires must include a response burden statement that gives an estimate of the time required
to complete the form, a confidentiality statement that the respondents’ information will be protected from disclosure, and
a statement saying that response to the survey is voluntary and not required by law. For the ARMS, respondents must be
informed that their data will be shared with the cosponsor.

2

Farm Production Expenditures Methodology and Quality Measures (July 2022)
USDA, National Agricultural Statistics Service

In addition to asking the specific economic and cost of production questions, all survey instruments collect information to
verify the sampled unit, determine any changes in the name or address, identify any partners to detect possible
duplication, verify the farm still qualifies for the target population, and identify any additional operations operated by the
sampled operator.
Mail, web, telephone, and face-to-face interview modes of data collection are utilized for the ARMS survey. The ARMS
III uses multiple questionnaire versions. Prior to 2012, a Core version of the questionnaire was used to obtain global farm
level expense, income, and household data. This version was used for the mail portion of the sample and for web-based
data collection in the fifteen leading cash receipt states. The Costs and Returns Report (CRR) asks for the same data as the
Core questionnaire, but in greater detail and in all states. Prior to 2012, the Core version was collected by face-to-face
interviews only; beginning in 2012, the Costs and Returns Report has been used for the mail and web -based portion of the
sample in addition to face-to-face interviews. A separate ARMS III production practices questionnaire version is
developed for each of the designated commodity samples with additional questions relating to the current year’s
designated crop and livestock commodities. A reminder postcard is mailed to sampled farms and ranches to request the
return of a completed questionnaire. The postcard also thanks respondents if they have already returned their
questionnaire.
Survey Edit: As survey data are collected and captured, they are edited for consistency and reasonableness using
automated systems. Reported data are typically first edited as a “batch” of data when first captured. The edit logic ensures
administrative coding follows the methodological rules associated with the survey design. Relationships between data
items on the current survey are verified and, in certain situations, items are compared to data from earlier surveys to make
sure certain relationships are logical. The edit determines the status of each record to be either “dirty” or “clean”. Dirty
records must be either updated or certified by an analyst to be accurate. Corrected data are reedited interactively. Only
clean records are eligible for analysis tools and summary.
Analysis Tool: Edited economic and cost of production data are processed through an interactive analysis tool that
displays data for all reports by questionnaire item. The tool provides various scatter plots, tables, charts, and special
tabulations that allow the analyst to compare an individual record to other similar records within their state. These tools
make outliers and unusual data relationships evident and Regional Field Offices and Headquarters staff review them to
determine if they are correct. Suspect data found to be in error are corrected, while data found to be correct are kept.
Nonsampling Errors: Nonsampling errors are present in any survey process. These errors include reporting, recording,
editing, and imputation errors. Steps are taken to minimize the impact of these errors, such as questionnaire testing,
comprehensive interviewer training, validation and verification of processing systems, detailed computer edits, and the
analysis tool.
Nonresponse Adjustment: Response to the ARMS III is voluntary. Some producers refuse to participate in the survey,
others cannot be located during the data collection period, and some submit incomplete reports. These nonrespondents
must be accounted for if accurate estimates of total farm expenditures are to be made. For this survey, item level
nonresponse is accounted for by imputing data where there are missing values using a multivariate approach. Prior to the
implementation of the multivariate approach, NASS used an un-weighted conditional means imputation system that
placed records into homogenous groups and imputed based on reported data from those groups. From 2014 to 2018 NASS
used a technique called Iterative Sequential Regression (ISR), a multivariate method where parameter estimates and
imputations are obtained using a Markov chain Monte Carlo sampling method. Beginning in 2019 the new multivariate
approach uses a regression-based hot deck imputation methodology called Predictive Mean Matching (PMM) which
creates a multivariate model of the fully and partially observed records to create a donor pool of records who reported
values are used and imputed values. PMM allows for the relationships within the data to be preserved and the imputed
values to better represent the variability of the data. Records with imputed data are re-edited to ensure the returned value
is acceptable.
Calibration: Calibration is a weighting technique used in survey sampling to adjust the survey weights for sampled
elements so that the weighted sum of a set of benchmark variables equals a pre-determined set of values for the
population. The input to the calibration algorithm used for ARMS III is the weights generated from the sampling
procedures. Sampling weights are calculated based on numerous factors so that the sample allocation can be
Farm Production Expenditures Methodology and Quality Measures (July 2022)
USDA, National Agricultural Statistics Service

3

representative of the entire population of farms at the state level for the fifteen leading cash receipts states and at the
regional level for all other states. Due to survey nonresponse and the possibility of disproportionate responses across
different farm types and economic sales classes, weights are adjusted through a calibration algorithm. Calibration adjusts
the sampling weights so that the expanded data will match several known commodity, livestock, and farm number
published totals. This ensures that the expense data collected will accurately represent the expense breakdowns for all
farm types and farm sizes as well as cover the expenses for the entire target population.
Estimators: The ARMS III utilizes calibrated direct expansions for all survey indications. For both the list and area frame
respondents, calibrated direct expansions are calculated by summing the reported or imputed economic and expenditure
values weighted by the calibration weights. Variance estimates are computed for all expansions. The dual frame calibrated
direct expansion and variance are the sum of the estimates from the list frame and the portion of the area frame that
contains operations not included on our list frame population for the ARMS III.
Outliers: NASS conducts a formal review of outliers found in the expanded data. Outliers may be caused by aging
control data resulting in misstratification, data errors, or the nonresponse and calibration adjustments to the sampling
weight. A preliminary calibration and summary are run and any individual record accounting for 0.5 percent of the
national expansion for total expenses or 2.5 percent of a regional expansion for total expenses is tagged as an outlier.
After verifying the data have not been misrecorded or mishandled, background information on these outliers is compiled
and presented to a National Outlier Board. This Board is a team of NASS and ERS analysts that meets to discuss the
national outliers and form a consensus on a course of action. Most outliers trace back to unique situations that do not exist
in the target population as often as a large calibrated sample weight indicates. The Board looks at other repo rts of the
same type and sales class as the reported data on the outlier. The Board examines the weights of the comparable reports
and most often overrides the outlier’s weight with the median weight of the comparable reports. After the extreme outliers
have been addressed, the Board reviews the national totals by expense category following the same methodology and,
when necessary, overrides the outlier’s weight with the median weight of the comparable reports. Finally, Headquarters
staff examines outliers found at the state level for the published expense categories. A determination is made as to
whether a weight adjustment is justified. Adjustments are not made to all outliers, but they are reviewed closely for
accuracy. Once all adjustments are made, the calibration program is executed again to create the final set of weights for
summary purposes.
Estimation: When all samples are accounted for, all responses fully edited, and the analysis material is reviewed,
Headquarters staff executes a summary that generates state level totals for the fifteen leading cash receipt states and
regional totals for the remaining states. Since all states (except Hawaii and Alaska) conduct identical surveys, the samples
can be pooled and national survey results computed. The summary results provide point estimates and their standard
errors for each estimated data series. It also provides information used to assess the performance of the current survey and
evaluate the quality of the survey estimates, such as expansions by farm types and economic sales classes, response rates,
and the effectiveness of calibration.
The ARMS III supports the annual estimates of total farm expenditures and the total expenses related to the following
categories: Livestock, Feed, Farm Services, Rent, Agricultural Chemicals, Fertilizer, Interest, Taxes, Labor, Fuels, Farm
Supplies and Repairs, Farm Improvements and Construction, Tractors and Self-Propelled Farm Machinery, Other Farm
Machinery, Seeds, Trucks and Autos, and Miscellaneous Capital Expenses.
Regional Field Offices are responsible for performing a detailed review and providing comments that justify survey
results. Any irregularities revealed by the summary must be investigated and, if necessary, resolved. Regional Field
Offices see their states’ survey results only and do not have access to other states’ results.
For the national estimates, NASS assembles another joint panel of NASS and ERS analysts to serve as the Agricultural
Statistics Board, which reviews the national level survey results and establishes the national estimates. Since larger
sample sizes yield more precise results, NASS employs the “top-down” approach by determining the national estimates
first and then reconciling the region and state estimates to the national number for all estimates. The Board has the
advantage of being able to examine results across states, review the state comments and justifications, and utilize external
administrative data available only at the U.S. level to corroborate survey results. The same estimators used in the state
summaries are utilized by the national summary. Upon Board consensus, national level summary indications are adopted
4

Farm Production Expenditures Methodology and Quality Measures (July 2022)
USDA, National Agricultural Statistics Service

as official national estimates except in cases where strong justification supports deviating from survey totals. In a separate
process, a team of Field Office and Headquarters staff must reexamine the region and state results and adjust state
estimates to sum to the national estimate. This process occurs both when indications are adopted as official estimates and
when estimates deviate from survey totals.
Estimates are open to revision on a predetermined schedule only if new information becomes available. In general,
revisions to the expenditure estimates may be considered one year later if there were revisions made to any of the
calibration targets or other information became available that significantly impacts the previous year estimates. External
information (administrative data) is also utilized in this process. In order to be considered, these data must be deemed to
be reliable and come from unbiased sources. Census of Agriculture farm production expenditure estimates are available
every five years and are used to assess the accuracy of the ARMS III results.

Farm Production Expenditures Methodology and Quality Measures (July 2022)
USDA, National Agricultural Statistics Service

5

Quality Metrics for Farm Production Expenditures
Purpose and Definitions: Under the guidance of the Statistical Policy Office of the Office of Management and Budget
(OMB), the United States Department of Agriculture’s National Agricultural Statistics Service (NASS) provides data
users with quality metrics for its published data series. The metrics tables below describe the performance data for the
survey contributing to the publication. The accuracy of data products may be evaluated through sampling and
nonsampling error. The measurement of error due to sampling in the current period is evaluated by the coefficient of
variation for each estimated item. Non-sampling error is evaluated by response rates and the percent of the estimate from
respondents.
Sample size is the number of observations selected from the population to represent a characteristic of the
population. Operations that did not have the item of interest or were out of business at the time of data collection
have been excluded.
Response rate is the proportion of the sample that completed the survey. This calculation follows Guideline 3.2.2 of
the Office of Management and Budget Standards and Guidelines for Statistical Surveys (Sept 2006).
Coefficient of variation provides a measure of the size for the standard error relative to the point estimate. It is used
to measure the precision of the results of a survey estimator.
Weighted item response rate is a ratio of reported survey data expanded by the original sampling weight compared
to final nonresponse adjusted summary totals.
Farm Production Expenditures Sample Size and Response Rates – United States: 2020 and 2021
State and region

Sample size
2020

Response rate
2021

(number)

2020

(number)

2021

(percent)

(percent)

Atlantic .................................................................
North Carolina .....................................................
Other States ........................................................

3,627
1,538
2,089

4,837
1,494
3,343

40.6
46.6
36.3

33.2
40.6
29.9

South ....................................................................
Arkansas .............................................................
Florida .................................................................
Georgia ...............................................................
Other States ........................................................

5,823
1,408
1,842
1,491
1,082

6,369
1,612
1,883
1,593
1,281

37.8
41.4
31.1
37.8
44.5

39.4
47.7
30.7
38.0
43.4

Midwest ................................................................
Illinois ..................................................................
Indiana .................................................................
Iowa .....................................................................
Minnesota ............................................................
Missouri ...............................................................
Wisconsin ............................................................
Other States ........................................................

10,371
1,627
1,512
1,713
1,587
1,623
1,311
998

12,272
1,774
1,759
1,825
1,725
1,842
1,832
1,515

34.4
32.4
30.0
34.9
37.4
35.2
39.1
30.7

36.3
36.0
32.9
38.4
41.9
34.0
36.5
33.9

Plains ....................................................................
Kansas ................................................................
Nebraska .............................................................
Texas ...................................................................
Other States ........................................................

6,928
1,619
1,644
2,494
1,171

7,481
1,731
1,612
2,660
1,478

31.5
27.8
28.7
37.4
28.1

34.3
31.7
30.0
41.4
29.5

West ......................................................................
California .............................................................
Washington .........................................................
Other States ........................................................

5,605
2,219
1,792
1,594

6,443
2,558
1,928
1,957

42.2
48.0
37.6
39.4

38.8
46.1
34.1
34.0

United States ..........................................................

32,354

37,402

36.4

36.5

6

Farm Production Expenditures Methodology and Quality Measures (July 2022)
USDA, National Agricultural Statistics Service

Quality Metrics for Farm Production Expenditures – Region, Selected States, Farm Type, and
Economic Class: 2020 and 2021
Total

Coefficient of variation
2020

Weighted item response rate

2021

(percent)

2020

(percent)

2021

(percent)

(percent)

Total farm production expenditures ....................................
Livestock, poultry, and related expenses ...........................
Feed .....................................................................................
Farm services ......................................................................
Rent ......................................................................................
Agricultural chemicals .........................................................
Fertilizer, lime, and soil conditioners ..................................
Interest .................................................................................
Taxes (real estate and property) ........................................
Labor ....................................................................................

2.0
6.4
4.0
2.6
2.8
3.2
2.4
3.6
2.5
4.4

2.4
9.1
5.6
3.0
2.4
2.8
2.2
3.8
2.1
4.9

36.3
50.0
39.0
35.2
30.0
30.8
31.3
34.2
34.9
33.9

36.2
56.9
40.2
33.2
30.7
31.9
30.6
34.7
34.8
31.0

Fuels .....................................................................................
Farm supplies and repairs ..................................................
Farm improvements and construction ................................
Tractors and self-propelled farm machinery ......................
Other farm machinery .........................................................
Seeds and plants .................................................................
Trucks and autos .................................................................
Miscellaneous capital expenses .........................................

2.7
3.0
5.3
5.1
6.8
3.5
10.5
13.3

2.1
2.7
4.5
4.0
5.6
3.0
4.6
5.5

34.1
34.0
44.0
34.3
34.7
30.5
41.4
43.0

32.8
32.0
33.8
32.7
34.0
30.4
38.7
37.7

Atlantic ..............................................................................
North Carolina .................................................................
Other States ....................................................................

6.7
4.6
8.9

7.1
13.2
8.5

39.1
38.0
39.5

32.0
37.4
30.0

South .................................................................................
Arkansas ..........................................................................
Florida ..............................................................................
Georgia ............................................................................
Other States ....................................................................

6.3
3.7
20.6
5.2
11.3

4.7
3.0
15.3
4.2
9.0

35.2
34.4
30.7
29.4
40.9

34.2
40.5
18.9
35.0
38.2

Midwest .............................................................................
Illinois ...............................................................................
Indiana .............................................................................
Iowa ..................................................................................
Minnesota ........................................................................
Missouri ............................................................................
Wisconsin ........................................................................
Other States ....................................................................

1.7
3.0
2.7
2.8
2.8
5.3
3.3
8.4

4.0
2.7
2.3
13.3
3.7
5.7
2.9
9.2

30.4
31.2
27.3
24.0
30.6
36.2
39.5
32.7

36.7
40.6
27.3
34.6
37.0
40.3
39.9
37.6

Plains .................................................................................
Kansas .............................................................................
Nebraska .........................................................................
Texas ...............................................................................
Other States ....................................................................

4.8
11.2
9.3
8.2
9.7

5.1
18.1
8.6
4.8
6.7

39.2
36.4
35.8
52.7
31.8

42.7
53.3
45.0
49.4
24.5

West ...................................................................................
California ..........................................................................
Washington ......................................................................
Other States ....................................................................

5.0
5.2
11.4
10.2

6.1
5.5
10.1
13.1

40.2
32.8
43.2
48.3

31.2
33.9
23.1
30.1

2.5
3.3

2.3
4.5

31.3
42.1

31.7
41.2

7.8
5.4
5.0
4.0
4.7
3.8
3.1
5.9

6.5
3.9
5.4
3.6
3.7
3.6
3.0
7.2

51.6
48.3
41.2
36.7
30.5
28.3
29.2
44.9

37.7
46.4
40.0
34.3
31.3
30.0
31.5
43.1

Regional - total farm production expenditure

Farm type - total farm production expenditure
Crop .................................................................................
Livestock ..........................................................................
Economic class - total farm production expenditure
Less than $10,000 ...........................................................
$10,000-$49,999 .............................................................
$50,000-$99,999 .............................................................
$100,000-$249,999 .........................................................
$250,000-$499,999 .........................................................
$500,000-$999,999 .........................................................
$1,000,000-$4,999,999 ...................................................
$5,000,000 and over .......................................................

Farm Production Expenditures Methodology and Quality Measures (July 2022)
USDA, National Agricultural Statistics Service

7

Access to NASS Reports
For your convenience, you may access NASS reports and products the following ways:
➢ All reports are available electronically, at no cost, on the NASS web site: www.nass.usda.gov.
➢ Both national and state specific reports are available via a free e-mail subscription. To set-up this free
subscription, visit www.nass.usda.gov and click on “National” or “State” in upper right corner above “search”
box to create an account and select the reports you would like to receive.
➢ Cornell’s Mann Library has launched a new website housing NASS’s and other agency’s archived reports. The
new website, https://usda.library.cornell.edu. All email subscriptions containing reports will be sent from the new
website, https://usda.library.cornell.edu. To continue receiving the reports via e-mail, you will have to go to the
new website, create a new account and re-subscribe to the reports. If you need instructions to set up an account or
subscribe, they are located at: https://usda.library.cornell.edu/help. You should whitelist [email protected] in your email client to avoid the emails going into spam/junk folders.
For more information on NASS surveys and reports, call the NASS Agricultural Statistics Hotline at (800) 727-9540,
7:30 a.m. to 4:00 p.m. ET, or e-mail: [email protected].
The U.S. Department of Agriculture (USDA) prohibits discrimination against its customers, employees, and applicants for
employment on the basis of race, color, national origin, age, disability, sex, gender identity, religion, reprisal, and where
applicable, political beliefs, marital status, familial or parental status, sexual orientation, or all or part of an individual's
income is derived from any public assistance program, or protected genetic information in employment or in any program
or activity conducted or funded by the Department. (Not all prohibited bases will apply to all programs and/or
employment activities.)
If you wish to file a Civil Rights program complaint of discrimination, complete the USDA Program Discrimination
Complaint Form (PDF), found online at www.ascr.usda.gov/filing-program-discrimination-complaint-usda-customer, or
at any USDA office, or call (866) 632-9992 to request the form. You may also write a letter containing all of the
information requested in the form. Send your completed complaint form or letter to us by mail at U.S. Department of
Agriculture, Director, Office of Adjudication, 1400 Independence Avenue, S.W., Washington, D.C. 20250-9410, by fax
(202) 690-7442 or email at [email protected].