Farm Labor Methodology and Quality Measures

Farm Labor Methodology and Quality
Measures
ISSN: 2167-1893

Released November 20, 2014, by the National Agricultural Statistics Service (NASS), Agricultural Statistics Board, United States
Department of Agriculture (USDA).

Farm Labor Survey Methodology
Scope and Purpose: The NASS Farm Labor Survey provides the basis for employment and wage estimates for all
workers directly hired by U.S. farms and ranches (excluding Alaska), for each of four quarterly reference weeks. Selected
annual average estimates are summarized from the associated quarterly estimates.
The Farm Labor Survey is conducted in cooperation with the Department of Labor. The employment and wage estimates
published support USDA and Department of Labor programs, and are used by additional federal, state, and local
government agencies, educational institutions, farm organizations, and private sector employers of farm labor.
The NASS Farm Labor Survey is conducted semi-annually in April and October, in all surveyed states except California.
During the April data collection, data for both January and April reference weeks are collected. During the October data
collection, data for both July and October reference weeks are collected. The quarterly reference week is the Sunday to
Saturday period which includes the 12th day of the month. For California, collection of these data is currently conducted
on a quarterly basis, as part of the California Employment Development Department (EDD) monthly labor program.
Estimates published include number of hired workers during each quarterly reference week, the average hours worked,
and average wage rates by type of worker. Estimates are published for the U.S., each of 15 multi-state labor regions, and
the single-state regions of California, Florida, and Hawaii.
Survey Timeline: Data collection begins the week following the April or October reference week and progresses through
the end of the month, for all surveyed states except California. In California, data collection begins shortly after the
reference week in each of the four surveyed quarters and typically extends three to four weeks beyond the end of the
month. NASS Regional Field Offices (RFOs) have about four or five business days following the NASS semi-annual data
collection period, to complete editing and analysis, execute the summary, and interpret the survey results at the state level
for each member state. The Agricultural Statistics Board performs a national review, reconciles state estimates to regional
and national estimates, and prepares the official estimates for release in five to eight business days. Official Farm Labor
Survey estimates are published in May (for the January and April quarters) and November (for the July and October
quarters, and annual average estimates).
Sampling: The target population for the Farm Labor Survey program is all farms and ranches with $1,000 or more in
agricultural sales (or potential sales), excluding Alaska farms. NASS uses a dual frame approach, consisting of list frame
and area frame components, to provide complete coverage of this target population.
The list frame includes all known agricultural establishments. A profile, called control data, of each establishment is
maintained on the list frame to allow NASS to define list frame sampling populations for specific surveys and to employ
efficient sampling designs. The primary control datum for farm labor is the peak number of workers value, the most
recently reported annual peak number of hired workers for each record. List frame records with positive peak number of
worker control data are included in the list frame farm labor population. The California list frame labor population defined
by positive peak number of workers alone is sufficiently complete, due to the collaboration with the California EDD. For
all other states, records without peak number of worker control data that have a calculated farm value of sales of at least
$10,000, many of which are expected to employ agricultural workers, are also part of the list frame farm labor population.
In total, the list frame farm labor population includes approximately 1.3 million U.S. farms and ranches.

The area frame contains all land in the U.S. (except Alaska) and is therefore complete for the Farm Labor Survey
program. For all states, land is stratified according to intensity of agriculture using satellite imagery. The land in each
stratum is divided into segments of roughly one square mile. Segments are optimally allocated and sampled to effectively
measure crops and livestock. The sampled segments are fully enumerated each June, in all states except Hawaii. All farms
and ranches found operating tracts in enumerated segments are checked to see if they are included in the list frame farm
labor population. The farms and ranches that are not included in the list frame labor population, called nonoverlap tracts,
are eligible for the farm labor nonoverlap sample.
The farm labor list frame sample is selected using a hierarchical stratified sampling design with strata defined by peak
number of farm workers or calculated farm value of sales. The sample is a panel sample and is designed to achieve a U.S.
level coefficient of variation of four percent of the point estimate for all hired workers and one percent of the ratio
estimate for wage rates of all workers. The U.S. list frame sample size for the Farm Labor Survey in recent years is
approximately 10,500. Beginning with the July and October 2014 data collection, the U.S. sample consists of an
additional 1,000 to 2,000 sample units to meet data collection requirements. An additional 1,500 to 2,500 sample units are
selected to support the California EDD labor program exclusively. Each list frame sampling unit is assigned a sampling
weight which is used to create the survey estimates.
The farm labor area frame nonoverlap sample is selected using a stratified sample design based on data collected each
June. An area frame nonoverlap sample is selected for each surveyed state except California and Hawaii. The California
sample does not include a nonoverlap portion because the list frame is assumed to be complete. For Hawaii, the area
frame is excluded from sampling because this frame is not updated on an annual basis. The total farm labor area frame
nonoverlap sample, which includes the remaining surveyed states, consists of approximately 1,500 sampling units. Each
area frame nonoverlap sampling unit is assigned a sampling weight which is used to create the survey estimates.
Data Collection: Data collection proceeds with utilization of NASS data collection instruments and follow-up
procedures, in all surveyed states except California. In California, the California EDD conducts data collection utilizing
EDD-specific instruments and follow-up procedures which are similar to NASS procedures.
Data Collection for All States Except California
For consistency across modes, the paper version is considered the master questionnaire and the Computer Assisted
Telephone Interview (CATI) and web reporting instruments are built to model the paper instrument. Questionnaire
content and format are evaluated annually through a specifications process where requests for changes are evaluated and
approved or disapproved. Input may vary from question wording or formatting to a program change involving the deletion
or modification of current questions or addition of new ones. If there are significant changes to either the content or
format proposed, a NASS survey methodologist will pre-test the changes for usability. Prior to the start of data collection,
all instruments are reviewed including the CATI and web instruments, and the web-supported Computer Assisted
Personal Interview (CAPI) instrument.
All federal data collections require approval by the Office of Management and Budget (OMB). NASS must document the
public need for the data, show the design applies sound statistical practice, ensure the data do not already exist elsewhere,
and that the public is not excessively burdened. The Farm Labor Survey questionnaire must display an active OMB
number that gives NASS the authority to conduct the survey, a statement of the purpose of the survey and the use of the
data being collected, a response burden statement that gives an estimate of the time required to complete the form, a
confidentiality statement that the respondent’s information will only be used for statistical purposes in combination with
other producers, and a statement saying that response to the survey is voluntary and not required by law.
In addition to asking the specific farm labor items, all instruments collect information to verify the sampled unit,
determine any changes in the name or address, identify any partners to detect possible duplication, verify the farm still
qualifies for the target population, and identify any additional operations operated by the sampled operator.
During each semi-annual data collection period, sampled farms and ranches receive a pre-survey letter explaining the
survey and that they will be contacted for survey purposes only. The letter provides a paper copy of the questionnaire to
allow respondents to respond by mail or to prepare in advance for a follow-up interview and also provides a pass code
they can use to complete the survey on the internet. All questionnaires completed on paper are returned to the NASS
2

Farm Labor Methodology and Quality Measures (November 2014)
USDA, National Agricultural Statistics Service

National Operations Center where they are visually reviewed and key entered. All modes of data collection are utilized for
the Farm Labor Survey. While mail is the least costly mode of collection, the short data collection period and the
uncertainty of postal delivery times limit its effectiveness. Most of the data are collected by phone follow-up interviews
from NASS Data Collection Centers, using CATI. Personal interviews, conducted via CAPI, are generally reserved for
large operations or those with special handling arrangements. Data collection is coordinated for farms sampled for
multiple on-going NASS surveys.
Data Collection for California
The California EDD utilizes a mail out phase with follow-up similar to the NASS procedure. All sample units from the
NASS California labor sample receive an EDD labor questionnaire which includes the NASS Farm Labor Survey
questions as well as additional content.
Because the EDD program is monthly, collection of California labor data begins shortly after each respective quarterly
reference week, and complete datasets are received at NASS by the following month. The EDD data collection period
typically extends two to three weeks beyond the concurrent NASS April and October data collection periods. For this
reason, the final California datasets for the April and October reference weeks are not fully processed until the following
NASS semi-annual survey period.
Throughout the EDD data collection period, electronic files containing labor data are regularly transmitted securely to the
NASS Pacific RFO in California. These files are made available to the NASS editing and analysis instruments, so that all
subsequent data handling proceeds according to the NASS data analysis and estimation program.
Survey Edit: As survey data are collected and captured, data are edited for consistency and reasonableness using
automated systems. Reported data are edited as a batch of data when first captured. The edit logic ensures the coding of
administrative data follows the methodological rules associated with the survey design. Relationships between data items
(i.e. responses to individual questions) on the current survey are verified. Some data items in the current survey are
compared to data items from earlier surveys to ensure certain relationships are logical. The edit assigns a status to each
record, indicating whether or not the record passes or fails the edit requirements for consistency and reasonableness.
Records that fail edit requirements must be updated or must be certified by an analyst to be exempt from the failed edit
requirement. All records must pass edit requirements, or be certified exempt, before further analysis and summary.
Analysis Tools: Edited Farm Labor Survey data are processed and analyzed with a standard interactive data analysis tool
which displays data for all reports by item. The tool provides scatter plots, tables, charts, and special tabulations that allow
the analyst to compare record level data with previously reported data for the same record, and reported data from similar
records. Atypical responses, unusual data relationships, and statistical outliers for all labor items are revealed by the
analysis tool. RFO and NASS Headquarters (HQ) staff review such relationships to determine if they are correct. Data
found to be in error are corrected, while accepted data are retained.
Nonsampling Error: Nonsampling error is present in any survey process. This error includes reporting, recording, and
editing errors, as well as nonresponse error. Steps are taken to minimize the impact of these errors, such as questionnaire
testing, comprehensive interviewer training, validation and verification of processing systems, application of detailed
computer edits, and evaluation of the data via the analysis tool. The respondent pool is monitored and reviewed during
and after data collection, and data collection strategies modified where necessary, to continually minimize nonresponse
error.
Nonresponse Adjustment: Response to the Farm Labor Survey is voluntary. Some producers refuse to participate in the
survey, others cannot be located during the data collection period, and some submit incomplete reports. These
nonrespondents must be accounted for if accurate estimates of farm labor are to be made. For the Farm Labor Survey,
nonrespondents are accounted for by adjusting the weights of the respondents. The adjustment occurs by stratum as all
strata represent homogeneous groupings of similar sized farms. The adjustment is performed by individual data item so
adjustments for item nonresponse (partial reports) and unit nonresponse (refusals and inaccessibles) are done in a single
calculation.

Farm Labor Methodology and Quality Measures (November 2014)
USDA, National Agricultural Statistics Service

3

Estimators: Each farm and ranch in the sample has an initial sampling weight. This is the inverse of the sampling
fraction. For example, if a stratum has 1,000 farms in the population and 200 are sampled for this survey, each sampled
farm has a weight of five. In other words, each sampled farm represents five farms. The nonoverlap tracts sampled to
measure the labor not accounted for by the list have a weight determined by adjusting their original area frame weight by
any second stage sampling weight.
The Farm Labor Survey uses a “reweighted” estimator to compute direct measures of hired farm workers. The reweighted
estimator uses a global weight adjustment across all usable reports, which is the initial sampling weight adjusted for
nonresponse. Using the previous example, if 180 of the original 200 respond, the weights of the 180 will be adjusted to
1,000 divided by 180, or 5.56.
Point estimates, called direct expansions, are calculated by multiplying the reported value by the final adjusted weight and
summing to a stratum total. A variance estimate is also computed at the stratum level. The nonoverlap tracts are treated as
an additional stratum. Totals and variances are additive across strata to form state estimates.
Ratio estimates are also computed for many items. For example, wage rates are calculated as the ratio of total wages to
total hours worked. Ratio estimators use the reweighted estimator described above for the numerator and denominator
direct expansions. Both the numerator and denominator must be usable in order for that record to be used in the ratio
estimator.
Outliers: Both RFO and HQ statisticians conduct a review of worker and wage outliers, identified through the interactive
data analysis tool, to ensure the most accurate data and indications possible. The RFO statisticians review outliers for
states within their regions and the HQ statistician examines outliers across all states. Many outliers trace back to unique
situations that do not exist in the target population as much as the survey weight would indicate. In some cases, aging
control data result in misstratification, and this misstratification can also give rise to outliers. A determination is made as
to whether an adjustment to final estimates will be required for each outlier.
Estimation: The number of hired workers, average hours worked, average wage rate data, and all associated sub-items for
each surveyed reference week are summarized from the dataset. Because identical data collection instruments are used for
all states, as well as identical editing and analysis processes, state data can be summarized to regional and national survey
point estimates. For estimation purposes, point estimates are adopted as survey indications for all data items. The
summary results provide multiple direct and ratio indications for each data series being estimated. The results also provide
information used to assess the performance of the current survey and evaluate the quality of the survey indications. RFOs
interpret state level survey indications and submit state level recommendations for all member states to HQ, providing
justification in cases where recommendations deviate from survey indications, as well as other state specific comments
and justifications. HQ executes the regional and U.S. level summaries, which utilize the same estimators and produce the
same indications as the state level summaries.
The estimation process is facilitated at all levels with a second interactive analysis tool, which selectively displays current
and historic summary indications and measures of indication quality, state level recommendations and comments, and
final estimates for all published data items. The instrument generates tables and charts of this database content, allowing
statisticians to assess trends, evaluate current and historic relationships between summary indications and preliminary or
final estimates, and current and historic state level comments on local conditions and data assessment. Statisticians view
and analyze all necessary database content through the instrument, to complete the estimates. RFO statisticians review
only data and analyses relevant to their state level estimates, and HQ statisticians review all state, regional, and national
data and analyses. All entry and transmittal of preliminary and official estimates and comments is accomplished through
this data tool. Additionally, the annual average estimates are calculated using this data tool after all final quarterly
estimates are established.
For the final step in the estimation process, NASS assembles a panel of statisticians to serve as the Agricultural Statistics
Board (ASB) for the Farm Labor Survey, which reviews the U.S. level, regional, and state level summary indications and
establishes all final, official estimates for each surveyed reference week. Since larger sample sizes yield more precise
results, the ASB employs the “top-down” approach by determining the U.S. level estimates first and reconciling the state
recommendations to the U.S. and regional numbers for all estimates including number of hired workers, average hours
4

Farm Labor Methodology and Quality Measures (November 2014)
USDA, National Agricultural Statistics Service

worked, average wage rates, and associated sub-items. As part of the semi-annual process, the ASB also considers revised
California recommendations submitted after interpretation of final April or October datasets from the prior survey period,
and issues revisions of previously published California and U.S. level estimates where appropriate. Through examining
indications across states and regions, re-examining outliers and historic indications and estimates, and considering state
level recommendations and comments, the ASB is able to establish all final estimates according to both local conditions
and regional and national conditions and trends. After all final quarterly estimates are established, the ASB verifies the
annual average estimates for the November publication, which are summarized weighted averages of the final U.S. and
regional level estimates for each of the four quarters.

Farm Labor Methodology and Quality Measures (November 2014)
USDA, National Agricultural Statistics Service

5

Quality Metrics for the Farm Labor Survey
Purpose and Definitions: Under the guidance of the Statistical Policy Office of the Office of Management and Budget
(OMB), the United States Department of Agriculture’s National Agricultural Statistics Service (NASS) provides data users
with quality metrics for its published data series. The metrics tables in this document describe the performance data for the
survey contributing to the publication. The accuracy of data products may be evaluated through sampling and nonsampling
error. The measurement of error due to sampling in the current period is evaluated by the coefficient of variation for each
estimated item. Nonsampling error is evaluated by response rates.
Sample size is the number of observations selected from the population to represent a characteristic of the
population. Operations that did not have the item of interest or were out of business at the time of data collection
have been excluded.
Response rate is the proportion of the sample that completed the survey. This calculation follows Guideline 3.2.2 of
the Office of Management and Budget Standards and Guidelines for Statistical Surveys (Sept. 2006).
Coefficient of variation provides a measure of the size for the standard error relative to the point estimate and is
used to measure the precision of the results of a survey estimator.

6

Farm Labor Methodology and Quality Measures (November 2014)
USDA, National Agricultural Statistics Service

Coefficient of Variation for Wage Rate by Type of Worker – United States
Reference week

Field

Livestock

Field and livestock combined

(percent)

(percent)

(percent)

October 6 - 12, 2013 .......................

1.0

1.3

0.8

October 12 - 18, 2014 .....................

0.9

1.2

0.7

July 7 - 13, 2013 ..............................

0.9

1.4

0.8

July 6 - 12, 2014 .............................

1.4

1.3

1.1

Farm Labor Sample Size and Response Rate – Region and United States: October 6-12, 2013 and
October 12-18, 2014
Region

Sample size
2013

Response rate
2014

(number)

2013

(number)

2014

(percent)

(percent)

Northeast I ..................................
Northeast II .................................

630
534

745
618

49.5
61.8

56.4
58.3

Appalachian I ..............................
Appalachian II .............................

567
638

633
745

69.5
67.7

66.4
67.9

Southeast ...................................
Florida ........................................

600
500

661
527

66.3
59.4

64.3
59.6

Lake ...........................................

840

907

58.7

54.9

Cornbelt I ....................................
Cornbelt II ...................................

886
697

943
724

60.8
62.4

55.2
59.1

Delta ...........................................

647

754

61.4

68.0

Northern Plains ...........................

942

1,017

59.4

57.8

Southern Plains ..........................

1,046

1,140

67.0

70.9

Mountain I ...................................
Mountain II ..................................
Mountain III .................................

396
322
313

460
398
367

59.6
64.6
59.2

59.6
61.8
61.6

Pacific .........................................
California ....................................

538
899

632
910

50.4
61.0

54.7
61.0

Hawaii ........................................

384

411

68.5

72.0

United States ..............................

11,379

12,592

61.5

61.5

Farm Labor Methodology and Quality Measures (November 2014)
USDA, National Agricultural Statistics Service

7

Farm Labor Sample Size and Response Rate – Region and United States: July 7-13, 2013 and
July 6-12, 2014
Region

Sample size
2013

Response rate
2014

(number)

2013

(number)

2014

(percent)

(percent)

Northeast I ..................................
Northeast II .................................

630
534

745
618

48.9
61.2

55.0
57.6

Appalachian I ..............................
Appalachian II .............................

567
638

633
745

68.3
67.4

64.8
66.2

Southeast ...................................
Florida .........................................

600
500

661
527

66.0
59.4

62.8
57.5

Lake ............................................

840

907

58.9

54.1

Cornbelt I ....................................
Cornbelt II ...................................

886
697

943
724

60.7
62.3

54.3
58.4

Delta ...........................................

647

754

60.3

66.3

Northern Plains ...........................

942

1,017

59.4

57.3

Southern Plains ...........................

1,046

1,140

66.5

68.9

Mountain I ...................................
Mountain II ..................................
Mountain III .................................

396
322
313

460
398
367

59.3
64.6
59.1

58.9
60.6
60.8

Pacific .........................................
California ....................................

538
899

632
910

50.6
49.1

53.3
65.1

Hawaii .........................................

384

411

68.2

71.3

United States ..............................

11,379

12,592

60.3

60.7

8

Farm Labor Methodology and Quality Measures (November 2014)
USDA, National Agricultural Statistics Service

Coefficient of Variation for All Hired Workers and Wage Rate – Region and United States:
October 6-12, 2013 and October 12-18, 2014
Region

All hired workers
2013

Wage rate
2014

(percent)

2013

(percent)

2014

(percent)

(percent)

Northeast I ...............................................
Northeast II ..............................................

16.7
9.1

8.3
8.5

3.1
2.9

2.6
3.2

Appalachian I ...........................................
Appalachian II ..........................................

10.8
16.4

8.7
12.5

2.4
3.9

2.6
3.7

Southeast ................................................
Florida .....................................................

11.9
9.6

10.8
11.3

2.9
2.4

2.8
2.2

Lake ........................................................

8.6

8.7

2.5

1.8

Cornbelt I .................................................
Cornbelt II ................................................

10.2
12.8

8.9
13.9

2.2
6.1

2.1
4.8

Delta ........................................................

10.6

14.7

2.3

1.9

Northern Plains ........................................

12.3

11.9

4.5

3.5

Southern Plains .......................................

12.5

10.6

2.9

2.7

Mountain I ................................................
Mountain II ...............................................
Mountain III ..............................................

13
15.7
10.9

11.1
11.7
11.8

3.3
3.3
3.0

2.7
2.8
3.6

Pacific ......................................................
California .................................................

10.7
3.8

18.7
3.8

2.8
1.3

2.3
1.0

Hawaii .....................................................

9.6

10.3

2.1

4.8

United States ...........................................

2.6

2.8

0.8

0.7

Farm Labor Methodology and Quality Measures (November 2014)
USDA, National Agricultural Statistics Service

9

Coefficient of Variation for All Hired Workers and Wage Rate – Region and United States:
July 7-13, 2013 and July 6-12, 2014
Region

All hired workers
2013

Wage rate
2014

(percent)

2013

(percent)

2014

(percent)

(percent)

Northeast I ...............................................
Northeast II ..............................................

16.1
10.1

8.9
7.5

4.0
2.6

2.6
3.3

Appalachian I ...........................................
Appalachian II ..........................................

11.4
16.3

11.6
14.6

2.6
3.5

3.0
3.6

Southeast ................................................
Florida ......................................................

10.6
10.4

9.3
12.6

2.7
3.0

2.0
2.6

Lake .........................................................

8.2

8.5

2.6

2.2

Cornbelt I .................................................
Cornbelt II ................................................

10.6
13.3

8.9
13.9

3.3
6.2

2.5
5.6

Delta ........................................................

10.8

9.3

2.4

1.9

Northern Plains ........................................

10.9

12.4

3.5

3.3

Southern Plains ........................................

16.0

10.6

4.4

3.0

Mountain I ................................................
Mountain II ...............................................
Mountain III ..............................................

17.5
15.5
11.1

12.4
11.2
17.0

3.3
4.6
3.4

2.4
2.6
4.3

Pacific ......................................................
California .................................................

9.7
4.4

15.6
4.0

2.3
1.1

3.9
1.0

Hawaii ......................................................

10.1

9.6

2.3

4.3

United States ...........................................

2.9

3.0

0.8

0.9

10

Farm Labor Methodology and Quality Measures (November 2014)
USDA, National Agricultural Statistics Service

Access to NASS Reports
For your convenience, you may access NASS reports and products the following ways:
 All reports are available electronically, at no cost, on the NASS web site: http://www.nass.usda.gov
 Both national and state specific reports are available via a free e-mail subscription. To set-up this free
subscription, visit http://www.nass.usda.gov and in the “Follow NASS” box under “Receive reports by Email,”
click on “National” or “State” to select the reports you would like to receive.
For more information on NASS surveys and reports, call the NASS Agricultural Statistics Hotline at (800) 727-9540,
7:30 a.m. to 4:00 p.m. ET, or e-mail: [email protected].

The U.S. Department of Agriculture (USDA) prohibits discrimination against its customers, employees, and applicants for
employment on the basis of race, color, national origin, age, disability, sex, gender identity, religion, reprisal, and where
applicable, political beliefs, marital status, familial or parental status, sexual orientation, or all or part of an individual's
income is derived from any public assistance program, or protected genetic information in employment or in any program
or activity conducted or funded by the Department. (Not all prohibited bases will apply to all programs and/or
employment activities.)
If you wish to file a Civil Rights program complaint of discrimination, complete the USDA Program Discrimination
Complaint Form (PDF), found online at http://www.ascr.usda.gov/complaint_filing_cust.html, or at any USDA office, or
call (866) 632-9992 to request the form. You may also write a letter containing all of the information requested in the
form. Send your completed complaint form or letter to us by mail at U.S. Department of Agriculture, Director, Office of
Adjudication, 1400 Independence Avenue, S.W., Washington, D.C. 20250-9410, by fax (202) 690-7442 or email at
[email protected].

Information Contacts
Process

Unit

Estimation ........................................ Environmental, Economics, and
Demographics Branch
Data Collection ................................ Survey Administration Branch
Questionnaires ................................ Data Collection Branch
Sampling and Editing ....................... Sampling, Editing, and Imputation
Methodology Branch
Summary and Estimators ................. Summary Estimation and Disclosure
Methodology Branch
Dissemination .................................. Data Dissemination Office
Media Contact and Webmaster ........ Public Affairs Office

Telephone

Email

(202) 720-6146
(202) 720-3895
(202) 720-6201

[email protected]
[email protected]
[email protected]

(202) 720-8671

[email protected]

(202) 720-4008
(202) 720-3869
(202) 720-2639

[email protected]
[email protected]
[email protected]