Cost of Pollination Methodology and Quality Measures

Cost of Pollination Methodology and
Quality Measures
ISSN: 2572-4681

Released January 12, 2024, by the National Agricultural Statistics Service (NASS), Agricultural Statistics Board, United States Department
of Agriculture (USDA).

Cost of Pollination Survey Methodology
Scope and Purpose: The Cost of Pollination survey, conducted annually in all 50 states, collects information on acreage
pollinated, colonies used, and dollars spent for a variety of crops.
Survey Timeline: For the Cost of Pollination survey, data collection begins in October and concludes in December.
Estimates are released to the public in December on the date designated by the Agricultural Statistics Board (ASB) on its
annual publications calendar.
Sampling: The target population for Cost of Pollination estimation program is all farms and ranches with at least one acre
of a crop determined to be potentially pollinated by honey bees. There were 33 specific crops, identified to use honey bee
pollination, targeted in the Cost of Pollination sampling scheme. Additional crops were allowed to be reported in the “All
Other Crops” item code on the questionnaire.
The Cost of Pollination samples were selected using a Multivariate Probability Proportional to Size (MPPS) sampling
scheme. Each record was assigned a measure of size based on list frame data for multiple specified commodities.
Nonresponse groupings were formed based on each records probability of selection and previous pollination history. The
2023 sample size was 15,548 and the 2022 sample size was 15,590.
Data Collection: All federal data collections require approval by the Office of Management and Budget (OMB). NASS
must document the public need for the data, apply sound statistical practice, prove the data does not already exist
elsewhere, and ensure the public is not excessively burdened. The questionnaire must display an active OMB number that
gives NASS the authority to conduct the survey, a statement of the purpose of the survey and the use of the data being
collected, a response burden statement that gives an estimate of the time required to complete the form, a confidentiality
statement that the respondent’s information will only be used for statistical purposes in combination with other producers,
and a statement saying that response to the survey is voluntary and not required by law.
All Regional Field Offices (RFO) use the same standardized questionnaire for data collection. For consistency across
modes, the paper version is considered the master questionnaire and the Computer Assisted Self Interview (CASI), mobile
Computer Assisted Telephone Interview (mCATI), and Computer Assisted Telephone Interview (CATI) instruments are
built to model the paper questionnaire. The questionnaire content and format are evaluated annually through a
specification process where requests for changes are evaluated and approved or disapproved. Input may vary from
question wording or formatting to a program change involving the deletion or modification of current questions or
addition of new ones. If there are significant changes to either the content or format proposed, a NASS survey
methodologist will pre-test the changes for usability. Prior to the start of data collection, all modes of instruments are
reviewed, and CASI, mCATI, and CATI instruments are thoroughly tested.
Sampled operations receive a pre-survey letter explaining the purpose and importance of the survey and that they are
being contacted for survey purposes only. Attached to the letter is a complete copy of the paper questionnaire and a pass
code, which can be used to complete the survey securely online. RFOs are responsible for their data collection strategy,
but RFOs must include provisions for respondents to report securely online. If response is not received by mail or online,
respondents are contacted by CATI. Limited personal interviewing may be conducted, generally for large operations or
those with special handling arrangements.

Survey Edit: As survey data are collected and captured, they are edited for consistency and reasonableness using
automated systems. Relationships between data items on the current survey are verified and in certain situations those
items may be compared to data from earlier surveys to make sure certain relationships are logical. The edit will determine
the status of each record to be either “dirty” or “clean”. Dirty records must be updated and reedited or certified by an
analyst to be clean. If updates are needed, they are reedited interactively. Only clean records are eligible for analysis and
summary.
Analysis Tool: Edited data are processed through an interactive analysis tool which displays data for all reports by item.
The tool provides scatterplots, tables, charts, and special tabulations that allow the analyst to compare an individual record
to similar records. Outliers and unusual data relationships become evident and assigned RFO staff review them to
determine if they are correct. The tool allows comparison to an operation’s previously reported data to detect large
changes in the operation. Data found to be in error are corrected, while data found to be correct are retained.
Non-sampling Errors: Non-sampling errors are present in every survey process. These errors include reporting,
recording, and editing errors. Steps are taken to minimize these errors, such as comprehensive interviewer training,
validation, and verification of processing systems, application of detailed computer edits, and evaluation of the data via
the analysis tools.
Estimators: Response to the survey is voluntary. Some producers refuse to participate in the survey. Others cannot be
located during the data collection period, and some submit incomplete reports. The nonrespondents are accounted for in
the estimation process.
Point estimates, called direct expansions, are calculated by multiplying the reported value by the nonresponse adjusted
weight and summing to a nonresponse grouping total. The nonresponse adjustment is calculated by summing the weights
for all sample records within the group and dividing by the sum of the weights from the usable records. This adjustment
assumes that the data of the nonrespondents are similar to the data of the respondents. A variance estimate is also
computed for each nonresponse grouping. Totals and variances are additive across nonresponse groupings to form a state
estimate and states are additive to regional estimates.
Ratio estimates are also computed for many items. For example, dollars per acre values are calculated as the ratio of total
dollars paid to acres paid for pollination. Both the numerator and denominator must be usable for that record to be used in
the ratio estimator.
Estimation: Estimates were prepared by the Agricultural Statistics Board after reviewing recommendations and analysis
submitted by each Regional Field Office. All data were analyzed for unusual values. Data from each operation were
compared to their own past operating profile and to trends from similar operations. Data for missing operations were
covered by weighting positive data of similar operations based on location and nonresponse grouping. National and State
survey data were reviewed for reasonableness with each other, estimates from the previous survey cycles, and other
USDA, NASS reports. To be published individually, a crop must have an appropriate threshold of paid pollinated acres in
a region and meet USDA/NASS's confidentiality policy. If a crop did not meet either of these requirements, it was
combined with all other unpublished crops under the “All Other” heading. Due to the differences in regions and years, the
aggregate and other published estimates may include different crops.

2

Cost of Pollination Methodology and Quality Measures (January 2024)
USDA, National Agricultural Statistics Service

Estimation Regions: To improve the reliability and increase the number of estimates which could be published,
estimates were published at a regional level, based on the regions used for the 2012 Census of Agriculture. Regions 6 and
7 were combined. The states in each region were as follows:
Region 1:

Connecticut, Illinois, Indiana, Iowa, Kansas, Massachusetts, Maine, Michigan,
Nebraska, New Hampshire, New Jersey, New York, Ohio, Pennsylvania,
Rhode Island, Vermont, Wisconsin.

Region 2:

Alabama, Delaware, Georgia, Kentucky, Maryland, North Carolina,
South Carolina, Tennessee, Virginia, West Virginia.

Region 3:

Arkansas, Florida, Louisiana, Missouri, Mississippi, New Mexico, Oklahoma,
Texas.

Region 4:

Colorado, Minnesota, Montana, Nevada, North Dakota, South Dakota, Utah,
Wyoming.

Region 5:

Alaska, Idaho, Oregon, Washington.

Region 6 & 7:

Arizona, California, Hawaii.

Cost of Pollination Methodology and Quality Measures (January 2024)
USDA, National Agricultural Statistics Service

3

Quality Metrics for Cost of Pollination Statistics
Purpose and Definitions: Under the guidance of the Statistical Policy Office of the Office of Management and Budget
(OMB), the United States Department of Agriculture’s National Agricultural Statistics Service (NASS) provides data
users with quality metrics for its published data series. The metrics tables below describe the performance data for the
survey contributing to the publication. The accuracy of data products may be evaluated through sampling and non sampling error. The measurement of error due to sampling in the current period is evaluated by the coefficient of variation
for each estimated item. Non-sampling error is evaluated by response rates and the weighted item response rates.
Sample size is the number of observations selected from the population to represent a characteristic of the
population. For Cost of Pollination, this number reflects operations with list frame acreage of targeted crops that
were selected for the survey.
Response rate is the proportion of the sample that completed the survey, excluding those operations that did not
have the item of interest or were out of business at the time of data collection. This calculation follows
Guideline 3.2.2 of the OMB Standards and Guidelines for Statistical Surveys (September 2006).
Weighted item response rate is a ratio of reported survey data expanded by the original sampling weight compared
to final nonresponse adjusted summary totals.
Coefficient of variation (CV) provides a measure of the size for the standard error relative to the point estimate and
is used to measure the precision of the results of a survey estimator. Specifically, it is the standard error of a point
estimate divided by that estimate, generally multiplied times 100 so that it can be reported as a percentage. This
relative measure allows the reliability of a range of estimates to be compared. For example, the standard error is
often larger for large population estimates than for small population estimates, but the large population estimates
may have a smaller CV, indicating a more reliable estimate. Selected estimates on acreage paid for pollination and
colonies used for pollination have CVs published on the USDA, NASS Quick Stats system:
www.nass.usda.gov/Quick_Stats/
o
o
o

High Reliability Estimate. CV less than 15 percent.
Medium Reliability Estimate. CV between 15 percent and 29.9 percent.
Low Reliability Estimate. CV 30 percent or higher. Caution should be used when using this estimate in any
form. Please consult NASS for more information or guidance.

Cost of Pollination Survey Sample Size, Response Rate, and Weighted Item Response Rate –
United States: 2022 and 2023
Sample size

Region 1

Colonies Used for Pollination

Response rate

Weighted Item Response Rate

2022

2023

2022

2023

2022

2023

(number)

(number)

(percent)

(percent)

(percent)

(percent)

1 ...............................
2 ...............................
3 ...............................
4 ...............................
5 ...............................
6 and 7 ....................

5,812
2,833
1,530
667
1,905
2,843

5,615
2,688
1,397
662
1,983
3,203

46.3
31.5
25.7
48.5
53.1
42.5

41.6
35.8
28.0
48.9
41.2
41.8

67.2
58.8
46.2
77.7
54.6
52.5

54.6
69.3
63.0
46.2
42.0
50.4

United States ..........

15,590

15,548

42.3

40.0

(NA)

(NA)

(NA) Not available.
1
See Estimation Regions on page 3.

4

Cost of Pollination Methodology and Quality Measures (January 2024)
USDA, National Agricultural Statistics Service

Information Contacts
Process
Estimation ...................................
Data Collection ...........................
Questionnaires ...........................
Sampling and Editing .................
Summary and Estimators ..........
Dissemination .............................
Media Contact and Webmaster .

Unit
Livestock Branch
Survey Administration Branch
Data Collection Branch
Sampling Editing and Imputation Methodology Branch
Summary Estimation and Disclosure Methodology Branch
Data Dissemination Office
Public Affairs Office

Telephone
(202) 720-3570
(202) 720-3895
(202) 720-6201
(202) 690-8141
(202) 690-8141
(202) 720-3400
(202) 720-2639

Email
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]

Access to NASS Reports
For your convenience, you may access NASS reports and products the following ways:
➢ All reports are available electronically, at no cost, on the NASS web site: www.nass.usda.gov.
➢ Both national and state specific reports are available via a free e-mail subscription. To set-up this free
subscription, visit www.nass.usda.gov and click on “National” or “State” in upper right corner above “search”
box to create an account and select the reports you would like to receive.
➢ Cornell’s Mann Library has launched a new website housing NASS’s and other agency’s archived reports. The
new website, https://usda.library.cornell.edu. All email subscriptions containing reports will be sent from the new
website, https://usda.library.cornell.edu. To continue receiving the reports via e-mail, you will have to go to the
new website, create a new account and re-subscribe to the reports. If you need instructions to set up an account or
subscribe, they are located at: https://usda.library.cornell.edu/help. You should whitelist [email protected] in your email client to avoid the emails going into spam/junk folders.
For more information on NASS surveys and reports, call the NASS Agricultural Statistics Hotline at (800) 727-9540,
7:30 a.m. to 4:00 p.m. ET, or e-mail: [email protected].
The U.S. Department of Agriculture (USDA) prohibits discrimination against its customers, employees, and applicants for
employment on the basis of race, color, national origin, age, disability, sex, gender identity, religion, reprisal, and where
applicable, political beliefs, marital status, familial or parental status, sexual orientation, or all or part of an individual's
income is derived from any public assistance program, or protected genetic information in employment or in any program
or activity conducted or funded by the Department. (Not all prohibited bases will apply to all programs and/or
employment activities.)
If you wish to file a Civil Rights program complaint of discrimination, complete the USDA Program Discrimination
Complaint Form (PDF), found online at www.ascr.usda.gov/filing-program-discrimination-complaint-usda-customer, or
at any USDA office, or call (866) 632-9992 to request the form. You may also write a letter containing all of the
information requested in the form. Send your completed complaint form or letter to us by mail at U.S. Department of
Agriculture, Director, Office of Adjudication, 1400 Independence Avenue, S.W., Washington, D.C. 20250-9410, by fax
(202) 690-7442 or email at [email protected].