Cost of Pollination Methodology and Quality Measures

Cost of Pollination Methodology and
Quality Measures
ISSN: 2572-4681

Released December 21, 2017, by the National Agricultural Statistics Service (NASS), Agricultural Statistics Board, United States
Department of Agriculture (USDA).

Cost of Pollination Survey Methodology
Scope and Purpose: The Cost of Pollination survey, conducted annually in all 50 states, collects information on acreage
pollinated, colonies used, and dollars spent for a variety of different crops.
Survey Timeline: The Cost of Pollination survey began collecting data for the 2017 production year in October 2017 and
concluded December 2017.
Sampling: The target population for Cost of Pollination estimation program is all farms and ranches with at least one acre
of a crop determined to be potentially pollinated by honey bees. There were 34 specific crops, identified to use honey bee
pollination, targeted in the Cost of Pollination sampling scheme, but additional crops were allowed to be reported in the
“All Other Crops” field on the questionnaire.
The Cost of Pollination samples were selected using a Multivariate Probability Proportional to Size (MPPS) sampling
scheme. Each record was assigned a measure of size based on list frame data for multiple specified commodities. Nonresponse groupings were formed based on each records probability of selection and previous pollination history.
The 2017 sample size was 14,532 and the 2016 sample size was 19,931. Information provided in the 2016 survey was
leveraged to reduce the sample size significantly in 2017 by targeting records with higher probability of paid pollination.
Data Collection: All federal data collections require approval by the Office of Management and Budget (OMB). NASS
must document the public need for the data, show the design applies sound statistical practice, ensure the data does not
already exist elsewhere, and show that the public is not excessively burdened. The Cost of Pollination questionnaires must
display an active OMB number that gives NASS the authority to conduct the survey, a statement of the purpose of the
survey and the use of the data being collected, a response burden statement that gives an estimate of the time required to
complete the form, a confidentiality statement that the respondent’s information will be protected from disclosure, and a
statement that response to the survey is voluntary and not required by law.
All sampled operations were mailed a questionnaire and given adequate time to respond by mail or electronic data
reporting (EDR). Those that did not respond by mail or EDR were telephoned or enumerated in person.
Survey Edit: As survey data were collected and captured, they were edited for consistency and reasonableness using
automated systems. Reported data were first edited as a “batch” of data when first captured. The edit logic ensures
administrative coding follows the methodological rules associated with the survey design. Relationships between data
items on the current survey were verified. The edit determined the status of each record to be either “dirty” or “clean”.
Dirty records were either updated or certified by an analyst to be accurate. Corrected data were reedited interactively.
Only clean records were eligible for analysis tools and summary.
Analysis Tool: Edited data were processed through an interactive analysis tool that displays data for all reports by
questionnaire item. The tool provided various scatter plots, tables, charts, and special tabulations that allowed the analyst
to compare an individual record to other similar records within their state and region. These tools made outliers and
unusual data relationships evident and Regional Field Office and Headquarters staff reviewed them to determine if they
were correct. Suspect data found to be in error were corrected, while data found to be correct were kept.

Non-sampling Errors: Non-sampling errors are present in any survey process. These errors include reporting, recording,
editing, and imputation errors. Steps were taken to minimize the impact of these errors, such as questionnaire testing,
comprehensive interviewer training, validation, and verification of processing systems, detailed computer edits, and the
analysis tool.
Non-response Adjustment: Some producers refused to participate in the survey, others could not be located during the
data collection period, and some submitted incomplete reports. These non-respondents were accounted for in order to
make accurate estimates.
Estimators: Point estimates, called direct expansions, are calculated by multiplying the reported value by the nonresponse-adjusted weight and summing to a non-response grouping total. A variance estimate is also computed for each
non-response grouping. Totals and variances are additive across non-response groupings to form a state estimate and
states are additive to regional estimates.
Ratio estimates are also computed for many items. For example, dollars per acre values are calculated as the ratio of total
dollars paid to acres paid for pollination. Both the numerator and denominator must be usable in order for that record to be
used in the ratio estimator.
Estimation: Estimates were prepared by the Agricultural Statistics Board after reviewing recommendations and analysis
submitted by each Regional Field Office. All data were analyzed for unusual values. Data from each operation were
compared to their own past operating profile and to trends from similar operations. Data for missing operations were
covered by weighting positive data of similar operations based on location and non-response grouping. National and State
survey data were reviewed for reasonableness with each other, estimates from the previous year, and other USDA, NASS
reports. In order to be published individually, a crop must have an appropriate threshold of paid pollinated acres in a
region and meet USDA, NASS's confidentiality policy. If a crop did not meet either of these requirements, it was
combined with all other unpublished crops under the “All Other” heading. Due to the differences in regions and years, the
aggregate and other published estimates may include different crops. Due to the differences in regions and years, the
aggregate and other published estimates may include different crops.
Estimation Regions: To improve the reliability and increase the number of estimates which could be published, estimates
were published at a regional level, based on the regions used for the 2012 Census of Agriculture. Regions 6 and 7 were
combined. The states in each region were as follows:
Region 1:

Connecticut, Illinois, Indiana, Iowa, Kansas, Massachusetts, Maine, Michigan, Nebraska, New
Hampshire, New Jersey, New York, Ohio, Pennsylvania, Rhode Island, Vermont, Wisconsin.

Region 2:

Alabama, Delaware, Georgia, Kentucky, Maryland, North Carolina, South Carolina, Tennessee,
Virginia, West Virginia.

Region 3:

Arkansas, Florida, Louisiana, Missouri, Mississippi, New Mexico, Oklahoma, Texas.

Region 4:

Colorado, Minnesota, Montana, Nevada, North Dakota, South Dakota, Utah, Wyoming.

Region 5:

Alaska, Idaho, Oregon, Washington.

Region 6 & 7:

Arizona, California, Hawaii.

2

Cost of Pollination Methodology and Quality Measures (December 2017)
USDA, National Agricultural Statistics Service

Quality Metrics for Cost of Pollination Statistics
Purpose and Definitions: Under the guidance of the Statistical Policy Office of the Office of Management and Budget
(OMB), the United States Department of Agriculture’s National Agricultural Statistics Service (NASS) provides data
users with quality metrics for its published data series. The metrics tables below describe the performance data for the
survey contributing to the publication. The accuracy of data products may be evaluated through sampling and nonsampling error. The measurement of error due to sampling in the current period is evaluated by the coefficient of variation
for each estimated item. Non-sampling error is evaluated by response rates and the percent of the estimate from
respondents.
Sample size is the number of observations selected from the population to represent a characteristic of the
population. For Cost of Pollination, this number reflects operations with list frame acreage of targeted crops.
Response rate is the proportion of the sample that completed the survey, excluding those operations that did not
have the item of interest or were out of business at the time of data collection. This calculation follows Guideline
3.2.2 of the OMB Standards and Guidelines for Statistical Surveys (September 2006).
Cooperation rate is the proportion of the sample that completed the survey. This includes records that did not have
the item of interest or were out of business at the time of data collection.
Coefficient of variation (CV) is a measure of the relative amount of error associated with a sample estimate.
Specifically, it is the standard error of a point estimate divided by that estimate, generally multiplied times 100 so
that it can be reported as a percentage. This relative measure allows the reliability of a range of estimates to be
compared. For example, the standard error is often larger for large population estimates than for small population
estimates, but the large population estimates may have a smaller CV, indicating a more reliable estimate. Selected
estimates on acreage paid for pollination and colonies used for pollination have CVs published on the USDA, NASS
Quick Stats system: www.nass.usda.gov/Quick_Stats/
o
o
o

High Reliability Estimate. CV less than 15 percent.
Medium Reliability Estimate. CV between 15 percent and 29.9 percent.
Low Reliability Estimate. CV 30 percent or higher. Caution should be used when using this estimate in any
form. Please consult NASS for more information or guidance.

Cost of Pollination Survey Sample Size, Response Rate, and Cooperation Rate – United States:
2016 and 2017
Region 1

Sample size
2016

Response rate
2017

(number)

2016

(number)

Cooperation rate
2017

(percent)

2016

(percent)

2017

(percent)

(percent)

1 ............................
2 ............................
3 ............................
4 ............................
5 ............................
6 and 7 ..................

6,256
2,819
2,697
999
2,505
4,655

5,146
2,175
2,227
656
1,806
2,522

54.7
45.0
39.7
47.4
61.5
49.5

54.3
47.4
40.5
51.8
58.3
48.7

69.8
65.2
63.0
67.2
72.7
62.8

69.5
66.0
67.4
71.2
65.9
62.1

United States .........

19,931

14,532

50.8

51.0

66.8

67.0

1

See Estimation Regions on page 2.

Cost of Pollination Methodology and Quality Measures (December 2017)
USDA, National Agricultural Statistics Service

3

Information Contacts
Process
Estimation .................................
Data Collection .........................
Questionnaires ..........................
Sampling and Editing ................
Summary and Estimators ..........
Dissemination ...........................
Media Contact and Webmaster .

Unit
Livestock Branch
Survey Administration Branch
Data Collection Branch
Sampling Editing and Imputation Methodology Branch
Summary Estimation and Disclosure Methodology Branch
Data Dissemination Office
Public Affairs Office

Telephone
(202) 720-3570
(202) 720-3895
(202) 720-6201
(202) 690-8141
(202) 720-2248
(202) 720-3400
(202) 720-2639

Email
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]

Access to NASS Reports
For your convenience, you may access NASS reports and products the following ways:
 All reports are available electronically, at no cost, on the NASS web site: www.nass.usda.gov
 Both national and state specific reports are available via a free e-mail subscription. To set-up this free
subscription, visit www.nass.usda.gov and click on “National” or “State” in upper right corner above “search”
box to create an account and select the reports you would like to receive.
For more information on NASS surveys and reports, call the NASS Agricultural Statistics Hotline at (800) 727-9540,
7:30 a.m. to 4:00 p.m. ET, or e-mail: [email protected].

The U.S. Department of Agriculture (USDA) prohibits discrimination against its customers, employees, and
applicants for employment on the basis of race, color, national origin, age, disability, sex, gender identity,
religion, reprisal, and where applicable, political beliefs, marital status, familial or parental status, sexual
orientation, or all or part of an individual's income is derived from any public assistance program, or protected
genetic information in employment or in any program or activity conducted or funded by the Department. (Not
all prohibited bases will apply to all programs and/or employment activities.)
If you wish to file a Civil Rights program complaint of discrimination, complete the USDA Program
Discrimination Complaint Form (PDF), found online at http://www.ascr.usda.gov/complaint_filing_cust.html,
or at any USDA office, or call (866) 632-9992 to request the form. You may also write a letter containing all of
the information requested in the form. Send your completed complaint form or letter to us by mail at U.S.
Department of Agriculture, Director, Office of Adjudication, 1400 Independence Avenue, S.W., Washington,
D.C. 20250-9410, by fax (202) 690-7442 or email at [email protected].