Honey Methodology and Quality Measures

Honey Methodology and Quality
Measures
ISSN: 2326-0378

Released March 18, 2013, by the National Agricultural Statistics Service (NASS), Agricultural Statistics Board, United States Department of
Agriculture (USDA).

Annual Bee and Honey Inquiry Methodology
Scope and Purpose: The Bee and Honey Inquiry is conducted annually in all States. The population consists of all known
producers with five or more colonies. Data are collected on the following: current year honey production, total colonies,
colonies harvested, current year quantity, and value of sales for honey produced in the current year and in the previous
year, and honey stocks as of December 15. Stocks exclude those held under the commodity loan program.
Data are summarized for all operations reporting five or more colonies harvested. If honey stocks or colonies were present
in more than one state, each state is counted. Estimates are published for honey producing colonies, yield per colony,
production, stocks on December 15, average price per pound, and value of production at a National and a State level. The
publication contains combined estimates for ten States with low levels of production labeled as “Other States”. Estimates
for honey prices by color class and marketing channel are also included in the publication.
Survey Timeline: For the Bee and Honey Inquiry data collection begins in December using the current year as the
reference period. Honey Stocks are recorded for a December 15 reference date. Questionnaires are mailed in early
December. Telephone follow-up is conducted for nonresponse records beginning in late January. Enumerators at NASS’s
National Operations Center (NOC) collect data for a period of approximately one month. Assigned Field Offices have
approximately two weeks from the end of data collection in February to complete editing and analysis, execute the
summary, interpret the survey results, and submit recommendations for estimates. Over the next three weeks a National
review is completed and National estimates are established. Estimates are released to the public in March as specified by
the Agricultural Statistics Board on its annual publications calendar.
Sampling: The target population for the Bee and Honey Inquiry is all agricultural operations with five or more total
colonies of bees. A profile of each operation is maintained on the NASS List Sampling Frame (hereafter List Frame) to
allow NASS to identify operations eligible for sampling. For bee and honey operations the List Frame is a current and
unduplicated list of agricultural operations, and all current bee and honey operations are assumed to be on the list. Survey
population definition and sampling are completed annually. Each bee and honey operations is classified into one of
several mutually exclusive strata based on honey production, presence of operation in multiple States, and percent of State
total colonies.
If an unlisted apiary is identified at any time, the operation is added to the List Frame. The survey sample is drawn
annually; hence, apiaries discovered after the sample has been drawn may be ineligible to be part of the survey sample
until the following year.
Data Collection: All federal data collections require approval by the Office of Management and Budget (OMB). NASS
must document the public need for the data, show the design applies sound statistical practice, and ensure the data do not
already exist elsewhere and that the public is not excessively burdened. The Bee and Honey Inquiry questionnaire must
display an active OMB number that gives NASS the authority to conduct the survey, a statement of the purpose of the
survey and the use of the data being collected, a response burden statement that gives an estimate of the time required to
complete the form, a confidentiality statement that the respondent’s information will only be used for statistical purposes
in combination with other producers, and a statement saying that response to the survey is voluntary and not required by
law.

All States use the same, standardized questionnaire for data collection. For consistency across modes, the paper version is
considered the master questionnaire and the web and Computer Assisted Telephone Interview (CATI) instruments are
built to model the paper instrument. Questionnaire content and format are evaluated annually through a specifications
process where requests for changes are evaluated and approved or disapproved.
Sampled operations receive a pre-survey letter explaining the purpose and importance of the survey and that they are
being contacted for survey purposes only. Attached to the letter is a complete copy of the paper questionnaire and a pass
code, which can be used to complete the survey securely online. States are responsible for their data collection strategy,
but States must include provisions for respondents to report securely online. If response is not received by mail or online,
respondents are contacted by computer-assisted telephone interviews (CATI). Limited personal interviewing is conducted,
generally for large operations or those with special handling arrangements.
Survey Edit: As survey data are collected and captured, they are edited for consistency and reasonableness using
automated systems. The edit logic ensures administrative coding follows the methodological rules associated with the
survey design. Relationships between data items on the current survey are verified. Some data items in the current survey
are compared to data items from earlier surveys to ensure certain relationships are logical. Records that fail edit
requirements must be updated or certified by an analyst to be exempt from the failed edit requirement. Only records that
pass edit requirements are eligible for final summary.
During the data edit operations reporting colonies in more than one state must have a special review completed. A
reconciliation review is completed between the sending and receiving States. All States involved must agree upon the
allocation of honey amongst states. This reconciliation procedure eliminates duplication and ensures that the honey is
recorded in the state in which the honey was harvested or stocks were stored.
Analysis Tools: Edited data are processed through an interactive analysis tool which displays data for all reports by item.
The tool provides scatter plots, tables, charts, and special tabulations that allow the analyst to compare an individual
record to similar records. Outliers and unusual data relationships become evident, and assigned Field Office staff review
them to determine if they are correct. The tool allows comparison to an operation’s previously reported data to detect
large changes in the operation. Data found to be in error are corrected, while data found to be correct are retained.
Nonsampling Errors: Nonsampling errors are present in any survey process. These errors include reporting, recording,
and editing errors. Steps are taken to minimize these errors, such as comprehensive interviewer training, validation, and
verification of processing systems, application of detailed computer edits, and evaluation of the data via the analysis tools.
Nonresponse Adjustment: Response to the Bee and Honey Inquiry is voluntary. Some producers refuse to participate in
the survey. Others cannot be located during the data collection period and some submit incomplete reports. The
nonrespondents are accounted for in the estimation process.
Estimators: Each bee and honey operation in the sample has an initial sampling weight, which is the inverse of the
sampling fraction for the stratum to which the bee and honey operation is assigned. For the Bee and Honey Inquiry,
nonrespondents are accounted for in one of two ways. For bee and honey operations in strata made up of large and, often
unique, operations production, colonies, stocks, and production sold must be manually imputed. Nonrespondents in all
other strata are accounted for by adjusting the weights of the usable responses to account for responses that are not usable.
The adjustment occurs at the stratum level for all strata that represent bounded homogenous groupings of similar sized
operations. The adjustment is performed by individual items on the questionnaire (number of colonies, honey sales) so
adjustments for item nonresponse (partial reports) and unit nonresponse (refusals and inaccessible) can be done in a single
calculation.
Point estimates, called direct expansions, are calculated by multiplying the reported value by the combined nonresponse
weight and sampling weight and summing to a stratum total. A variance estimate is also computed at the stratum level.
Totals and variances are additive across strata to form a State estimate and State estimates are additive to a National
estimate.

2

Honey Methodology and Quality Measures (March 2013)
USDA, National Agricultural Statistics Service

Ratio estimates are also computed for many items including prices. Ratio indications use the reweighted estimator
described above for the numerator and denominator direct expansions. Both the numerator and denominator must be
usable in order for that record to be used in the ratio indication.
Estimation: When all samples are accounted, all responses fully edited and the analysis material is reviewed, each
assigned Field Office executes a summary to evaluate and analyze the data under its responsibility. When all assigned
Field Offices have run summaries, headquarters receives all State files and pools the data. Since identical surveys are
conducted in each State, the samples can be pooled and National survey results computed. The summary results provide
multiple indications and their standard errors. It also provides information to assess the performance of the current survey
and evaluate the quality of survey indications, such as response rates and strata level expansions. Field Offices are
responsible for performing a detailed review of their survey results. Any irregularities revealed by the summary must be
investigated and, if necessary, resolved. Assigned Field Offices interpret the survey results and submit a recommended
estimate to Headquarters for honey production, honey producing colonies, yield, and stocks. Price and value of production
estimates are set at the state level with United States estimates summed from each State. NASS assembles a panel of
statisticians to serve as the Agricultural Statistics Board (ASB), which reviews the National results and establishes the
National estimates. The survey indications used in State summaries are produced by the National summary. ASB follows
the same approach the assigned Field Offices do in determining the National estimate.
Previous year’s estimates are subject to revision when current year’s estimates are set. Revisions are the result of late
reports or corrected data. Price revisions can result from additional sales reported the following year. Every five years,
NASS conducts the Census of Agriculture, which is an exhaustive data collection effort for all known farm operations
across the United States. Estimates are thoroughly reviewed for possible revision after data from the five-year Census of
Agriculture are available. The information gathered from the Census of Agriculture is used to establish “bench mark”
levels by which the survey estimators can be compared and bias determined.

Quality Metrics for Honey
Purpose and Definitions: Under the guidance of the Statistical Policy Office of the Office of Management and Budget
(OMB), the United States Department of Agriculture’s National Agricultural Statistics Service (NASS) provides data
users with quality metrics for its published data series. The metrics table below describe the performance data for all
surveys contributing to the publication. The accuracy of data products may be evaluated through sampling and nonsampling error. The measurement of error due to sampling in the current period is irrelevant for a fully enumerated data
series. Non-sampling error is evaluated by response rates and the percent of the estimate from reported data.
Sample size is the number of observations selected from the population to represent a characteristic of the
population.
Response rate is the proportion of the sample that responds to the survey.
Bee and Honey Inquiry Sample Size and Response Rates: To assist in evaluating the performance of the estimates in
the honey report, the sample size and response rates are displayed. Response rates overall for 2011 and 2012 are
displayed.

Honey Methodology and Quality Measures (March 2013)
USDA, National Agricultural Statistics Service

3

Honey Survey Sample Size and Response Rates – United States: 2011-2012
2011
Sample size

2012
Response rate

(number)

Sample size

(percent)

Response rate

(number)

(percent)

Alabama .......................................
Arizona .........................................
Arkansas .......................................
California ......................................
Colorado .......................................
Florida ...........................................
Georgia .........................................
Hawaii ...........................................
Idaho .............................................
Illinois ............................................

161
42
102
418
77
388
163
44
64
161

78.3
71.4
68.6
73.9
67.5
65.7
71.8
56.8
75.0
78.9

189
40
109
413
87
666
166
42
61
190

80.4
72.5
76.1
71.2
80.5
70.4
82.5
78.6
82.0
76.3

Indiana ..........................................
Iowa ..............................................
Kansas ..........................................
Kentucky .......................................
Louisiana ......................................
Maine ............................................
Michigan .......................................
Minnesota .....................................
Mississippi ....................................
Missouri ........................................

139
139
66
210
66
33
240
177
80
121

73.4
74.1
77.3
74.3
68.2
69.7
77.9
76.3
67.5
78.5

157
169
71
208
72
35
259
206
85
120

77.7
78.7
80.3
76.0
61.1
80.0
77.6
81.1
63.5
74.2

Montana ........................................
Nebraska ......................................
New Jersey ...................................
New Mexico ..................................
New York ......................................
North Carolina ...............................
North Dakota .................................
Ohio ..............................................
Oregon ..........................................
Pennsylvania ................................

86
66
72
25
199
342
140
406
118
371

74.4
75.8
81.9
72.0
75.9
79.8
75.7
71.4
84.7
73.3

98
70
91
24
205
392
142
532
119
458

81.6
88.6
75.8
70.8
76.6
81.4
71.8
71.6
82.4
81.0

South Dakota ................................
Tennessee ....................................
Texas ............................................
Utah ..............................................
Vermont ........................................
Virginia ..........................................
Washington ...................................
West Virginia .................................
Wisconsin .....................................
Wyoming .......................................

106
268
170
47
36
151
149
152
220
36

73.6
78.4
74.7
80.9
80.6
75.5
77.2
80.9
82.7
80.6

110
308
183
55
34
163
183
245
224
38

73.6
81.5
74.3
65.5
85.3
70.6
72.7
82.9
82.1
86.8

Other States ..................................

441

73.5

547

76.1

United States ................................

6,492

74.9

7,566

76.5

4

Honey Methodology and Quality Measures (March 2013)
USDA, National Agricultural Statistics Service

Information Contacts
Process
Estimation .............................................
Data Collection .....................................
Questionnaires and Editing ...................
Sampling ..............................................
Analysis and Estimators ........................
Dissemination and Webmaster .............
Media Contact ......................................

Unit
Livestock Branch
Survey Administration Branch
Editing and Questionnaire Branch
Survey Sampling Branch
Statistical Methods Branch
Data Dissimination Section
Public Affairs Section

Telephone
(202) 720-3570
(202) 720-3895
(202) 720-6201
(202) 720-3895
(202) 720-4008
(202) 720-7017
(202) 720-7017

Email
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]

Access to NASS Reports
For your convenience, you may access NASS reports and products the following ways:
 All reports are available electronically, at no cost, on the NASS web site: http://www.nass.usda.gov
 Both national and state specific reports are available via a free e-mail subscription. To set-up this free
subscription, visit http://www.nass.usda.gov and in the “Follow NASS” box under “Receive reports by
Email,” click on “National” or “State” to select the reports you would like to receive.
For more information on NASS surveys and reports, call the NASS Agricultural Statistics Hotline at (800) 727-9540,
7:30 a.m. to 4:00 p.m. ET, or e-mail: [email protected].
The United States Department of Agriculture (USDA) prohibits discrimination in all its programs and activities on the
basis of race, color, national origin, age, disability, and where applicable, sex, marital status, familial status, parental
status, religion, sexual orientation, political beliefs, genetic information, reprisal, or because all or a part of an individual's
income is derived from any public assistance program. (Not all prohibited bases apply to all programs.) Persons with
disabilities who require alternative means for communication of program information (Braille, large print, audiotape, etc.)
should contact USDA's TARGET Center at (202) 720-2600 (voice and TDD).
To file a complaint of discrimination, write to USDA, Assistant Secretary for Civil Rights, Office of the Assistant
Secretary for Civil Rights, 1400 Independence Avenue, S.W., Stop 9410, Washington, DC 20250-9410, or call toll-free at
(866) 632-9992 (English) or (800) 877-8339 (TDD) or (866) 377-8642 (English Federal-relay) or (800) 845-6136
(Spanish Federal-relay). USDA is an equal opportunity provider and employer.