Milk Production Methodology and Quality Measures

Milk Production Methodology and
Quality Measures
ISSN: 2167-1885

Released March 10, 2023, by the National Agricultural Statistics Service (NASS), Agricultural Statistics Board, United States Department of
Agriculture (USDA).

Scope and Purpose: The Milk Production Survey, conducted in all states in January, April, July, and October, collects
data for the number of all milk cows in the herd, number of cows milked on the first day of the survey month, and total
milk produced on the first day of the survey month. In addition, milk cow replacement prices are collected each quarter
and heifer replacement prices are collected in January. In April and October, producers are asked to report the amount of
milk used on the farm for food or drink and the amount fed to calves.
Survey Timeline: The reference date for the Milk Production Survey is the 1st day of each quarter (January, April, July,
and October), with a data collection period of approximately 15 calendar days. Regional Field Offices (RFOs) may begin
data collection one day prior to the reference date. Data collection continues until a scheduled ending date, and RFOs have
four to five business days to complete editing and analysis, execute the summary, and interpret the survey results. The
National milk production statistician must review the State estimates and prepare the official estimates for release in about
four business days. Milk Production, usually released around the 18th or 19th of each month, includes monthly estimates
of milk production, milk per cow, and number of milk cows for the 24 major States. Quarterly estimates of milk
production and number of milk cows are published for all states in the January, April, July, and October Milk Production
publications. Annual milk production revisions are published for all states in February.
Sampling: The target population for the Milk Production Survey is all agricultural establishments with one or more head
of milk cows on the land operated. The Milk Production Survey is conducted in every state.
NASS uses the list frame for the Milk Production Survey. The list frame includes all known agricultural establishments.
Livestock inventory of each establishment is maintained on the list frame to allow NASS to define list frame sampling
populations for specific surveys and to employ efficient sampling designs. Only list frame records with positive milk cow
inventory data are included in the Milk Production list frame population.
The sample size for the Milk Production Survey is approximately 11,000, though some operations will be found to be out
of business or not have the item of interest. The sample is selected using a stratified sampling design with strata defined
by the total number of milk cows. The sample is first used in the January survey quarter or the “base” survey quarter,
which is the initial quarter of the survey year. The “follow-on” quarters in the survey year use the same “base”
sample. New milk operations found during the survey year can be added to the sample if the new operation qualifies for a
stratum with a probability of selection equal to 1.
Data Collection and Editing: For consistency across modes, the paper version is considered the master questionnaire and
the Computer Assisted Telephone Interview (CATI), Computed Assisted Self Interview (CASI), and Mobile Computer
Assisted Personal Interview (mCAPI) instruments are built to model the paper instrument. Questionnaire content and
format are evaluated annually through a specifications process where requests for changes are evaluated and approved or
disapproved. Input may vary from question wording or formatting to a program change involving the deletion or
modification of current questions or addition of new ones. If there are significant changes to either the content or format
proposed, a NASS survey methodologist will pre-test the changes for usability. Prior to the start of data collection, all
modes of instruments are reviewed and the paper, mCAPI, CASI, and CATI instruments are thoroughly tested.
All federal data collections require approval by the Office of Management and Budget (OMB). NASS must document the
public need for the data, apply sound statistical practice, prove the data does not already exist elsewhere, and ensure the
public is not excessively burdened. The questionnaire must display an active OMB number that gives NASS the authority
to conduct the survey, a statement of the purpose of the survey, and the use of the data being collected. Also, a response

burden statement that gives an estimate of the time required to complete the form, a confidentiality statement that the
respondent’s information will only be used for statistical purposes in combination with other producers, and a statement
saying that response to the survey is voluntary and not required by law must be on the questionnaire.
Sampled farms and ranches receive a presurvey letter explaining the survey and that they will be contacted for survey
purposes only. The letter provides the questions to be asked to allow respondents to prepare in advance and also provides
a pass code they can use to complete the survey on the internet (CASI). All modes of data collection are utilized for the
Milk Production Survey. RFOs are given the option of conducting a mail out/mail back phase. While mail is the least
costly mode of collection, the short data collection period and the uncertainty of postal delivery times li mit its
effectiveness. Most of the data are collected by CATI by individual RFOs and Data Collection Centers. A program is run
to determine if any sampled farms are in multiple on-going surveys so data collection can be coordinated.
Survey Edit: As survey data are collected and captured, they are edited for consistency and reasonableness using
automated systems. Relationships between data items on the current survey are verified and in certain situations those
items may be compared to data from earlier surveys to make sure certain relationships are logical. The edit will determine
the status of each record to be either “dirty” or “clean”. Dirty records must be updated and reedited or certified by an
analyst to be clean. If updates are needed, they are reedited interactively. Only clean records are eligible for analysis and
summary.
Analysis Tools: Edited data are processed through an interactive analysis tool which displays data for all reports by item.
The tool provides scatter plots, tables, charts, and special tabulations that allow the analyst to compare an individual
record to other similar records within their state. Outliers and unusual data relationships become evident and RFO staff
will review them to determine if they are correct. The tool also allows comparison to previously reported data to detect
large changes in the operation. Suspect data found to be in error are corrected, while data found to be correct are kept.
Nonsampling Errors: Nonsampling errors are present in any survey process. These errors include reporting, recording,
editing, and imputation errors. Steps are taken to minimize the impact of these errors, such as questionnaire testing,
comprehensive interviewer training, validation and verification of processing systems, detailed computer edits, and the
analysis tool.
Estimators: Each milk cow farm and ranch in the sample has an initial sampling weight. This weight is the inverse of the
sampling fraction. For example, if a milk cow stratum has 1,000 farms in the population and 200 are sampled for this
survey, each sampled farm has a weight of 5. In other words, each sampled farm represents 5 farms.
Response to the Milk Production Survey is voluntary. Some producers refuse to participate in the survey. Others cannot
be contacted during the data collection period, and some submit incomplete reports. These nonrespondents must be
accounted for if accurate estimates of milk are to be made. For the Milk Production Survey, nonrespondents are accounted
for by adjusting the weights of the respondents. The adjustment occurs by stratum as the bounded strata represent
homogeneous groupings of similar sized milk cow farms. The adjustment is also performed for each individual item (total
milk cows, cows milked, milk production) because sometimes only a partial report is obtained. Using the previous
example, if 160 of the original 200 respond, the weights of the 160 will be adjusted to 1,000 divided by 160, or 6.25. The
largest stratum is unbounded and is made up of operations with a large number of milk cows. Nonrespondents in the
unbounded stratum must be manually estimated by RFO statisticians, and their stratum sampling weights are not adjusted.
Two estimators are used to compute direct measures of the milk items. The “reweighted” estimator and the “adjusted”
estimator are computationally identical except in how the nonresponse adjustments are made. The reweigh ted estimator
uses a global weight adjustment across all reported and estimated reports. The nonresponse weight adjustment for the
adjusted estimator uses an additional piece of information, based on the presence/absence of milk cows. When a sampled
farm refuses to cooperate, interviewers can probe to determine the presence of milk cows even though the number of milk
cows is not known. Also, automated edit logic that is based on historic and administrative data ensures the coding of the
presence indicator value. This presence/absence indicator is used in the weight adjustment.

2

Milk Production Methodology and Quality Measures (March 2023)
USDA, National Agricultural Statistics Service

Point estimates, called direct expansions, for both estimators are calculated by multiplying the reported value by the
nonresponse weight and summing to a stratum total. A variance estimate is also computed at the stratum level. Totals and
variances are additive across strata to form a state estimate and states are additive to a National estimate.
Ratio estimates are also computed for many items. For example, milk yield is estimated as a ratio of milk production to
total milk cows. Ratio estimates use the reweighted estimator described above for the numerator and denominator. Both
the numerator and denominator must be complete for that record to be included in the ratio estimator.
Estimation: Indicators from the quarterly Milk Production Survey, along with external information (administrative data)
from various sources, provide data for estimating number of milk cows and milk production on a monthly and quarterly
basis. To be considered, these administrative data must be deemed to be reliable and come from unbiased sources. The
most common administrative data are Federal Milk Marketing Order statistics.
When all Milk Production Survey samples are accounted for, all responses fully edited, and the analysis material is
reviewed, each RFO executes a summary for their state. When all RFOs have run summaries, Headquarters executes the
National summary. Since all states conduct identical surveys, the samples can be pooled and National surv ey results
computed. The summary results provide point estimates and precision measures for each item being estimated. It also
provides information used to assess the performance of the current survey and evaluate the quality of the survey estimates,
such as strata level expansions, response rates, and percent of the expansion from usable reports.
RFOs are responsible for performing a detailed review of their survey results. Any irregularities revealed by the summary
must be investigated and, if necessary, resolved. Using the historical relationship of the survey estimates to the official
estimate, RFOs must interpret the survey results and submit recommended estimates to Headquarters for all milk cows,
milk production, and milk per cow. The data are viewed in tabular and graphical form. RFOs see their survey results only
and do not have access to other states’ results.
The National estimates are determined using the “bottom-up” approach. After the RFOs submit the recommended
estimates for each state, the National milk production statistician reviews the individual state estimates for reasonableness.
When analyzing the state estimates, the National statistician has an advantage in being able to examine results across
states and compare the state recommendations. When the review of the individual state estimates is complete, the National
estimates are determined by summing the estimates for each state. The Agricultural Statistics Board reviews the National
estimates for reasonableness.
Milk production, milk per cow, and number of milk cows are subject to revision the following month after initial
publication for monthly states or the following quarter for the quarterly states. Normally, administrative data from Federal
Market Orders, State Departments of Agriculture, or other sources are the main basis for revisions. However,
administrative data for all states may not be available in time for these revisions. Estimates are again subject to revision in
February each year based on additional administrative data. In the event that additional changes are necessary, a third
revision is possible in February the following year.
Every five years, NASS conducts the Census of Agriculture, which is an exhaustive data collection effort for all known
farm operations across the United States. Estimates are thoroughly reviewed for possible revision after data from the
Census of Agriculture are available. The information gathered from the Census of Agriculture is used to establish
benchmark levels by which the survey estimators can be compared, and bias determined. Survey based estimators can also
be impacted by outliers – individual reports that have excessive influence on the results due to either improper
classification or extremely unusual data for a given operation (i.e., operation is not representative of other operations).
NASS thoroughly reviews the survey data to identify these situations and consider their impact on the survey results when
establishing the official estimates.

Milk Production Methodology and Quality Measures (March 2023)
USDA, National Agricultural Statistics Service

3

Quality Metrics for Milk Production
Purpose and Definitions: Under the guidance of the Statistical Policy Office of the Office of Management and Budget
(OMB), the United States Department of Agriculture’s National Agricultural Statistics Service (NASS) provides data
users with quality metrics for its published data series. The metrics tables below describe the performance data for all
surveys contributing to the publication. The accuracy of data products may be evaluated through sampling and
nonsampling error. The measurement of error due to sampling in the current period is irrelevant for a fully enumerated
data series. Nonsampling error is evaluated by response rates and the weighted item response rate.
Sample size is the number of observations selected from the population to represent a characteristic of the
population. Operations that did not have the item of interest or were out of business at the time of data collection
have been excluded.
Response rate is the proportion of the above sample that completed the survey.
Weighted item response rate is a ratio of reported survey data expanded by the original sampling weight
compared to final nonresponse adjusted summary totals.
Coefficient of variation provides a measure of the size for the standard error relative to the point estimate and is
used to measure the precision of the results of a survey estimator.
Milk Production Survey Sample Size and Response Rate: To assist in evaluating the performance of the estimates in
the quarterly milk production report, the sample size and response rate are displayed.

Quality Metrics for Milk Production Survey - United States: January 1, 2022-2023
Sample size

Response rate

2022

2023

2022

2023

(number)

(number)

(percent)

(percent)

United States ...............................................................................

11,157

11,355

42.5

44.4

Quality Metrics for Milk Production Survey - United States: January 1, 2022-2023
Weighted item
response rate

Class

Coefficient
of variation

2022

2023

2022

2023

(percent)

(percent)

(percent)

(percent)

Milk production .............................................................................

45.0

45.6

0.4

0.4

Milk cows ......................................................................................

44.8

45.4

0.4

0.4

4

Milk Production Methodology and Quality Measures (March 2023)
USDA, National Agricultural Statistics Service

Milk Production Survey Sample Size and Response Rate – States and United States:
January 1, 2022-2023
Sample size

State

2022

Response rate
2023

(number)

2022

(number)

2023

(percent)

(percent)

Alabama .......................................................................................
Alaska ..........................................................................................
Arizona .........................................................................................
Arkansas ......................................................................................
California ......................................................................................
Colorado ......................................................................................
Connecticut ..................................................................................
Delaware ......................................................................................
Florida ..........................................................................................
Georgia ........................................................................................

47
(D)
46
39
547
95
65
20
82
103

47
(D)
45
43
553
97
56
18
74
88

17.0
(D)
30.4
33.3
49.9
49.5
47.7
25.0
15.9
34.0

36.2
(D)
35.6
23.3
40.9
45.4
44.6
38.9
16.2
43.2

Hawaii ..........................................................................................
Idaho ............................................................................................
Illinois ...........................................................................................
Indiana .........................................................................................
Iowa ..............................................................................................
Kansas .........................................................................................
Kentucky ......................................................................................
Louisiana ......................................................................................
Maine ...........................................................................................
Maryland ......................................................................................

(D)
293
204
277
368
109
180
68
116
205

(D)
300
194
289
368
108
182
77
114
201

(D)
41.3
52.9
43.7
32.3
29.4
53.3
42.6
46.6
43.9

(D)
38.3
53.1
49.8
44.6
48.1
49.5
42.9
55.3
48.3

Massachusetts .............................................................................
Michigan .......................................................................................
Minnesota ....................................................................................
Mississippi ....................................................................................
Missouri ........................................................................................
Montana .......................................................................................
Nebraska ......................................................................................
Nevada .........................................................................................
New Hampshire ...........................................................................
New Jersey ..................................................................................

81
647
929
50
192
37
78
26
87
45

79
611
930
70
208
35
80
31
79
49

44.4
43.3
42.0
38.0
44.8
62.2
39.7
34.6
50.6
33.3

60.8
41.2
42.3
32.9
45.7
40.0
26.3
19.4
53.2
24.5

New Mexico .................................................................................
New York .....................................................................................
North Carolina .............................................................................
North Dakota ................................................................................
Ohio ..............................................................................................
Oklahoma .....................................................................................
Oregon .........................................................................................
Pennsylvania ...............................................................................
Rhode Island ................................................................................
South Carolina .............................................................................

107
1,509
91
46
441
63
94
1,050
13
41

96
1,499
100
39
456
101
92
1,092
19
39

18.7
40.2
65.9
50.0
35.8
28.6
44.7
39.4
7.7
26.8

22.9
41.2
66.0
59.0
49.8
35.6
53.3
46.2
5.3
46.2

South Dakota ...............................................................................
Tennessee ...................................................................................
Texas ...........................................................................................
Utah ..............................................................................................
Vermont .......................................................................................
Virginia .........................................................................................
Washington ..................................................................................
West Virginia ................................................................................
Wisconsin .....................................................................................
Wyoming ......................................................................................

107
89
288
113
249
164
182
48
1,388
15

109
90
330
101
245
177
189
44
1,475
18

50.5
43.8
39.9
58.4
49.8
56.1
42.3
56.3
46.0
46.7

55.0
33.3
33.3
67.3
55.1
48.6
42.3
61.4
47.0
55.6

Other States 1 ..............................................................................

23

18

52.2

66.7

United States ...............................................................................

11,157

11,355

42.5

44.4

(D) Withheld to avoid disclosing data for individual operations.
1
Other States includes Alaska and Hawaii.

Milk Production Methodology and Quality Measures (March 2023)
USDA, National Agricultural Statistics Service

5

Quality Metrics for Milk Production Survey - States and United States: January 1, 2022-2023
Milk production
State

Milk cows
Coefficient
of variation

Weighted item response rate
2022

2023

(percent)

(percent)

Weighted item response rate

2022

2023

2022

2023

(percent)

(percent)

(percent)

(percent)

Coefficient
of variation
2022

2023

(percent)

(percent)

Alabama .............
Alaska .................
Arizona ...............
Arkansas ............
California ............
Colorado .............
Connecticut ........
Delaware ............
Florida .................
Georgia ...............

32.4
(D)
44.5
50.9
57.3
55.2
44.0
56.9
13.5
48.8

70.7
(D)
43.9
49.5
50.4
52.8
40.2
59.2
9.7
55.3

32.4
(D)
2.0
9.6
1.3
4.2
6.5
30.1
1.2
4.1

17.8
(D)
1.9
22.1
1.7
0.7
3.4
17.0
7.5
1.5

36.1
(D)
43.8
47.3
56.7
54.7
44.2
59.6
14.5
46.1

71.1
(D)
41.6
45.9
50.1
53.8
41.0
61.4
11.7
53.0

39.7
(D)
1.4
10.5
1.2
2.8
5.0
27.6
1.5
3.2

13.5
(D)
2.2
19.2
1.6
0.6
6.6
13.2
6.6
1.8

Hawaii .................
Idaho ...................
Illinois ..................
Indiana ................
Iowa ....................
Kansas ...............
Kentucky .............
Louisiana ............
Maine ..................
Maryland .............

(D)
40.0
50.2
39.3
29.0
21.1
56.5
43.3
43.0
49.4

(D)
31.7
56.4
57.5
44.2
33.8
55.1
64.2
60.5
49.4

(D)
0.5
2.6
1.6
2.2
0.9
3.7
4.6
2.3
2.7

(D)
1.2
3.5
1.0
1.9
1.2
3.3
6.6
3.6
2.8

(D)
41.3
50.5
39.3
28.9
23.2
56.7
44.3
46.0
47.9

(D)
31.9
57.1
55.0
45.3
33.4
56.1
64.6
59.7
49.5

(D)
0.4
2.3
2.2
2.1
0.8
3.4
8.5
2.6
2.5

(D)
1.2
3.2
1.1
1.9
1.2
3.1
6.3
3.4
2.6

Massachusetts ...
Michigan .............
Minnesota ...........
Mississippi ..........
Missouri ..............
Montana .............
Nebraska ............
Nevada ...............
New Hampshire .
New Jersey ........

42.6
41.4
37.0
62.4
52.4
65.5
56.6
26.5
33.9
31.9

68.4
43.7
38.2
76.1
53.5
44.7
39.7
2.1
49.3
14.4

2.6
2.3
1.5
7.8
3.9
5.3
2.0
1.5
3.0
5.6

5.1
1.0
1.3
5.5
3.3
3.2
2.2
1.3
6.2
14.9

42.3
42.0
36.0
63.2
50.3
64.5
55.6
22.8
34.6
34.0

66.5
43.3
37.5
74.9
48.6
46.3
40.3
2.9
48.5
17.3

2.8
2.0
1.4
6.5
3.9
4.8
1.9
1.5
2.8
3.7

4.8
1.0
1.2
5.8
4.8
1.6
2.0
1.4
5.7
13.5

New Mexico ........
New York ............
North Carolina ....
North Dakota ......
Ohio ....................
Oklahoma ...........
Oregon ...............
Pennsylvania ......
Rhode Island ......
South Carolina ...

31.3
48.1
53.2
72.3
37.6
12.7
34.0
47.2
(Z)
48.4

29.3
53.2
50.1
82.8
56.3
34.0
68.3
53.4
(Z)
47.1

3.5
1.0
1.9
1.5
2.5
3.8
3.3
2.6
(Z)
4.9

1.8
1.1
1.8
1.4
2.7
3.5
3.0
2.4
(Z)
8.1

31.2
47.8
54.9
69.3
37.6
13.3
32.7
47.4
2.4
43.7

29.3
52.7
51.8
81.0
55.9
34.1
67.9
53.8
0.5
51.1

3.4
1.0
2.6
2.5
2.5
3.6
2.7
2.4
9.7
2.9

1.7
1.0
1.6
1.8
2.7
4.0
2.9
2.4
3.4
10.8

South Dakota .....
Tennessee .........
Texas ..................
Utah ....................
Vermont ..............
Virginia ...............
Washington ........
West Virginia ......
Wisconsin ...........
Wyoming ............

73.4
48.4
32.9
60.0
46.9
55.0
38.2
65.9
49.1
11.7

68.6
35.7
25.9
72.5
57.0
54.3
42.0
62.1
47.6
100.0

0.6
6.9
0.7
1.2
2.2
3.4
1.4
6.3
1.4
1.1

1.4
11.5
1.4
0.8
2.1
4.0
2.2
9.8
1.2
0.1

73.1
51.3
31.9
59.3
46.7
54.7
38.5
70.0
49.0
14.7

69.4
39.7
25.1
72.1
56.1
52.9
41.6
68.6
48.1
98.7

0.6
6.6
0.6
1.4
2.1
3.2
1.4
4.0
1.5
3.5

1.0
11.4
1.2
0.9
2.1
4.0
2.7
7.9
1.2
1.9

Other States 1

63.3

96.1

16.2

6.8

78.6

96.4

6.7

4.7

United States .....

45.0

45.6

0.4

0.4

44.8

45.4

0.4

0.4

(D) Withheld to avoid disclosing data for individual operations.
(Z) Less than half of the unit shown.
1
Other States includes Alaska and Hawaii.

6

Milk Production Methodology and Quality Measures (March 2023)
USDA, National Agricultural Statistics Service

Information Contacts
Process
Estimation ......................................
Data Collection ..............................
Questionnaires ..............................
Sampling and Editing ....................
Summary and Estimators .............
Dissemination ................................
Media Contact and Webmaster ....

Unit
Livestock Branch
Survey Administration Branch
Data Collection Branch
Sampling Editing and Imputation Methodology Branch
Summary Estimation and Disclosure Methodology Branch
Data Dissemination Office
Public Affairs Office

Telephone
(202) 720-3570
(202) 720-3895
(202) 720-6201
(202) 690-8141
(202) 690-8141
(202) 720-3869
(202) 720-2639

Email
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]

Access to NASS Reports
For your convenience, you may access NASS reports and products the following ways:
➢ All reports are available electronically, at no cost, on the NASS web site: www.nass.usda.gov.
➢ Both national and state specific reports are available via a free e-mail subscription. To set-up this free
subscription, visit www.nass.usda.gov and click on “National” or “State” in upper right corner above “search”
box to create an account and select the reports you would like to receive.
➢ Cornell’s Mann Library has launched a new website housing NASS’s and other agency’s archived reports. The
new website, https://usda.library.cornell.edu. All email subscriptions containing reports will be sent from the new
website, https://usda.library.cornell.edu. To continue receiving the reports via e-mail, you will have to go to the
new website, create a new account and re-subscribe to the reports. If you need instructions to set up an account or
subscribe, they are located at: https://usda.library.cornell.edu/help. You should whitelist [email protected] in your email client to avoid the emails going into spam/junk folders.
For more information on NASS surveys and reports, call the NASS Agricultural Statistics Hotline at (800) 727-9540,
7:30 a.m. to 4:00 p.m. ET, or e-mail: [email protected].
The U.S. Department of Agriculture (USDA) prohibits discrimination against its customers, employees, and applicants for
employment on the basis of race, color, national origin, age, disability, sex, gender identity, religion, reprisal, and where
applicable, political beliefs, marital status, familial or parental status, sexual orientation, or all or part of an individual's
income is derived from any public assistance program, or protected genetic information in employment or in any program
or activity conducted or funded by the Department. (Not all prohibited bases will apply to all programs and/or
employment activities.)
If you wish to file a Civil Rights program complaint of discrimination, complete the USDA Program Discrimination
Complaint Form (PDF), found online at www.ascr.usda.gov/filing-program-discrimination-complaint-usda-customer, or
at any USDA office, or call (866) 632-9992 to request the form. You may also write a letter containing all of the
information requested in the form. Send your completed complaint form or letter to us by mail at U.S. Department of
Agriculture, Director, Office of Adjudication, 1400 Independence Avenue, S.W., Washington, D.C. 20250 -9410, by fax
(202) 690-7442 or email at [email protected].