0213-ags-2020 - SSB

Supporting Statement – Part B

AGRICULTURAL SURVEYS PROGRAM

OMB No. 0535-0213

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection has been conducted previously, include the actual response rate achieved during the last collection.

NASS’s List Frame universe consists of approximately two million farms in the United States. The List Frame surveys are drawn from the NASS Sampling Frame which is comprised of all active operations on NASS’s List Frame that have positive control data for targeted commodities in program states.

NASS’s Area Frame universe consists of all land in the United States except Alaska. Land in each State has been stratified by type of land use. The area within a land use stratum is divided into substrata based on geographic similarities.

The following table represents the sample sizes and response rates for the surveys previously conducted under this approval.

NASS recognizes that the following surveys in this docket fall below 80% response rate and below 70% coverage rate.

1. Area Frame Survey Agricultural Tracts

2. Agricultural Surveys – Quarterly (Crop Acreage and Grain Stocks)

3. Agricultural Land Values (Supplemental List Survey)

4. On Farm Rice Stocks

5. Sheep and Goat Report (January)

A non-response bias analysis has been conducted on the quarterly surveys conducted in 2019 and the report is attached to this submission.

The priority for nonresponse bias analysis was focused on the Quarterly Crops Acreage, Production and Stocks (APS) Surveys. These surveys are the largest and most critical surveys in this docket as it is the primary surveys for collecting crop acreage, end of season production, and on-farm grain stocks. Data from the 2017 Census of Agriculture and USDA administrative data was used to assess bias in acreage and yield indications.

This study did show some bias in the acreage indications from the survey. However, NASS uses USDA Farm Service Agency (FSA) crop planted acreage data to complement the survey data and no additional bias correction is needed. FSA data coverage at the time estimates are set is generally more than 95%. A harvested to planted acres’ ratio from the December Crops APS survey is then used to set harvested acres’ estimates. While a direct analysis of this ratio was not possible, ratios generally have less bias than totals.

A comparison of yields to two different sources (2017 Census of Agriculture and USDA administrative data) indicated bias but in opposite directions. There was a noticeable bias in the production indications for upland cotton. However, we have supplemental data from our cotton ginning’s survey. This survey collects ginning data biweekly from all active cotton gins as identified by the Agricultural Marketing Service (AMS). A more detail report is included with this submission.

2. Describe the procedures for the collection of information including:

• statistical methodology for stratification and sample selection,

• estimation procedure,

• degree of accuracy needed for the purpose described in the justification,

• unusual problems requiring specialized sampling procedures

NASS utilizes multiple frame sampling (list and area) on many of the surveys contained in this docket.

The area frame is constructed by first placing all land in the United States (except Alaska) into one of several large land use strata. These land use strata are sub-stratified into Primary Sampling Units (PSUs) and are systematically sampled.  Selected PSUs are further stratified into secondary sampling units (SSUs) or segments before a simple random sample is employed.  

Segments in predominately cultivated areas are about one-half to two square miles in size while those in residential and urban areas are about one tenth to one-quarter square mile. Segments in open range and woodland areas vary in size from 1 to 100 square miles. Rotation of segments is achieved by replacing one or more complete replications: each year 20 percent of the segments are rotated out and 20 percent are rotated in.

The Agricultural Surveys (quarterly crops) and Agricultural Yield Surveys are sampled from NASS’s Sampling Frame, which is comprised of all active operations on NASS’s List Frame that have positive control data for targeted crops in program states.  Similarly, the Cattle Report, Hog Report, Sheep and Goats Report, Cattle on Feed Inquiry are sampled from NASS’s Sampling Frame from all active operations on NASS’s List Frame that have positive control data for Cattle, Hogs, Sheep or Goats and Cattle on Feed, respectively, in program states.   

 

The Cattle Report, Sheep and Goats Report, Cattle on Feed Inquiry and the Agricultural Land Values Surveys utilize a stratified design.  The Cattle Report and the Sheep and Goats Report are systematically sampled within strata; the Hog Report and Agricultural Land Values Survey utilize a simple random sample within strata; and Cattle on Feed Inquiry is sampled at 100%.  

The Agricultural Surveys (quarterly crops), Cattle Reports, Hogs Reports and Sheep and Goats Reports, also utilize Area Frame samples which are composed of all operations identified by the June Area Survey who do not overlap with a NASS List Frame record. The Area Frame operations that have cattle are sampled at a rate less than 100%, while the operations with hogs, sheep, and goats are sampled at 100%. These area frames and corresponding samples provide a mechanism to measure the incompletes of NASS’s List Frame for these commodities.

Due to the coronavirus situation, NASS will not conduct the 2020 June Area Survey (JAS) because it requires face-to-face enumeration. To obtain crops, cattle, hog, and sheep and goat area indications, non-overlap (NOL) samples will be added to the 2020 Agricultural, Cattle, Hog and Sheep and Goats List samples. NASS will identify NOL entities by overlapping 2019 JAS tracts with 2020 ELMO List Frame.

The Agricultural Surveys (quarterly crops) and the Agricultural Yield Surveys utilizes a Multivariate Probability Proportional to Size (MPPS) sampling design^[1].  The MPPS design takes advantage of the efficiencies of a Probability Proportional to Size (PPS) design, while adding the dimension of utilizing multiple variables from partially overlapping frames in the sample allocation.  Sampled replicates are assigned to each of the four survey quarters. The probability of selection for an MPPS design is:

Where:

is the maximum probability of selection for farm operator in state .

is the target commodity .

is the number of farm operators in the sampling frame for target commodity , and

is the value of target commodity for the thfarm operator in state .

The Agricultural Surveys (quarterly crops) provide detailed estimates of crop acreage, yields and production, and quantities of grain and oilseeds stored on farms. These surveys are conducted in all states quarterly. Farm operations are selected from an area frame and a list frame to produce "multiple frame" estimates. Farm and ranch operators from the list frame are selected by size depending on the proportion of the commodities of interest the operation has in comparison with other operators on the list. The area frame sample is added to account for land not covered by the list frame. The sample targets producers of row crops and small grains and farms operations with grain storage capacity. Note: Due to the coronavirus situation, NASS will not conduct the 2020 June Area Survey (JAS) because it requires face-to-face enumeration.

The Agricultural Yield survey provides farmer reported survey data of expected crop yields used to forecast and estimate crop production levels throughout the growing season. These surveys are conducted in all states except Alaska and Hawaii. Samples of farm operators are selected from the March Agricultural Survey (small grains) and the June Agricultural Survey (late season crops and tobacco). Farmers reporting acreage of at least one commodity of interest are included in the monthly data collection to forecast crop yields.

The Cattle Report provides basic inventory data that describe the nation’s cattle herd. The reports provide estimates of the number of breeding animals for beef and milk production as well as the number of heifers being held for breeding herd replacement. Estimates of cattle and calves being raised for meat production are also included. The number of calves born during the previous year is also measured. These surveys are conducted in all states. A sample of cattle producers is selected from the NASS list frame. A sample of area tracts is selected to measure incompleteness of the list. This ensures statistical coverage of all cattle operations in each state. Note: Due to the coronavirus situation, NASS will not conduct the 2020 June Area Survey (JAS) because it requires face-to-face enumeration.

The Cattle on Feed Inquiry provides estimates of the number of cattle being fed a ration of grain, silage, hay and/or protein supplements for the slaughter market that are expected to produce a carcass that will grade select or better. These surveys are conducted in the 16 largest cattle-feeding states. About 2,000 known cattle feeders with a capacity of 1,000 or more head are enumerated. Feedlots with 1,000 or more head capacity represent about 85 percent of all fed cattle in the U.S. The 16 largest states represent 98 percent of U.S. cattle on feed in lots of 1,000 head or more capacity. Data are used in conjunction with Cattle Inventory data from January and July to obtain a measure of cattle on feed not included in the survey.

A supplemental sample of feedlots is selected in Iowa for operations with less than 1,000 head capacity. A census of these operations is conducted in January. From the positive reports a stratified sample of approximately 500 operations is drawn and contacted in February through December.

The Hog Reports provide detailed inventory of breeding and marketing hogs and the future supply of market hogs on a quarterly basis. Hog owners, including contractors, are the target population. The sampling universe for the hog estimation program is all hog operations with capacity to raise breeding or market hogs. A sample of hog operations from the list frame maintained by NASS is supplemented by a sample of area tracts to ensure complete coverage. All states are included in the December Hog survey while producers in the largest 29 hog producing states are surveyed during the following three quarters. Hog operations who own hogs in more than one state report inventories by state so the hogs and pigs are allocated to the proper state. Note: Due to the coronavirus situation, NASS will not conduct the 2020 June Area Survey (JAS) because it requires face-to-face enumeration.

The Agricultural Land Values survey is designed to supplement the land value data collected during the June Area Survey. This survey is conducted in ten states.

Due to the coronavirus situation, NASS will not conduct the 2020 JAS because it requires face-to-face enumeration. To ascertain Land Value indications for all states except AK, NASS will instead conduct a 2020 Land Values Survey (LVS) using 2019 JAS area tracts as the sampling frame. There will be two major differences between a JAS and the 2020 LVS: the LVS will not be enumerated face-to-face and survey questions will pertain to the whole farm, not specific JAS tracts.

The On-Farm Rice Stocks survey is conducted in August for six states with on-farm rice stocks. The on-farm rice stocks data for March, June and December are collected in the same six states by the Agricultural Survey (quarterly crops).

NASS’s sets coefficient of variation (CV) national and state targets for major probability surveys to ensure the published estimates are precise and accurate. The national Target CVs for targeted commodities range from 1.5 to 6.0 in the Crops APS Quarterly Surveys and range from 1.0 to 8.0 in the livestock surveys. Every year, survey results are analyzed and sample allocations are adjusted to ensure Target CVs are met.

Data Collection, Quality Control, and Estimation Activities

Data for the June Area Survey are obtained by personal contact. Data for refusals must be imputed by visual observation or based on similar type operations where data was collected. The Area Screening form is used to screen for farm operators living inside the segment or having any land inside the segment; the Area Version questionnaire is used to record data for agricultural tract operators. The reporting unit for crop acreage is the number of acres located inside the segment. Acreage within a segment is expanded to the State level using the sampling probability assigned to each segment: this is called the “closed segment” expansion. Estimates of livestock inventories and number of farms use the “weighted farm” approach in which each farm is weighted by the ratio of acres inside the segment to the total acres of the entire farm, both inside and outside the segment.

For the List Frame surveys, NASS begins data collection by mailing out a blank copy of the questionnaire with a cover letter explaining the importance of the survey along with who relies on this information. NASS also sends out instructions on how the respondent can complete the survey either online or on paper and mail it back in if they choose to. After about a week NASS begins phone follow-up of non-respondents. Mail and telephone non-contacts are followed up with face-to-face enumeration. The entire sample is accounted for. As a rule, large operations and operations requiring special handling (previous survey refusals and inaccessibles, complex operations, etc.) are contacted for a face-to-face interview only. The mix of data collection modes is determined by the Regional Offices according to resources available. Over half of the information is obtained by telephone. Data collection begins on or around the first of the survey month. Reference dates for livestock inventories and grain stocks are the first of the survey month; crop acreage and production are collected for the current crop year.

Quality control for the area agricultural survey consists of two parts: (1) review of two segments during the data collection period by the supervisory enumerator for each field enumerator under his or her supervision and (2) telephone calls made to at least one additional randomly selected tract operator from each enumerator's assignment list. Telephone contacts are made by either a statistician from the Regional Field Office, Data Collection Center, or a supervisory enumerator.

For survey quality control NASS uses telephone interview monitoring and a telephone quality control worksheet. The telephone check is conducted twice a year for each enumerator to ensure the original contact provided reliable data and that the enumerators are collecting the data accurately and in a professional manner. The quality control forms are not conducted on the internet due to the nature of these surveys.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

All surveys conducted by NASS have very thorough data collection plans and processes. For many of our larger surveys, NASS has a well-planned marketing strategy to help publicize the survey in hopes of increasing response. NASS will send informational postcards, provide data fact sheets, or include letters from industry groups that explain the benefits of the survey and illustrate the importance each operation’s response is to the overall project.

Most of the surveys conducted by NASS have been made available to the targeted operators by mail, phone, internet, fax, and personal interview. However, due to the COVID-19 virus some of the surveys have had to have the personal interviews suspended for part of 2020. Also, our phone enumerators have been equipped with IPads so that they can continue to call farmers to collect data during this period, but they will be able to do this from their homes rather than from centralized phone centers.

Nonresponse follow-up is a critical component of our data collection plans. Nonresponse follow-up includes additional mailings of the questionnaire, telephone contacts, and personal visits to the operator. The exact methods used vary by survey depending on the modes of data collection available for that particular survey. Also, in some surveys, NASS identifies strata(s) of “extreme operators.” This term refers to operations within a state that are considered large enough to be highly influential to the state estimates for the survey they are sampled. NASS will often special handle these respondents and collect data through personal visits or have a data collection plan already coordinated with the operator to help ensure a successful interview. This helps guarantee a sufficient level of coverage of the item of interest.

Even with these efforts to collect and account for as much data as possible, nonresponse still exists and needs further attention when data is analyzed and summarized. NASS utilizes various methods for automated nonresponse adjustments that often include imputation routines and/or weight adjustments. The completed reports that we receive that are in-scope for the item of interest will be used in either approach. Every effort is made to form homogenous groupings of respondents so that similar operations are used to represent other similar operations to ensure the most accurate results as possible.

NASS will be incorporating more publicity materials in future years with each survey. Operators who are selected for each survey will be encouraged to complete the surveys using the internet at their convenience. Hopefully, this will result in an increased response rate and reduced data collection costs.

4. Describe any tests of procedures or methods to be undertaken.

Periodically, NASS conducts cognitive interviews with farm or ranch operators to see if the questionnaires are being interpreted in the manner in which they are intended. In addition, NASS measures the time it takes the respondent to complete the survey as well as any questions they need to access their operations records in order to complete the survey. NASS also checks to see if the terminology and language used in the questionnaires are still relevant with current farming practices. Any significant changes to the questionnaire are tested prior to their implementation.

Cognitive testing was conducted this past year on value of sales questions, which are included in several of the surveys in this docket. Minor changes were made as a result of this testing. In addition, NASDA Field Enumerators were asked about any issues they encountered while collecting data that involved agricultural land values and minor adjustments were made to reflect this.

Quality control for the area agricultural survey consists of two parts: (1) review of two segments during the data collection period by the supervisory enumerator for each enumerator under his or her supervision and (2) telephone calls made to at least one additional randomly selected tract operator from each enumerator's assignment. Telephone contacts are made by either a statistician from the Regional Field Office, Data Collection Center, or a supervisory enumerator.

The telephone quality control worksheet is also used to test methods for the list surveys. The telephone check is conducted twice each year for each enumerator collecting Agricultural Surveys data. The purpose of this is to ensure the original contact provided reliable data and that the enumerators are collecting the data accurately and in a professional manner.

In Supporting Statement A, Item 12 (table) additional burden was included this time to allow for re-contacts of respondents to see if weather conditions that occurred during the time of data collection may have impacted the quality of the data,

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Specifications and survey design were developed by Summary, Estimation, and Disclosure Methodology Branch, Methodology Division; Branch Chief is Jeff Bailey (202) 690-8141.

The sampling plan was developed by the Sampling, Editing, and Imputation Methodology Branch, Methodology Division; Branch Chief is Mark Apodaca, (202) 690-8141.

Data collection is carried out by NASS Regional Field Offices; Eastern Field Operation’s Director is Jay Johnson, (202) 720-3638 and the Western Field Operation’s Director is Troy Joshua (202) 720-8220.

The NASS survey administrators in Headquarters for the Agricultural Surveys are in the Commodity Surveys Section of the Survey Administration Branch, Census and Survey Division; Branch Chief is Gerald Tillman, (202) 720-3895. The survey administrators are responsible for coordination of sampling, questionnaires, data collection, training, Interviewer’s Manuals, Survey Administration Manuals, data processing, and other Regional Office support.

The NASS commodity statisticians in Headquarters for the Agricultural Surveys are in the Crops Branch and Livestock Branch of Statistics Division; Branch Chiefs are Lance Honig (crops) (202)720-2127, and Travis Averill (livestock) (202)720-3570. Commodity statisticians are responsible for the Estimation Manuals, national and regional summaries, analysis, presentation to the Agricultural Statistics Board for final estimates, and publication.

April 2020

[1]^1]Bailey, Jeff and Kott, Phillip (1997),” An Application of Multipurpose List Frame Sampling for Multi-Purpose Surveys, Proceedings of the Section on Survey Research Methods, American Statistical Association, pp. 496-500.

1