0218 - ARMS Chem Use 2023 - PartB - 04-20-2023

Supporting Statement – Part B

AGRICULTURAL RESOURCE MANAGEMENT, CHEMICAL USE,

AND POST-HARVEST CHEMICAL USE SURVEYS

OMB No. 0535-0218

The information that is provided in this supporting statement serves as an overview of the sampling, statistical methodology, weighting of data for non-response, methods for increasing response rates, measurements for accuracy, testing of instruments, etc. The more detailed information for the ARMS and Chemical Use surveys can be found in the attachments to this submission.

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection has been conducted previously, include the actual response rate achieved during the last collection.

Respondent Universe: ARMS and the Vegetable Chemical Use Surveys are screened together to identify records for sampling. The target population for ARMS is the official NASS farm population with the exclusion of institutional farms, approximately 2 million operations. The target populations for the ARMS Chemical Use (Phase 2), the Fruit and Vegetable Chemical Use Surveys, and most of the Field Crop Production Practice and Chemical Use Surveys are operations that produce the commodity of interest.

Sampling: The ARMS Phase 1/Integrated Screening Survey as well as Phase 2 surveys use a list frame of farms identified on the NASS list frame. Sample list strata are developed using major categories, a combination of targeted crops, livestock, fruit, vegetables, and horticulture. Strata are developed by grouping operators by the total value of sales followed by the presence of the targeted commodity. The list is an efficient sampling frame because it contains most of the farms with the largest production, have the commodity of interest, and economic activity.

The list classification process is very extensive, examining many crop and livestock control data values. After classification is completed, list records are partitioned into size groups based on qualifying control data for the current year commodities, type of farm, and estimated size. Operations will always belong to one of the standard expenditure strata and are eligible to also belong to a targeted commodity stratum.

Generally, only positive Phase 1 respondents are eligible for Phase 2 selection. However, inaccessible records are sampled, if needed, to meet Phase 2 target sample sizes. Multiple operating arrangements are identified during the screening phase and only one operating arrangement is randomly selected for inclusion in Phase 2.

Sequential Interval Poisson (SIP) sampling is used for the PPCR surveys in ARMS. In SIP, the sampling probabilities are defined to ensure each operation is in one and only one sample. The probabilities of selection can be based on any type of probability scheme. The SIP procedure was used to minimize overlap with the previous year’s ARMS sample as well as the current year’s Agricultural Production Survey (APS) samples used for OMB Control Number 0535-0213.

The Vegetable Chemical Use Survey is screened in the spring with the ARMS screening to identify operations with targeted crops. The sample design for the Fruit and Vegetable Chemical Use Survey is a multivariate probability proportional to size (MPPS) design. Acreage of all targeted crops that the grower reported in the screening phase or on the list frame are included when determining a grower’s probability of selection.

Response Rates: Following are average response rates for survey phases 1 and 2 based on the last three survey cycles.

(Overall completion average for the last three years for the ARMS surveys was 49.9%). Response dropped from an average of 57 percent prior to 2019. It is likely that questionnaire fatigue and an increase in page length contributed to the drop, in addition to the data collection shift from in-person interviewing due to the COVID-19 pandemic.

For the 2020 surveys, NASS utilized phone enumeration and computer assisted web interviews when possible for all surveys, which may have affected response. This was due to the need for social distancing during the pandemic. Field enumeration of the phase 2 and chemical use surveys will resume when it is safe to do so as determined by NASS and the National Association of State Departments of Agriculture (NASDA).

NASS continues its efforts at reducing respondent burden while improving response rates. With the combined work of our Research and Development Division, Methods Division, Census and Survey Division, Public Affairs Office, and our Training Group, NASS is looking at what factors work for some surveys and not others. NASS also utilizes Farm Service Agency records to complete Phase I records where possible. This effort reduces response burden for the Phase I survey. Through the use of project management techniques and building on to lessons we have learned from previous surveys and the Census of Agriculture we are able to make changes to internet versions of questionnaires to make them more user friendly (utilizing an adaptive web design), combining smaller surveys so that we can reduce the frequency of contacting the farmers, and improve on our sampling of farmers.

The NASS Public Affairs Office (PAO) promotes NASS survey efforts and educates respondents about the need and uses for the data they are asked to provide. This group has developed survey-specific materials enumerating the benefits and uses of the data gathered from the economic surveys as well as the chemical use efforts. PAO works with data users and industry leaders to provide concrete examples of instances where the data that respondents provide are used to service the respondents. They also work with commodity organizations based the commodity of interest each year. They are also actively publicizing survey activities by generating and distributing news reports and drop-in articles for industry publications and news outlets.

2. Describe the procedures for the collection of information including:

• statistical methodology for stratification and sample selection,

• estimation procedure,

• degree of accuracy needed for the purpose described in the justification,

• unusual problems requiring specialized sampling procedures

Agricultural Resource Management Survey (ARMS) - The annual surveys collect production practices and cost of production data on selected commodities from a representative sample of farms and ranches from across the country. To accomplish this, the ARMS are conducted in multiple data collection phases. In many ways, the multiple phases can be viewed operationally as independent surveys. However, the power of the ARMS design is that data across phases are related and can be combined and analyzed. Estimated sample sizes are shown in the Supporting Statement A, item 12, table.

The ARMS Phase 1 or Integrated Screening Survey (ISS) is conducted from May through July, and it collects general farm data such as crops grown, livestock inventory, and value of sales. The Integrated Screening Form is used in years that NASS will be conducting the Vegetable Chemical Use Survey. The integrated form will be used for both the ARMS 2 and the Vegetable Chemical Use Survey. These data are used to qualify or screen farms for these surveys to make sure the samples are as accurate as possible. The sample questionnaires are attached.

The ARMS Phase 2 is conducted from September through December. This phase collects data associated with agricultural production practices (field operations, pest management practices, etc.), resource use (pesticide applications, fertilizer and nutrient application, types of equipment used, etc.), and variable costs of production for specific commodities. The respondent is given an information booklet for each crop with code definitions and conversion tables to help complete the questionnaire. Samples of the Phase 2 advance letter and flyer, respondent booklets, questionnaires, and telephone quality control sheets are attached to this renewal submission.

As questionnaires are updated each year to accommodate changes in the farming conditions for that year or for a particular commodity, the final versions will be submitted to OMB as they become available.

Vegetable Chemical Use Survey - The vegetable survey targets operators with selected commodities. Vegetable operations are screened as described above. Only active operations with the crops of interest become part of the population for the fall survey.

Vegetable producers selected for the survey are asked to complete an interview with questions pertaining to whole farm acreage and production, chemical products used and application rates, pest management practices, organic practices, and operator characteristics. Collection of vegetable chemical use data begins in early October. Samples of the questionnaire versions, along with advance letters, respondent booklets, and a telephone quality control sheet are attached.

Fruit Chemical Use Survey - The fruit survey targets operators with selected commodities. Fruit operations are sampled from the NASS List Frame. Only active operations with the crops of interest become part of the population for the fall survey.

Fruit producers selected for the survey are asked to complete an interview with questions pertaining to whole farm acreage and production, chemical products used and application rates, pest management practices, organic practices, and operator characteristics. Collection of fruit chemical use data begins in early October. Samples of the questionnaire versions, along with advance letters, respondent booklets, and a telephone quality control sheet are attached.

NASS Regional Field Office staff will receive a Survey Administration Manual which provides detailed aspects of the survey data collection and editing process. Field enumerators in each State will be given an Interviewer’s Manual.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

Based on previous studies, NASS feels that the best tool for increasing response rates is improving the training of our field enumerators. Enumerators who are better prepared to answer questions raised by respondents and to inform respondents of the importance of the data and how it will be used have had the best success rates. These data are very important to both the farming community as well as external data users (politicians, educators, banking industry, farm supply companies, etc.).

The following are the non-response adjustments for ARMS 2 and Chemical Use.

Unit non-response in the Vegetable Chemical Use Survey as well as ARMS 2 is accounted for using reweighting. The records are stratified by State and size group, and a non-response adjustment is calculated as the sample size divided by the number of completed reports. This process redistributes the survey weights for the non-respondents to the usable records. For ARMS 2, the weights are then scaled so that the expanded total of the target commodity’s planted acres is equal to the planted acreage number set by the ASB (Agricultural Statistics Board). For the Vegetable Chemical Use Survey, a calibration program adjusts the weights so that the expanded planted acreage totals for each target crop match the planted acreage set by the ASB.

1

Item non-response in ARMS 2 and the Vegetable Chemical Use Surveys is handled with mean imputation. Missing fertilizer and pesticide rates are replaced with average rates by State, commodity, and product code (or nutrient in the case of fertilizer rates). If no records exist in that category, then the groups are collapsed. No other items in these surveys are imputed.

NASS deploys several data collection evaluation and monitoring tools. These tools enhance NASDA data collection tactics producing maximum positive impact on coverage/calibration. Real-time NASDA data collection oversight allows for flexibility of collection tactics and an improvement in survey quality metrics. Over several years, results from these survey monitoring tools are blended into a cohesive ARMS 2 data collection strategy, tailored to each State.

4. Describe any tests of procedures or methods to be undertaken.

NASS uses an OMB-approved generic clearance docket (OMB Control # 0535-0248), to conduct testing and evaluation of most NASS questionnaires. In this PRA approval request, NASS is including an allowance to conduct a total of 50 cognitive test interviews (annually) on the various questionnaires included in this docket. If a different method of testing is necessary or a larger sample is needed, NASS will submit a request using the generic clearance docket (0535-0248). The generic testing docket allows for a variety of testing methods, including cognitive testing, focus groups, split sample field tests, etc., that can be used to test ARMS and other NASS surveys. NASS does not plan to create a cognitive laboratory facility due to the geographic dispersion of farm operators needed for testing. As is typical in establishment surveys, most testing is conducted with onsite visits.

NASS plans to conduct cognitive interviews for all major changes that are proposed for the ARMS and Chemical Use Surveys. These interviews would address specific questions and sections to assess modified content and formatting.

Web-based data collection is available for the ARMS I survey samples nationally. Additionally, Computer-Aided Self-Administered Interview (CASI) began in fall 2009 and is available for all enumerators to use.

NASS has experience from previous chemical use surveys that have been beneficial in designing the surveys explained in this docket. Pre-testing of restructured or rotated in sections of questionnaires will be done annually for each survey, refining the data collection instruments each year. The results of these tests and subsequent methods will be incorporated into the operational design.

Response improvement techniques will continue to be researched and tested to improve response rates in the area of questionnaire improvement, respondent relationship building, and soft refusal conversion techniques.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The sampling plans are developed by NASS. Questionnaire design, data edit, and initial summarization will also be completed by NASS, and for the ARMS surveys, these tasks will be done in consultation with ERS.

Survey design and methodology are determined by the Summary, Estimation, and Disclosure Methodology Branch, Statistics Division; Branch Chief is Jeff Bailey, (202) 690-8141.

Sample sizes for each State are determined by the Sampling, Editing, and Imputation Methodology Branch, Methods Division; Branch Chief is Mark Apodaca, (202) 690-8141.

Data collection is carried out by NASS Regional Field Offices (RFOs). The Western Field Operation’s Director is Troy Joshua, (202) 720-8220. The Eastern Field Operation’s Director is Jody McDaniel, (202) 720-3638.

The NASS survey statisticians in Headquarters listed below are responsible for coordination of sampling, questionnaires, data collection, and other Field Office support. Branch Chief is Gerald Tillman, (202) 720-3198; Section Head is Torey Lawrence (202) 720-5921.

The NASS commodity statisticians in Headquarters listed below are responsible for national summaries, analysis, and publication. Branch Chief is Tony Dorn, (202) 720-6146, Section Head is Bruce Boess (202) 720-4447.

Survey	Survey Statisticians Census and Survey Division, Survey Administration Branch, Environmental and Economic Surveys Section	Commodity Statisticians Statistics Division, Environmental, Economics and Demographics Branch, Economic, and Environmental and Demographics Sections
ARMS 1 Screening	Pam Coleman (202) 720-6564	Vacant
ARMS 2	Jeff Lemmons (202) 690-3692	Doug Farmer (202) 690-3229
Fruit Chemical Use	Jeff Lemmons (202) 690-3692	Doug Farmer (202) 690-3229
Vegetable Chemical Use

April 2023

8