0218-arms-08_PartB

Supporting Statement – Part B

AGRICULTURAL RESOURCE MANAGEMENT, CHEMICAL USE,

AND POST-HARVEST CHEMICAL USE SURVEYS

OMB No. 0535-0218

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection has been conducted previously, include the actual response rate achieved during the last collection.

Respondent Universe: ARMS and the Vegetable Chemical Use Surveys are screened together to identify records for sampling. The target population for ARMS is the official NASS farm population with the exclusion of institutional farms, approximately 2 million operations. The target populations for the ARMS Chemical Use (Phase II) and Fruit and Vegetable Chemical Use Surveys are operations that produce the commodity of interest. The respondent universe for the Post-harvest Chemical Use Surveys are operations that store or handle the target commodities. The respondent universe for the Contractor Expense Surveys comes from known contractors on our list frame. No screening to identify these operations is necessary.

Sampling: The ARMS is a multiple frame survey using a list frame of small to large farms identified on the NASS list frame and a complementary area frame. Sample list strata are developed using major categories, a combination of targeted crops, livestock, fruit, vegetables, and horticulture. Strata are developed by grouping operators by the total value of sales followed by the presence of the targeted commodity. The list is an efficient sampling frame because it contains most of the farms with the largest production and economic activity. The area frame, stratified by land use, provides the completeness missing from the list. The multiple frame expansions are unbiased and more precise than expansions which could be obtained using one frame alone. For sampling organic producers, the NASS list frame will be supplemented with lists of certified organic producers originating from USDA-Agricultural Marketing Service (AMS) and lists of producers changing to organic which are supplied by industry contacts to ERS. These lists will be screened to verify that the operation is still in business and to determine presence of targeted commodity.

The list classification process is very extensive, examining many crop and livestock control data values. After classification is completed, list records are partitioned into size groups based on qualifying control data for the current year commodities, type of farm, and estimated size. The size groups are then collapsed into two sets of strata. One set consists of farms believed to have one of the targeted commodities for the current year. The other set contains all other identified medium to large farms, stratified by size and type of operation.

Operations will always belong to one of the standard expenditure strata and are eligible to also belong to a targeted commodity stratum. Targeted commodity strata take precedence over the expenditure strata in most instances but for some rarer commodities it is possible that the more rare strata can take precedence over the more common target commodity strata. Since targeted commodities have the potential to change each year, the classification and stratification may change as well.

The area frame sample consists of a subset of respondents from the June Agricultural Survey (JAS), included in OMB No. 0535-0213. During the JAS, farm operators associated with the land area segments are classified according to whether they are overlap with the list frame. Those farm operators not eligible for selection on the list are eligible for selection in the ARMS area sample.

Another procedure is used to reduce respondent burden in the ARMS sample selection process. This procedure, called Perry-Burt, is a statistically defensible method of reducing burden. List frame records selected for other current year NASS probability surveys (crops/stocks, hogs, cattle, and agricultural labor) or the previous year’s ARMS are replaced where possible by similar sample units whose respondent burden is less. As a result this design reduces the number of consecutive ARMS contacts and multiple contacts for different surveys in the same year. It may not be possible to find similar sample units to use for replacement in higher strata where the sample interval is low.

The sampling population for the Fruit and Vegetable Chemical Use surveys will be obtained from the ARMS classify code as well. All records on each State’s list frame having target vegetables or a vegetable indicator will have a positive probability of selection. The purpose for using the ARMS classify for creating the Chemical Use sampling population is to more effectively control overlap between the ARMS and the Chemical Use surveys. The Vegetable Chemical Use Survey is screened in the spring with the ARMS screening to identify operations with targeted crops. The sample design for the Fruit and Vegetable Chemical Use surveys is a multi-variate probability proportional to size (MPPS) design. Acreage of all targeted crops that the grower reported in the screening phase or on the list frame are included when determining a grower’s probability of selection.

The population for Post-harvest Chemical Use surveys, are facilities that store the commodity of interest. Sampling is based on information maintained on the NASS list for volume stored or capacity. Sampling will vary significantly based on the commodity targeted for the survey; corn and soybeans, for example, were surveyed in 2003 and the population included warehouses and elevators where corn and soybeans were stored.

The sampling population for the Contractor Expense Surveys consists of all large contractors in each State for the four commodity groups (broilers/starter pullets, layers, turkeys and hogs).

Response Rates: Following are average response rates for all survey phases based on the last three survey cycles.

Average Response Rates
Survey	Survey Year *	Sample Size	Percent Response 2/	Percent Refusal	Percent Inaccessible
ARMS Screening 1/ 3/	2007	-	-	-	-
	2006	91,388	76.2	11.5	12.3
	2005	68,375	73.9	12.5	13.6
	2004	73,376	76.6	11.9	11.6
	Average		75.6
ARMS Phase II Production Practices	2007	4,128	71.8	20.5	7.8
	2006	8,075	81.3	13.8	4.9
	2005	7,935	80.5	12.8	6.7
	Average		77.9
ARMS III Cost and Returns 4/	2007	30,275	69.4	24.1	6.5
	2006	34,203	70.5	23.7	5.8
	2005	34,937	67.5	26.0	6.5
	Average		69.2
Fruit or Vegetable Chemical Use 5/	2007 Fruit	-	-	-	-
	2006 Veg	5417	86.6	7.8	5.5
	2005 Fruit	6,943	81.3	12.1	6.6
	2004 Veg	5,908	88.5	6.2	5.3
	Average		85.5
Post-harvest Chemical Use 3/	2007	-	-	-	-
	2006	2579	82.9	8.7	8.4
	2005	222	88.3	9.5	2.3
	2004	166	83.1	10.8	6.0
	Average		84.8
ARMS Phase II Organic component 1/ 3/	2007	387	79.8	13.4	6.8
	2006	463	86.0	9.5	4.5
	2005	-	-	-	-
	Average		82.9

* All survey years correspond to calendar years, except for ARMS Phase III. ARMS Phase III is conducted in Jan-Apr of year following the survey year. E.g.: 2010 ARMS Phase III is conducted in Jan-Apr 2011.

^1/ ARMS screening identifies operations that have target commodities for Phase II

and for the Vegetable Chemical Use Survey. Survey was not conducted in 2007.

^2/ Includes operations that responded and were out-of-scope.

^3/ Survey was not conducted in 2007 due to budget constraints.

^4/ ARMS Phase III counts Include organic component samples in 2005 for Dairy,

2006 - Soybeans, and 2007 - Apples.

^5/ Postharvest Chemical Use commodities were: 2004 – Oranges,

2005 – Peanuts, 2006 – Oats and Potatoes.

The Marketing and Information Services Office (MISO) promotes NASS survey efforts and educates respondents about the need and use for the data they are asked to provide. This group has developed survey-specific materials enumerating the benefits and uses of the data gathered from the economic surveys as well as the chemical use efforts. MISO works with data users and industry leaders to provide concrete examples of instances where the data that respondents provide are used to service the respondents. They are also actively publicizing survey activities by generating and distributing news reports and drop-ins for industry publications and news outlets.

Several studies and projects are underway in the NASS Research and Development Division that relate to ways to increase response rates and reduce respondent burden. One project is the development of a mail version of the core set of questions that are critical to producing data for the income accounts and will be repeated annually. Preliminary research shows that the core version reduces respondent burden by approximately one-third. Another effort that is underway is research into identifying the validity of using previously reported data for the completion of certain sections of the ARMS Phase III.

2. Describe the procedures for the collection of information including:

• statistical methodology for stratification and sample selection,

• estimation procedure,

• degree of accuracy needed for the purpose described in the justification,

• unusual problems requiring specialized sampling procedures

Agricultural Resource Management Survey - The annual ARMS collects production practices and cost of production data on selected commodities and also detailed whole farm financial information from a representative sample of farms and ranches across the country. To accomplish this, the ARMS is conducted in three data collection phases. In many ways, the three phases can be viewed operationally as independent surveys. However, the power of the ARMS design is that data across phases are related and can be combined and analyzed. Estimated sample sizes are shown in the table for Item A.12.

The ARMS Phase I is conducted from May through July, and it collects general farm data such as crops grown, livestock inventory, and value of sales. These data are used to qualify or screen farms for the other phases. The mail and telephone forms of the questionnaire are attached (two versions to cover all questions); this is the Integrated Survey Screening form used for both ARMS, and the Fruit & Vegetable Chemical Use Survey.

The ARMS Phase II is conducted from September through December. This phase collects data associated with agricultural production practices (field operations, pest management practices, etc.), resource use (pesticide applications, fertilizer and nutrient application, types of equipment used, etc.), and variable costs of production for specific commodities. The respondent is given an information booklet for each crop with code definitions and conversion tables to help complete the questionnaire. The Phase II advance letter and flyer, respondent booklets, questionnaires, and telephone quality control sheet are attached.

The ARMS Phase III is conducted from January through April following the survey reference year to enable collection of full year financial data. This phase collects whole farm finance and operator characteristics information. The Phase III advance letter, respondent booklet, two types of cost questionnaires, telephone quality control sheet, 15-State mail core cost questionnaire, and additional commodity version questions are attached. Some of these respondents will be asked to complete a commodity-specific report to obtain financial, resource use, and cost of production data for the selected commodity and the entire farming operation. It is vital that operators who are selected for both the second and final phase complete both phases, so that we can collect data for the entire crop production process (physical activities and financial costs). Data from both phases provide the link between agricultural resource use and farm financial conditions; this is a cornerstone of the ARMS design. There commodity-specific versions consist of the Core questionnaire with appropriate customization of questions with a general scope as shown in example crop and livestock questionnaires. Due to the commodity mix for 2008 ARMS III, the goal and lines of questioning for the 2008 ARMS Phase III tobacco, biofuels, and cow/calf versions are attached. As these questionnaires are still under development, the actual questionnaires will be submitted to OMB by October 1, before the January 2009 ARMS Phase III data collection.

Fruit and Vegetable Chemical Use Surveys - The fruit and vegetable surveys target operators with selected commodities. Vegetable operations are screened as described above while fruit operations are selected from the NASS list frame. Only active operations with the crops of interest become part of the population for the fall survey.

Fruit and Vegetable producers selected for the survey are asked to complete an interview with questions pertaining to whole farm acreage and production, chemical products used and application rates, pest management practices, organic practices, and operator characteristics. Collection of fruit or vegetable chemical use data begins in early October. The two questionnaire versions, along with advance letters, respondent booklets, and a telephone quality control sheet are attached.

Post-harvest Chemical Use Surveys - Data collection for this survey is conducted by personal interview after the target commodity has been harvested. Sample sizes and commodities for the next 3 years are shown in the table for Item A-12. An introductory letter from the State Field Offices about the survey is mailed to the selected operations prior to the survey. Normally, an endorsement from representatives of the commodity industry requests their membership to cooperate on the survey. The advance letter, questionnaire, respondent booklet, and telephone quality control sheet are attached.

Contractor Expense Surveys - These surveys are used to collect average contractor expenses for the four commodity groups. The sample size will vary from State to State, dependent upon the number of contractors and the amount of influence they each have on the market in their respective States.

NASS Field Office staff will receive a Survey Administration Manual which provides detailed aspects of the survey data collection and editing process. Field enumerators in each State will be given an Interviewer’s Manual.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

NASS increased enumerator training on soft refusal conversion and on interviewing skills

A pilot study was conducted in 2004 to raise awareness of the importance of the ARMS in an attempt to increase response rates. A one-hour program was produced and televised nationally in September 2004 and February 2005 to promote the 2004 ARMS. Results were that 11% of respondents completing the 2004 ARMS phase III economic enterprise (face-to-face) version indicated they had seen the program. Another one-hour program was produced and televised during similar time frames for the 2005 ARMS.

Enumerator incentives were piloted in 3 States for the 2004 ARMS phase III enterprise economic face-to-face version resulting in an increase in response rates of over 10%. An additional research study of respondent incentives was conducted using the core economic 2004 ARMS phase III version resulting in an increase in response rates of nearly 7%. A respondent incentive research pilot was conducted for the 2005 ARMS phase III enterprise face-to-face version. As a result NASS requested and implemented non-monetary incentives for the 2006 ARMS Phase III non-core sample (face to face interviews). The results were favorable.

NASS is concerned about the long-term effects of respondent incentive use and will track and measure the long-term effects. Data have been collected for several years and analysis will begin in FY2009. Thus, monetary respondent incentives will remain in limited use until the full impact of ARMS incentives on willingness to respond to other NASS surveys can be measured.

RESEARCH: The findings from 2007 NASS research report on: Assessing the Effect of Calibration on Non-response Bias in the 2005 ARMS Phase III Sample Using Census 2002 Data are summarized in the next two paragraphs.

Records sampled for the 2005 ARMS Phase III were matched with those from the 2002 Census of Agriculture, and means of census data were calculated for matching records which had also provided 2002 expenditure data for the census. Nonresponse bias in ARMS data was assessed, using census data as a proxy, in terms of the degree to which the mean based on all sample cases versus respondent cases differed. Three means were computed and compared across 20 regions in order to assess relative bias: 1) the mean of all matching cases using base sampling weights, 2) the mean for only matching ARMS respondents using the same base sampling weights, and 3) the mean for matching ARMS respondents using the sampling weights as adjusted through calibration.

Using 17 “study variables,” relative bias of the mean was assessed using a variation of the formula provided by OMB in Guideline 3.2.9. Although significant biases were exhibited in 9 of 17 variables using the 2005 ARMS III base sampling weights, the 2005 ARMS III calibration weights were able to reduce the bias so that it was no longer significantly different from zero (p < .05) in almost 90% (8/9) of the study variables. For this analysis the calibration process varied slightly from that of the 2005 ARMS III, in that egg and milk production were not included, since they were not accounted for by the 2002 Census; this may in part account for the one variable, fertilizer expenses, still demonstrating a significant level of bias after the use of calibrated weights. This study suggests that the process of calibration is an effective tool in reducing nonresponse bias levels, so they are no longer significantly different from zero.

Research is underway to see if sample size reductions can be gained using more advanced calibration techniques during ARMS phase III summarization processes.

Several research projects will be launched based on recommendations of the National Academies of Sciences, Committee on National Statistics (NAS-CNSTAT) comprehensive review of the ARMS. Copies of the November 2007 report are available via the web at: http://books.nap.edu/openbook.php?record_id=11990&page=R1.

The high priority areas focus on: respondent burden reduction strategies, response rate improvement, and improvement of data quality. Recommended action items may impact other NASS surveys as well as the next Census of Agriculture.

The following are the non-response adjustments for ARMS II and Chemical Use.

Unit nonresponse in the Fruit, Vegetable, and Post-Harvest Chemical Use Surveys as well as ARMS II is accounted for using reweighting. The records are stratified by state and size group, and a nonresponse adjustment is calculated as the sample size divided by the number of completed reports. This process redistributes the survey weights for the nonrespondents to the usable records. For ARMS II, the weights are then scaled so that the expanded total of the target commodity’s planted acres is equal to the planted acreage number set by the ASB (Agricultural Statistics Board). For the Fruit and Vegetable Chemical Use Surveys, a calibration program adjusts the weights so that the expanded planted acreage totals for each target crop matches the planted acreage set by the ASB. Scaling only occurs in the Post-Harvest Chemical Use survey if appropriate data from the ASB is available.

Item nonresponse in ARMS II and Fruit, Vegetable, and Post-Harvest Chemical Use Surveys is handled with mean imputation. Missing fertilizer and pesticide rates are replaced with average rates by state, commodity and product code (or nutrient for fertilizer rates). If no records exist in that category, then the groups are collapsed. No other items in these surveys are imputed.

The following are non-response adjustment for ARMS III.

Non-response is taken into account in the ARMS III sample allocations by State. ARMS III sets a target for positive usable responses by State and ARMS Region; the sample size is adjusted based on a 5-year historical response rate to achieve the targeted usable rates.

Unit nonresponse in the ARMS III all version weights is adjusted for using calibration. The calibration process modifies the survey weights so that certain targets are met. NASS uses official estimates of farm numbers, corn, soybean, wheat, cotton, fruit and vegetable acres as well as cattle, milk production, hogs, broilers, eggs and turkeys as calibration targets. For example, after calibration the weighted sum of the survey data will equal the NASS estimate for corn acres. The weights for the ARMS III commodity specific versions (Versions 2, 3, and 4) are scaled so that the targeted commodities’ expansion is equal to the number set by the ASB.

Item nonresponse in ARMS III is dealt with using machine imputation. About 150 survey variables that are critical to NASS analysis and/or ERS work are imputed using positive data from current survey respondents. Imputation is based on groups of operations by region, state, economic sales class, and type of farm. Further, groups are collapsed when not enough observations are present for a particular item.

4. Describe any tests of procedures or methods to be undertaken.

NASS has experience from previous chemical use and economic surveys that have been beneficial in designing the surveys explained in this docket. Pre-testing of restructured or rotated in sections of questionnaires will be done annually for each survey, refining the data collection instruments each year. The results of these tests and subsequent methods will be incorporated into the operational design. NASS has contracted with Washington State University to provide assistance and guidance in the development of the various questionnaire versions to incorporate lessons learned in recent years and complete testing to determine feasibility for a mail version of the enterprise ARMS Phase III questionnaire.

Additional questionnaire modifications may be tested in the process of assessing and incorporating suggestions from a review of the ARMS by the National Academies of Science. Results of this review are available via the web at:

http://books.nap.edu/openbook.php?record_id=11990&page=R1

In 2007, FSA data were used in lieu of ARMS Phase I Screening Survey to indicate cotton planted during the current year. NASS will continue to test the use of FSA data when appropriate in lieu of screening to reduce respondent burden.

Response improvement techniques will continue to be researched and tested to improve response rates in the area of questionnaire improvement, respondent relationship building, and soft refusal conversion techniques.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The sampling plans are developed by NASS. Questionnaire design, data edit, and initial summarization will also be completed by NASS, and for ARMS, in consultation with ERS. Washington State University provided design expertise for the development of the Web-based version of the Core Phase III questionnaire.

The sample size for each State is determined by the Sampling Branch, Census and Survey Division; Branch Chief is William Iwig, (202) 720-3895.

Data collection is carried out by NASS Field Offices; Deputy Administrator for Field Operations is Marshall Dantzler, (202) 720-3638.

The NASS survey statisticians in Headquarters listed below are responsible for coordination of sampling, questionnaires, data collection, and other Field Office support. Branch Chief is Norman Bennett, (202) 720-2248, Section Head is Dania Ferguson (202) 720-5921. The NASS commodity statisticians in Headquarters listed below are responsible for national summaries, analysis, and publication. Branch Chief is Kevin Barnes, (202) 720-6146, Section Heads are Mark R. Miller (202)720-0684 and Kevin Hintzman (202) 690-3223.

Survey	Survey Statisticians Environmental and Economic Surveys Section, Survey Administration Branch, Census and Survey Division	Commodity Statisticians Environmental and Demographics Section; Environmental, Economics, and Demographics Branch, Statistics Division
ARMS I Screening	Vice Perry Game (202) 720-2248	Jerry Campbell (202) 720-5581
ARMS II	Vice Stacy Wills (202) 720-2248
ARMS III	Rich Hopper (202) 720-2206	Richard Barton (202) 690-3347
Fruit Chemical Use	(Vacant) (202) 720-2248	Doug Farmer (202) 720-7492
Vegetable Chemical Use
Post-harvest Chemical Use		Liana Cuffman (202) 690-0392
Contractor Expense Survey	Rich Hopper (202) 720-2206		Richard Barton (202)690-3347

June 2008