0140-lsf-09-SSB

Supporting Statement

LIST SAMPLING FRAME SURVEY

OMB No. 0535-0140

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection has been conducted previously, include the actual response rate achieved during the last collection.

The response universe consists of new agricultural establishments and agricultural establishments not previously included on the NASS List Sampling Frame. These potential new agricultural establishments are obtained from list sources related to the agricultural industry and rural community. Information from these list sources, do not provide enough information to classify the establishment as a farm or identify the type or size of the operation. This information is critical to develop efficient samples for various NASS program surveys.

1/ Estimated

2/ Actual numbers. Over the last few years this survey has been conducted under several different names. Agricultural Identification Survey, National Criteria Survey, and now the National Agricultural Classification Survey.

2. Describe the procedures for the collection of information including:

• statistical methodology for stratification and sample selection,

• estimation procedure,

• degree of accuracy needed for the purpose described in the justification.

All available records that are potentially new farms are surveyed to obtain criteria data. No sampling is used. The majority of the data are collected using self-administered questionnaires, i.e., questionnaires sent to the respondent by mail. The master questionnaire is attached to this docket. The National Criteria Survey (NCS) and the List Sampling Farm Criteria questionnaires use questions from this master list in order to collect sufficient data to classify each list record’s eligibility to be sampled in annual surveys. Non-response telephone follow-up is used to increase response rates.

Reported data are reviewed and edited for reasonableness. No survey estimates are produced. The reported data are posted to the NASS List Sampling Frame for use in selecting samples for NASS surveys and selecting the Census Mail List.

Additional detailed information on the creation and maintenance of our List Frame, Census Frame, and Survey Frame can be found in the in the three attached Privacy Impact Assessments (PIA)’s.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

Since most criteria records are new to the list sampling frame the quality of the list source which the record originated has to be factored into the level and quality of response. NASS analyzes in scope rates, response rates and other quality factors by list source to maximize not only the response rate but also to maximize the number of new in scope records for the list sampling frame by targeting quality list sources.

Many of the recipients of these screening surveys are not farm operators or do not consider themselves as farm operators due to their small agricultural production. The front page of the National Criteria Survey, which is primarily used to survey new additions from general large list sources, is designed to encourage responses from all recipients, including those who do not consider themselves as farm operators.

Other applications are utilized to improve the quality prior to mailing. Record linkage applications are utilized to remove duplication and identify deceased individuals Also US Postal Service national change of address database and locatable address conversion system are utilized to improve the Mailability of the new records.

The accuracy and reliability of the information received is reflected in the quality of the operational survey estimates. The sample designs for these surveys are based on list frame control data which are collected from previous surveys, including these criteria surveys. The accuracy of survey estimates continues to meet target goals. In general, the accuracy and reliability of the control data collected from these criteria surveys is comparable to the quality of the control data collected on other surveys.

During the survey there are two mailings with a phone follow-up.

4. Describe any tests of procedures or methods to be undertaken.

NASS prepares long-range plans for improving the list frame and evaluates its coverage and composition each year. The annual List Frame Coverage Summary report (for internal use) contains an evaluation of the NASS list sampling frame coverage of the U.S. farm population and U.S. land in farms based on area frame data. The annual Composition of the List Frame report (internal use) contains an evaluation of the composition of control data on the list sampling frame. In it, current data are compared to previous years by size for more than 200 control data items to measure list building performance and tracks changes in composition.

The population of farm operators is constantly changing. Research has shown that there is a 35-40% farm turnover rate between one census and the next. Hence, there is a continual effort within NASS to keep the list of farm operators up to date. NASS obtains lists of potential farm operations from a variety of sources. Most of these outside source lists are matched to NASS’s list frame using record linkage. Through this processing, new potential farm records are identified and added to NASS’s list frame. Furthermore, existing records on NASS’s frame that are thought to no longer be in business are updated such that they are included in the Census if they match an incoming potential farm record.

NASS has a goal to produce an efficient Census Mail List. This means that the mail list is as small as possible while still having a high coverage rate. This goal is difficult to achieve because there is a correlation between the number of records sent out and the level of coverage. An efficient mail list requires that a high percentage of the records mailed are true farms. The 2007 Census of Agriculture found that the NASS list frame covered approximately 84% of the farms in the United States. The following table shows the Census Mail List sizes and respective coverage rates for the past 5 censuses.

Year	CML (mil)	Farm Coverage
1987	4.1	89.70%
1992	3.55	88.30%
1997	3.19	86.70%
2002	2.85	82.10%
2007	3.19	83.76%

After each census is complete, analysis is conducted to measure the effectiveness of each outside source list. The number of new records from each list is measured. The records are followed to obtain an in-scope rate for each list. Each list is also analyzed to see if it contributes to improvements in coverage of minority operations or coverage of specific commodities. The analysis results are used to decide which lists will be processed again in the future.

The National Agricultural Classification Survey (NACS) is used to screen farms in preparation for the Census. The survey focuses on gathering the minimum amount of data needed to determine whether an operation should be included in the Census Mail List. The NACS is a shorter survey than the Census. It is more efficient to screen out potential farm operations using the NACS than the Census. The following table shows the results of this survey in preparation for the 2007 Census of Agriculture.

Survey	Record Count	Returns	Response Rate	In Scope Rate
Oct-04	171,717	55,612	45%	59%
Jun-05	116,449	47,979	42%	55%
Nov-05	133,179	71,069	57%	67%
Mar-06	66,995	29,312	44%	39%
Jun-06	193,668	90,951	52%	57%
Dec-06	1,065,311	634,779	64%	75%
TOTAL	1,747,319	929,702	55%	69%

The first survey in preparation for the 2012 Census was mailed in January 2010. It is expected that subsequent surveys will also be mailed in January 2011 and January 2012. The results of surveys in preparation for the 2007 Census were evaluated and improvements have been made. The planned scheduling and types of records targeted in each survey have been designed based on the results of the earlier surveys. The NACS questionnaire has also been updated based on things learned through previous surveys. Data received from previous surveys were not run through an edit process. Beginning in 2010 reported data will be run through an edit. It is hoped that this will greatly improve the accuracy of the results.

Electronic Data Reporting (EDR) is being used for more and more reporters. NASS is continuing to research the results of EDR, specifically respondent characteristics. This will aid in expanding coverage for all farms categories and minimize costs associated with applied survey management practices.

Measures have been taken to improve name and address quality. The records for each NACS survey are processed through the National Change of Address and Locatable Address Conversion System prior to mailing. Records with poor mailability scores are researched to see if better addresses can be obtained. Undeliverable As Addressed (UAA) rates are calculated for each outside source list. Lists with high UAA rates may not be processed in the future. Records on the list with missing or invalid phone numbers are matched against a nationally available telephone database to obtain as many phone numbers as possible.

During 2009, some research was conducted by the NASS Research and Development Division to evaluate different options for the design and wording of the national criteria survey questionnaire that might improve responses from recipients who do not consider themselves as farm operators. Any recommended questionnaire changes from this research will be considered for future surveys.

NASS will analyze and research mailability scores provided by postal software to optimize mail effectiveness. These findings will be utilized to reduce mail costs for the 2012 Census of Agriculture. Additionally, NASS will continue to perform research on Electornic Data Reporting, specifically respondent characteristics. This will aid in expanding coverage for all farm categories and minimize costs associated with applied survey management practices. On the NACS questionnaire, NASS will incorporate the research conducted by Don Dillman at Washington State University on questionnaire layout (Agricultural Resource Management Surveys (0535-0218) and the Census of Agriculture surveys (0535-0226)).

5. Provide the name and telephone number of individuals consulted on

statistical aspects of the design and the name of the agency unit, contractor(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The efficiency of each State's list frame is reviewed by the agency's List Frame and Sample Design Sections of the Sampling Branch, Census and Survey Division; Branch Chief is Bill Iwig, (202)720-3895.

Data collection is carried out by NASS State Field Offices; Deputy Administrator for Field Operations is Marshall Dantzler, (202)720-3638.

June, 2009

Revised January, 2010

5