0140-lsf-15-SSB

1 Supporting Statement

LIST SAMPLING FRAME SURVEY

OMB No. 0535-0140

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection has been conducted previously, include the actual response rate achieved during the last collection.

The response universe consists of new agricultural establishments and potential agricultural establishments not previously included on the NASS List Sampling Frame. These potential new agricultural establishments are obtained from list sources related to the agricultural industry and rural community. Information from these list sources, do not provide enough information to classify the establishment as a farm or identify the type or size of the operation. This information is critical in developing efficient samples for various NASS program surveys.

We obtain some of the names and addresses of potential new farm operators from lists we acquire from commodity associations. Some of these lists have proven to be of marginal use. For example, some of the lists that we have received from floriculture, nursery, horticultural specialties, and equine associations have contained names of individuals who are hobbyists and join the associations so that they can receive monthly newsletters. Many of these individuals do not produce agricultural products. Due to limited data collection budgets we will limit phone follow-up on some of these groups of potential producers if we discover that the lists are only of marginal use. In 2016 the NACS survey will be conducted under this OMB approval as a voluntary survey. In 2017, the NACS survey will be conducted under the Census of Agriculture docket (0535-0226) as a mandatory survey in preparation for the 2017 Census of Agriculture (conducted in 2018).

2. Describe the procedures for the collection of information including:

• statistical methodology for stratification and sample selection,

• estimation procedure,

• degree of accuracy needed for the purpose described in the justification.

All available records that are potentially new farms are surveyed to obtain criteria data. No sampling is used. The majority of the data are collected using self-administered questionnaires, i.e., questionnaires sent to the respondent by mail. The National Agricultural Classification Survey (NACS) is used to identify new operators that need to be added to our List Frame. If anyone who completes this questionnaire identifies that they produce specialty crops they may receive a follow up interview, the List Sampling Farm Criteria (LSFC). We have included the Master Listing of the items that could appear in this questionnaire. The questionnaires will be customized to contain only the crops that are of interest for any particular State. This will keep the questionnaires as short as possible and to minimize respondent burden as much as possible. This master questionnaire version should help reduce the review and approval process for this docket. The Criteria for Horticultural Specialties (CHS) will be used for operations that indicate that they produce certain horticultural crops. This will help to improve the sampling for future floriculture or nursery type surveys. The two step survey process was developed to help reduce respondent burden. Since most US farmers do not produce specialty crops, we removed these commodities from the NACS and limited them to the LSFC and the CHS questionnaires.

Normal data collection procedures involve mailing a blank questionnaire to the target population along with instructions on how to respond via the internet if they choose to. We follow this up with a second mailing to non-respondents, and finally with a phone follow-up for the remaining non-respondents. If we discover that any of the lists of potential new farm or ranch operators, that we received from external sources, is out of date or not yielding good information we will reduce the frequency of our phone follow-up and concentrate more on the lists that are providing better results.

Reported data are reviewed and edited for reasonableness. No survey estimates are produced. The reported data are captured to the NASS List Sampling Frame for use in selecting samples for NASS surveys and selecting the Census Mail List.

Additional detailed information on the creation, maintenance and security of our List Frame, Census Frame, and Survey Frame can be found in the attached UNIX Privacy Impact Assessments (PIA)’s along with our System of Records Notice (SORN).

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

Since most criteria records are new to the list sampling frame the quality of the list source which the record originated has to be factored into the level and quality of response. NASS analyzes in scope rates, response rates and other quality factors by list source to maximize not only the response rate but also to maximize the number of new in scope records for the list sampling frame by targeting quality list sources.

Many of the recipients of these screening surveys are not farm operators or do not consider themselves as farm operators due to their small agricultural production. The National Agricultural Classification Survey (NACS), is primarily used to survey new additions from general large list sources, it is designed to encourage responses from all recipients, including those who do not consider themselves as farm operators.

Other applications are utilized to improve the quality prior to mailing. Record linkage applications are utilized to remove duplication and identify deceased individuals. Also, US Postal Service’s national change of address database and locatable address conversion systems are utilized to improve the mailability of the new records.

The accuracy and reliability of the information received is reflected in the quality of the operational survey estimates. The sample designs for these surveys are based on list frame control data which are collected from previous surveys, including these criteria surveys. The accuracy of survey estimates continues to meet target goals. In general, the accuracy and reliability of the control data collected from these criteria surveys is comparable to the quality of the control data collected on other surveys.

Data collection for this group of surveys involves two attempts to collect the information by mail or internet with phone follow-up for any remaining non-respondents. Due to cost constraints we do not attempt these surveys by field enumeration. In some years budget cuts have forced us to reduce phone follow-up to only target groups or target States.

4. Describe any tests of procedures or methods to be undertaken.

NASS prepares long-range plans for improving the list frame and evaluates its coverage and composition each year. The annual List Frame Coverage Summary report (for internal use) contains an evaluation of the NASS list sampling frame coverage of the U.S. farm population and U.S. land in farms based on area frame data. The annual Composition of the List Frame report (internal use) contains an evaluation of the composition of control data on the list sampling frame. In it, current data are compared to previous years by size for more than 200 control data items to measure list building performance and tracks changes in composition.

The population of farm operators is constantly changing. Research has shown that there is a 35-40% farm turnover rate between one census and the next. Hence, there is a continual effort within NASS to keep the list of farm operators up to date. NASS obtains lists of potential farm operations from a variety of sources. Most of these outside source lists are matched to NASS’s list frame using record linkage. Through this processing, new potential farm records are identified and added to NASS’s list frame. Furthermore, existing records on NASS’s frame that are thought to no longer be in business are updated such that they are included in the Census if they match an incoming potential farm record.

NASS has a goal to produce an efficient Census Mail List (CML). This means that the mail list is as small as possible while still having a high coverage rate. This goal is difficult to achieve because there is a correlation between the number of records sent out and the level of coverage. An efficient mail list requires that a high percentage of the records mailed are true farms. The 2012 Census of Agriculture found that the NASS list frame covered approximately 88% of the farms in the United States. The following table shows the Census Mail List sizes and respective coverage rates for the past 5 censuses.

Year	CML (mil)	Farm Coverage
1992	3.55	88.30%
1997	3.19	86.70%
2002	2.85	82.10%
2007	3.19	83.76%
2012	3.01	88.00%

After each census is complete, analysis is conducted to measure the effectiveness of each outside source list. The number of new records from each list is measured. The records are followed to obtain an in-scope rate for each list. Each list is also analyzed to see if it contributes to improvements in coverage of minority operations or coverage of specific commodities. The analysis results are used to decide which lists will be processed again in the future.

Electronic Data Reporting (EDR) is being used by more and more respondents. NASS is continuing to research the results of EDR, specifically respondent characteristics. This will aid in expanding coverage for all farm categories and minimize costs associated with applied survey management practices.

Measures have been taken to improve name and address quality. The records for each NACS survey are processed through the National Change of Address and Locatable Address Conversion System prior to mailing. Records with poor mailability scores are researched to see if better addresses can be obtained. Undeliverable As Addressed (UAA) rates are calculated for each outside source list. Lists with high UAA rates may not be processed in the future. Records on the list with missing or invalid phone numbers are matched against a nationally available telephone database to obtain as many phone numbers as possible.

NASS will analyze and research mailability scores provided by postal software to optimize mail effectiveness. These findings will be utilized to reduce mail costs for all NASS surveys. Additionally, NASS will continue to perform research on Electronic Data Reporting, specifically respondent characteristics. This will aid in expanding coverage for all farm categories and minimize costs associated with applied survey management practices. On the NACS questionnaire, NASS will incorporate the research conducted by Don Dillman at Washington State University on questionnaire layout (Agricultural Resource Management Surveys (0535-0218) and the Census of Agriculture surveys (0535-0226)).

5. Provide the name and telephone number of individuals consulted on

statistical aspects of the design and the name of the agency unit, contractor(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The completeness and efficiency of each State's list frame is reviewed and maintained by the agency's Frames Maintenance Groups located at the National Operations Center in St. Louis, Missouri and is overseen by the Deputy Director Brad Parks, (314) 595-9502.

Sampling and Frame Development Section is located in our Sampling, Editing and Imputation Methodology Branch of the Methodology Division; Branch Chief is Mark Apodaca (202) 720-5805.

Data collection is carried out by NASS State Field Offices; Eastern Field Operations Director Jay Johnson, (202) 720-3638 and the Western Field Operations Director Kevin Barnes, (202) 720-8220.

November, 2015