0140-lsf-18-SSB

1 Supporting Statement

LIST SAMPLING FRAME SURVEY

OMB No. 0535-0140

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection has been conducted previously, include the actual response rate achieved during the last collection.

NASS builds its List Frame by obtaining farm lists from various sources. These farm lists go through a record linkage process to ascertain if the farm is new to NASS’s List Frame or overlaps with an active or inactive farm (operation).

Operations that are new or overlap to inactive operations that have potential for agricultural activity are referred to as list criteria records: operations with unknown agricultural activity. The overlap group is further divided into subgroups using administrative data.

NASS administers a National Agricultural Classification Survey (NACS) to list criteria records. Budget resources dictates if some or all groups of list criteria records will be sampled. The primary purpose of NACS is to screen for general agricultural activity. Operations that report agricultural activity become part of NASS’s List Frame.

2. Describe the procedures for the collection of information including:

• statistical methodology for stratification and sample selection,

• estimation procedure,

• degree of accuracy needed for the purpose described in the justification.

The majority of the data are collected using self-administered questionnaires, i.e., questionnaires sent to the respondent by mail, with the option to complete the survey by internet or by mail. If anyone who completes this questionnaire identifies that they produce specialty crops they may receive a follow up interview such as the Criteria for Horticultural Specialties (CHS). The CHS will be used for operations that indicate that they produce certain horticultural crops. This will help to improve the sampling for future floriculture or nursery type surveys. The two step survey process was developed to help reduce respondent burden. Since most US farmers do not produce specialty crops, we removed these commodities from the NACS and limited them to the CHS questionnaires.

Normal data collection procedures involve mailing a questionnaire and instructions on how to respond via the internet if they choose to. We follow this up with a second mailing to non-respondents, and finally with a phone follow-up for the remaining non-respondents (budget permitting). If we discover that any of the lists of potential new farm or ranch operators, that we received from external sources, is out of date or not yielding good information we will reduce the frequency of our phone follow-up and concentrate more on the lists that are providing better results.

Reported data are reviewed and edited for reasonableness. No survey estimates are produced. The reported data are captured to the NASS List Frame.

Additional detailed information on the creation, maintenance and security of our List Frame, Census Frame, and Survey Frame can be found in the attached UNIX Privacy Impact Assessments (PIA)’s along with our System of Records Notice (SORN).

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

Since most criteria records are new to the list sampling frame the quality of the list source which the record originated has to be factored into the level and quality of response. NASS analyzes in scope rates, response rates and other quality factors by list source to maximize not only the response rate but also to maximize the number of new in scope records for the list sampling frame by targeting quality list sources.

Many of the recipients of these screening surveys are not farm operators or do not consider themselves as farm operators due to their small agricultural production. The National Agricultural Classification Survey (NACS), is primarily used to survey new additions from general large list sources, it is designed to encourage responses from all recipients, including those who do not consider themselves as farm operators.

Other applications are utilized to improve the quality prior to mailing. Record linkage applications are utilized to remove duplication and identify deceased individuals. Also, US Postal Service’s national change of address database and locatable address conversion systems are utilized to improve the mailability of the new records.

The accuracy and reliability of the information received is reflected in the quality of the operational survey estimates. The sample designs for these surveys are based on list frame control data which are collected from previous surveys, including these criteria type surveys. The accuracy of survey estimates continues to meet target goals. In general, the accuracy and reliability of the control data collected from these criteria surveys is comparable to the quality of the control data collected on other surveys.

Data collection for this group of surveys involves two attempts to collect the information by mail or internet with phone follow-up for any remaining non-respondents. Due to cost constraints we do not attempt these surveys by field enumeration. In some years budget cuts have forced us to reduce phone follow-up to only target groups or target States.

4. Describe any tests of procedures or methods to be undertaken.

NASS prepares long-range plans for improving the list frame and evaluates its coverage and composition each year. The annual List Frame Coverage Summary report (for internal use) contains an evaluation of the NASS list sampling frame coverage of the U.S. farm population and U.S. land in farms based on area frame data. The annual Composition of the List Frame report (internal use) contains an evaluation of the composition of control data on the list sampling frame. In it, current data are compared to previous years by size for more than 200 control data items to measure list building performance and tracks changes in composition.

The population of farm operators is constantly changing. Research has shown that there is a 35-40% farm turnover rate between one census and the next. Hence, there is a continual effort within NASS to keep the list of farm operators up to date. NASS obtains lists of potential farm operations from a variety of sources. Most of these outside source lists are matched to NASS’s list frame using record linkage. Through this processing, new potential farm records are identified and added to NASS’s list frame. Furthermore, existing records on NASS’s frame that are thought to no longer be in business are updated such that they are included in the Census if they match an incoming potential farm record.

NASS has a goal to produce an efficient Census Mail List (CML). This means that the mail list is as small as possible while still having a high coverage rate. This goal is difficult to achieve because there is a correlation between the number of records sent out and the level of coverage. An efficient mail list requires that a high percentage of the records mailed are true farms. The 2012 Census of Agriculture found that the NASS list frame covered approximately 88% of the farms in the United States. The following table shows the Census Mail List sizes and respective coverage rates for the past 5 censuses.

Year	CML (mil)	Farm Coverage
1992	3.55	88.30%
1997	3.19	86.70%
2002	2.85	82.10%
2007	3.19	83.76%
2012	3.01	88.00%
2017 1/

^1/ The 2017 Census of Agriculture publication will be available Feb. 2019.

After each census is complete, analysis is conducted to measure the effectiveness of each outside source list. The number of new records from each list is measured. The records are followed to obtain an in-scope rate for each list. Each list is also analyzed to see if it contributes to improvements in coverage of minority operations or coverage of specific commodities. The analysis results are used to decide which lists will be processed again in the future.

Electronic Data Reporting (EDR) is being used by more and more respondents. NASS is continuing to research the results of EDR, specifically respondent characteristics. This will aid in expanding coverage for all farm categories and minimize costs associated with applied survey management practices.

Measures have been taken to improve name and address quality. The records for each NACS survey are processed through the National Change of Address and Locatable Address Conversion System prior to mailing. Records with poor mailability scores are researched to see if better addresses can be obtained. Undeliverable As Addressed (UAA) rates are calculated for each outside source list. Lists with high UAA rates may not be processed in the future. Records on the list with missing or invalid phone numbers are matched against a nationally available telephone database to obtain as many phone numbers as possible.

NASS will analyze and research mailability scores provided by postal software to optimize mail effectiveness. These findings will be utilized to reduce mail costs for all NASS surveys. Additionally, NASS will continue to perform research on Electronic Data Reporting, specifically respondent characteristics. This will aid in expanding coverage for all farm categories and minimize costs associated with applied survey management practices. On the NACS questionnaire, NASS has incorporated the research conducted by Don Dillman at Washington State University on questionnaire layout (Agricultural Resource Management Surveys (0535-0218) and the Census of Agriculture surveys (0535-0226)).

In 2019 the NACS questionnaire will have three versions. Each version has slightly different screening questions and layout of the screening questions. Since this is primarily a self-administered survey, NASS is testing to see if respondents are more likely to respond to one version over another.

5. Provide the name and telephone number of individuals consulted on

statistical aspects of the design and the name of the agency unit, contractor(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The completeness and efficiency of each State's list frame is reviewed and maintained by the agency's Frames Maintenance Groups located at the National Operations Center in St. Louis, Missouri and is overseen by the Deputy Director Brad Parks, (314) 595-9502.

Sampling and Frame Development Section is located in our Sampling, Editing and Imputation Methodology Branch of the Methodology Division; Branch Chief is Mark Apodaca (202) 720-5805.

Data collection is carried out by NASS State Field Offices, the Field Operations Director is Jay Johnson, (202) 720-3638.

December 2018