Part B -small-scale livestock 10-22

B. Collections of Information Employing Statistical Methods

1. Describe the potential respondent universe and any sampling or other respondent selection methods to be used.

The potential respondent universe of the Small-Scale Livestock Operations study is all producers who are on the NASS list frame with total annual sales¹ between $10,000 and $500,000, and with a farm type² indicating that the primary agricultural enterprise is livestock, poultry, aquaculture or other animals/animal products. Based on the NASS list frame, there are about 564,000 operations that meet this criterion. Based on previous NAHMS studies, the estimated response rate for the Small-Scale Livestock Operations Study is 70 percent (Appendix B).

2. Describe the procedures for the collection of information including:

0. Statistical methodology for stratification and sample selection:

A total of 16,000 small-scale livestock operations with total annual sales between $10,000 and $500,000 will be selected from NASS’ list frame from a population of approximately 564,000 operations. The sample will be selected as a systematic random sample within the following strata:

1. Operations with annual sales between $10,000 and $99,999 and 1 to 3 commodities present³

2. Operations with annual sales between $10,000 and $99,999 and 4 or more commodities³ present

3. Operations with annual sales between $100,000 and $249,999 (1 or more commodities³ present)

4. Operations with annual sales between $250,000 and $499,999 (1 or more commodities³ present)

Total value of annual sales and number of commodities present are based NASS list frame control data.

Sampling methodology— 6,800 operations will be selected in stratum 1 above; 2,340 operations in stratum 2; 4,860 operations in stratum 3 and 2,000 operations in stratum 4 (see Appendix C for sample size determination). A relatively small sample will be taken in stratum 4, since the main focus of this study is operations that meet the USDA definition of small farms (under $250,000 in annual sales).

To ensure geographic coverage, NASS list frame data will be sorted by state, county and by stratum. Then, a systematic random sample will be taken within each stratum. The number of farms selected will be proportional to the number of farms in the state/stratum.

0. Estimation procedure:

The sampling design is a stratified random sample with unequal probabilities of selection between strata. The statistical estimation will be undertaken using either SAS survey procedures or SUDAAN. Both software packages use a Taylor series expansion to estimate appropriate variances for the stratified, weighted data.

0. Degree of accuracy needed for the purpose described in the justification:

The overall NAHMS program goal is to develop descriptive statistics with a coefficient of variation less than 20 percent. Analytical studies are designed with a goal of 80 percent power to detect odds ratios of greater than 2 for the factors identified as most important by the industry. Appendix C (Sample size estimates for three levels of prevalence using 95% confidence) shows sample size estimates for a 70% response rate without adjusting for design effect.

However, the complex survey design typically will result in variances that are inflated. In previous NAHMS studies, design effects ranged from less than one up to almost 6 for the selected variables.

For the sample of 9,140 operations with annual sales between $10,000 and $99,999 we can expect approximately 6,398 good NASS responses (70% response rate). Assuming a design effect = 2, this will allow national estimates of approximately 50% +/- 1.7%, 20% +/- 1.4%, 10% +/- 1.0%. Assuming a design effect = 3, this will allow national estimates of approximately 50% +/- 2.1 %, 20% +/- 1.7%, 10% +/- 1.3%.

For the sample of 4,860 operations with annual sales between $100,000 and $249,999 we can expect approximately 3,402 good NASS responses (70% response rate). Assuming a design effect = 2, this will allow national estimates of approximately 50% +/- 2.4%, 20% +/- 1.9%, 10% +/- 1.4%. Assuming a design effect = 3, this will allow national estimates of approximately 50% +/- 2.9 %, 20% +/- 2.3%, 10% +/- 1.7%.

For the sample of 2,000 operations with annual sales between $250,000 and $499,999 we can expect approximately 1,400 good NASS responses (70% response rate). Assuming a design effect = 2, this will allow national estimates of approximately 50% +/- 4%, 20% +/- 3%, 10% +/- 2%.

For regional reporting, the population will be divided into 4 approximately equal regions. Assuming a 70% response rate, we can expect approximately 2,800 good responses from each region ((16,000/4)*0.7). Assuming a design effect = 2, the expected level of precision for regional estimates would be 50% +/- 2.6%, 20% +/- 2.1%, 10% +/- 1.6%. Assuming a design effect = 3, the expected level of precision for regional estimates would be 50% +/- 3.2 %, 20% +/- 2.6%, 10% +/- 1.9%.

• Unusual problems requiring specialized sampling procedures and data collection cycles:

There are no unusual problems requiring specialized sampling procedures and data collection cycles.

3. Describe methods to maximize response rates and to deal with issues of non-response:

Study Design:

0. Minimizing collection of data to that which is absolutely necessary.

0. Numerous contacts and collaborative efforts have been made to identify the information needs and how best to ask for that information via the small-scale operations questionnaire.

• Mailing the questionnaire, the second request mailing, and telephone follow up will boost the response rate to the estimated 70 percent. The estimated 70 percent response rate is based upon consultation with NASS and previous experience using U.S. Mail with a telephone follow-up for data collection. Several previous NAHMS studies have used this procedure, and obtained response rates for survey completion ranging from 32.5% to 69.3% (Appendix B). Although over 70% of operations gave useable inventory data in each of these 3 previous studies, a considerable percentage reported zero inventory on hand for the species of interest, and therefore did not complete the questionnaire. We expect a higher completion rate for this study than for the General Goat Management Report (GGMR CATI). The GGMR targeted very small operations with less than 10 goats, a population that is expected to be more likely to go in or out of business than the population targeted for the Small-Scale Livestock Operations study. It is estimated that the response rate for the Small-Scale Livestock Operations study population will be approximately 70%, and may be slightly higher since operations will be eligible if they had any animal inventory in the last 12 months, as opposed to requiring inventory of one specific animal species like the previous studies.

0. NAHMS will develop a training CD for NASS enumerators that explains the purpose of the study and addresses anticipated difficulties with questions. Each enumerator will receive a CD.

Non-response:

0. The study is supported by agricultural extension professionals who have contributed to the study development. As operations in the target population are not part of a unified industry, there is no specific entity from which to seek support.

0. The questionnaire will be sent out via U.S. Mail with a cover letter and brochure announcing the study to give respondents more information on the study and why participation is important.

0. The questionnaire and cover letter will be sent out again, stamped as a “Second Request”, two weeks after the initial questionnaire is sent out if a response is not received.

0. If no response is received one month after the initial questionnaire is mailed out (two weeks after the second request) a NASS enumerator will contact the producer via telephone and attempt to get the producer to complete the questionnaire or schedule a convenient time to complete the questionnaire.

0. Producers will be called up to 7 times before they are listed as inaccessible.

Non-response adjustment:

0. Response rates, given the methods described above, are expected to be approximately 70% for this study. If the respondents differ substantially from the non-respondents there will be the potential for bias. There are two approaches that we will use to examine for potential bias. First, NASS’ list frame control data will be available for both respondents and non-respondents to allow for examination of potential differences in the types of responding and non-responding producers. The information will include number of animals owned as well as the number of animals on the land operated, operation type, and State. Secondly, we can compare estimates from the study with available indicators from other sources. For example, although we do not publish estimates of animal inventory, the study results will allow us to make estimates that we can use to compare against NASS’ inventory estimates. We will compare our results to values available from the scientific literature. We believe there will be limited opportunities for comparison because little national data exist for the type of information that is to be collected.

• The sampling design necessitates the use of weights which reflect the initial sample selection probabilities (the inverse of the selection interval). Weights of non-respondents will be transferred to responding operations that are most similar based on available data. Within categories, the sum of weights of the non-respondents and respondents will be divided by the sum of the weights of the respondents only. This factor will be used to adjust the weights of the respondents within the category. All weights for non-respondents will be set to zero.

4. Describe any tests of procedures or methods to be undertaken.

The proposed questionnaires will be pretested on less than 10 respondents. Results of these pretests will be utilized to refine the information collection in order to reduce respondent burden and improve the usefulness of the information.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and /or analyze the information for the agency.

The statistical aspects of the design were coordinated by Mr. George Hill, Statistician, USDA: APHIS, Veterinary Services, CEAH, Fort Collins, CO (970) 494-7250. The actual data collection will be conducted by NASS. Contact persons for data collection are:

-Norm Bennett, Chief, Survey Administration Branch, Mail Stop 2024, 1400 Independence Ave., S.W., Washington, D.C. 20250 (202) 720-2248.

- Dr. John Clifford, Deputy Administrator, USDA: APHIS, Veterinary Services, Washington, DC (202) 447-6835.

Analysis of the data will be accomplished by NAHMS veterinarians, epidemiologists, and statisticians under the direction of:

- Dr. Bruce Wagner, Center Leader, National Animal Health Monitoring Systems, USDA: APHIS, VS, CEAH, 2150 Centre Avenue, Building B MS2E7, Fort Collins, CO 80526-8117 (970) 494-7256.

Appendix A: U.S. Small-Scale Livestock Operations

Table 1: Number of small-scale livestock operations

Number Farms*
Annual sales $10,000-99,999 and 1-3 commodities present	Annual sales $10,000-99,999 and 4 or more commodities present	Annual sales $100,000-249,999	Annual sales $250,000-499,999	Total Farms
351,116	70,280	92,221	50,145	563,762
*NASS list frame on August 17, 2010. Individual state data are withheld to protect confidentiality.

Table 2: Commodities on small-scale livestock operations

Commodity	Number Small-Scale Farms	Percent Small-Scale Farms
Grains and oilseeds	135,706	24.1%
Tobacco, cotton, vegetables, fruits, nuts, berries, nursery/greenhouse, cut trees	34,067	6.0%
Other crops	387,951	68.8%
Hogs and pigs	4,034	0.7%
Milk and dairy	44,451	7.9%
Cattle and calves	533,622	94.7%
Sheep and goats	65,463	11.6%
Horses, ponies	232,987	41.3%
Poultry and eggs	71,451	12.7%
Aquaculture	7,425	1.3%
Other animals and products	20,919	3.7%

Appendix B: Predicted Response Rate

A questionnaire requiring approximately 20 minutes to fill out, will be mailed to producers selected from the NASS list frame for this study. If a response is not received after two weeks, a second questionnaire will be mailed. If a response is still not received two weeks after the second mailing (one month after the initial mailing), enumerators from the NASS will call producers and administer the questionnaire via Computer Assisted Telephone Interview (CATI). Up to seven calls will be made in an effort to complete an interview before coding the respondent as unavailable. There will not be any attempt to convert refusals other than a clear explanation, during the phone call, of the importance of their voluntary participation. The response rate to the questionnaire using this combination of data collection techniques is predicted to be 70%.

1. Review of Previous NAHMS Response Rates for Studies Conducted Using Mailing and CATI

a. Questionnaire response rates:

Year	Questionnaire	Collection dates	Sample	Useable1	Useable %	Complete2	Complete %
2004	Small-Enterprise Swine (NASS CATI)	8/2/07-9/18/07	2,567	2,050	79.9	1,778	69.3
2007	Small Enterprise Chicken Report (NASS CATI)	8/1/07 – 9/30/07	2,511	1,789	71.2	1,191	47.4
2009	General Goat Management Report (NASS CATI)	7/1/09 - 8/17/09	2,000	1,429	71.5	649	32.5

¹Respondent was contacted and provided at least inventory information. Includes operations with zero inventory on hand.

²Respondent provided answers to all or nearly all questions.

Appendix C: Sample Calculation-Sample Size Unadjusted for Sampling Design

To estimate a sample size, we estimated the number of farms in each stratum based on the NASS list frame in August of 2010.

Table 1. Number of farms that will be sampled by stratum.

Stratum	Total Population (N)	Allocation (n)	n expected at 70% response rate	Sampling Interval
$10K-99,999 and 1-3 commodities	351,116	6,800	4,760	51.6
$10K-99,999 and 4+ commodities	70,280	2,340	1,638	30.0
$100K-249,999	92,221	4,860	3,402	19.0
$250K-499,999	50,145	2,000	1,400	25.1
Total	563,762	16,000	11,200

Table 2. Accuracy of estimates based on sample sizes outlined in Table 1.

Prevalence (%)	$10K-99,999	$100K-249,999	$250K-499,999	All farms
50	± 1.2%	± 1.7%	± 2.6%	± 0.9%
20	± 1.0%	± 1.3%	± 2.1%	± 0.7%
10	± 0.7%	± 1.0%	± 1.6%	± 0.6%

Note: These calculations were done in EpiInfo, Version 3.3.2 using finite population corrections.

1 Value of annual sales as calculated from control data on the NASS list frame at time of sample selection.

2 Farm type on the NASS list frame identifies the commodity with the highest percentage of total annual sales. Sixteen commodities are considered. Eight of these are animals or animal products: 1. hogs and pigs; 2. milk and dairy; 3. cattle and calves; 4. sheep and goats; 5. horses and other equine; 6. poultry and eggs; 7. aquaculture and 8. other animals. The other 8 commodities are crops: 1.grains, oilseeds, dry beans or dry peas, 2. tobacco, 3. cotton and cottonseed, 4. vegetables, melons, potatoes and sweet potatoes 5. fruit, tree nuts and berries, 6. nursery, greenhouse floriculture and sod, 7. cut Christmas tress and short rotation woody crops, 8. other crops and hay.

3 Eleven commodity categories are considered: 1. grains, oilseeds, dry beans, dry peas; 2. tobacco, cotton, vegetables, fruit, tree nuts, nursery; 3. other crops and hay; 4. hogs and pigs; 5. milk and dairy; 6. cattle and calves; 7. sheep and goats; 8. horses and other equine; 9. poultry and eggs; 10. aquaculture and 11. other animals.

8