0579-0079 2020 SS Part B (20200123)

Health Management on U.S. Feedlots, 2020 Study

B. Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, state and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

The Health Management on U.S. Feedlots study has two components, both part of the same study: a large capacity feedlot component and a small capacity feedlot component. The potential respondent universe for the large component is feedlots with 1,000 or more feedlot capacity in 17 states¹. For the small component, the potential respondent universe is feedlots with 50-999 head capacity in 18 states². Cattle on feed are defined as steers and heifers being fed a ration of grain, silage, hay, and/or protein supplement for slaughter market that are expected to produce a carcass that will grade select or better. It excludes cattle being “backgrounded only” for later sale as feeders or later placement in another feedlot (NASS Cattle on Feed, June 2019).

Feedlots with 1,000 or more head inventories account for approximately 78 percent of all cattle on feed in the U.S. The 17 States for the large component were chosen because they account for 95.4 percent of the cattle on feed on feedlots with 1,000 or more head inventories, and 94.2 percent of feedlots with 1,000 or more head inventory (NASS 2017 Census of Agriculture)³.

Although large feedlots account for 78 percent of U.S. cattle on feed, the Study includes a small component because small feedlots may have different management practices than large feedlots. Also, small feedlots account for 91.5 percent of all feedlots with 50 or more head inventory in the U.S. The 18 states chosen for the small component account for 95.5 percent of the inventory on feedlots with 50-999 head and 93.9 percent of feedlots with 50-999 head.

NASS will select all feedlots with 1,000 or more head capacity in the 17 States on their list frame to be in the survey. Thus, there is no sampling for the large component of the study. About 2,200 feedlots meet these criteria (NASS Cattle on Feed, Mar 2019). For the small component, a sample of about 3,193 feedlots will be selected from feedlots with 50-999 head capacity in the 18 states (NASS 2017 Census of Agriculture). Expected response rates within each of the components are presented in Table C.1 in Appendix C.

2. Describe the procedures for the collection of information including:

0. Statistical methodology for stratification and sample selection:

For the large component of the study, there is no sample selection because all of the approximately 2,200 feedlot feedlots with 1,000 or more head capacity in the 17 States will be included in the survey (NASS unpublished).

For the small component, eligible feedlots with 50-999 feedlot capacity in the 18 States from NASS’ List Frame will be stratified by State and by size (50-99, 100-199, 200-499, and 500-999). APHIS will allocate the total sample to strata based on a weighted average of inventory and number of feedlots in each stratum. Within each stratum, a simple random sample will be chosen.

0. Estimation procedure:

The study is two-phase for both large and small components. For the large component, probabilities of selection are equal to one while in the small component, probabilities of selection will be unequal across sampling strata. APHIS will construct sampling weights using this information combined with response information in order to adjust selection probabilities by nonresponse observed in the sample. We will perform the statistical estimation using either SAS survey procedures and/or SUDAAN. Both software packages use a Taylor series expansion to estimate variances appropriate to the survey design.

0. Degree of precision needed for the purpose described in the justification:

The overall NAHMS program goal is to develop descriptive statistics with a coefficient of variation (CV) less than 20 percent. If possible, given adequate response rates, APHIS will produce estimates by size category (small and large capacities).

In order to meet the precision criteria within each of the two reporting stratification categories, APHIS requires approximately 433 feedlots assuming that a simple random sample with a perfect response rate is taken. However, due to practical considerations, we must account for the expected response of approximately 30 percent at Phase I, an expected response of 53 percent at Phase II, and an expected design effect of approximately 2 to obtain estimates with appropriate standard errors. An overall sample size of approximately 5,450 feedlots is required after adjusting for these factors to produce estimates with CV not exceeding 20 percent within each of the reporting stratification cells. However, due to the limited number of large capacity feedlots in the population, the number of large feedlots in the sample is capped at the number of large feedlots in the population, reducing the required sample size to 5,393.

Table B.1 in Appendix B shows estimates of precision based on the total sample of 5,393, for size category cells, and nationally, for Phase I and Phase II. Response rates from previous studies (see Table C.1 in Appendix C for response rate information from previous NAHMS studies) and a design effect of 2 (calculated using information from previous studies) were assumed. All of the estimated CVs for Phase I and all but one of the CVs for Phase II are expected to be within the desired range with the given allocation.

APHIS will report estimates at the national level and by feedlot capacity category (Small and Large). Where possible, APHIS will report estimates by Size-Region category. Reporting strata for Phase II estimates may be adjusted depending on the number of respondents. In general, if sample sizes are too small or CVs too large for any estimates, those estimates are not published or are reported at a more aggregate level.

• Unusual problems requiring specialized sampling procedures and data collection cycles:

There are no unusual problems requiring specialized sampling procedures and data collection cycles.

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

The data collection described is not planned to be carried out on an annual or less than annual frequency basis for either component of this study.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

Questionnaire design and training:

0. Minimizing collection of data to that which is absolutely necessary. APHIS has solicited industry and producer input in the questionnaire to ensure that information collected is relevant and timely. Based on this feedback, we have reduced the number and complexity of questions compared to previous surveys of a similar scope.

0. We conducted meetings and discussions with industry representatives from the National Cattlemen’s Beef Association, (NCBA) get their support. The initial pre-survey mailing to selected participants will include a letter of support from NCBA and other feedlot industry representatives.

0. APHIS will develop specialized training for both the Phase I and Phase II data collectors to address any potential difficulties with procedures or items on the questionnaires.

0. When NASS enumerators visit the feedlot to solicit consent for study participation and administer the NASS questionnaire, the enumerators will provide information on Phase II of the study and its benefits and will provide a link to the Phase II questionnaire to ensure the participant is aware of what will be covered in that phase of the study.

1. APHIS will build on its efforts to collect data using an electronic data capture system. We plan to collect a portion of the data for this study using an electronic data capture system during Phase II in-person interviews. We will train APHIS data collectors who collect data in this way on collection of data and data security using this system.

Contacting respondents:

0. NASS will send a pre-survey letter to selected feedlots to inform them of the study objectives and benefits as well as timelines and what they can expect if they decide to participate.

1. We conducted meetings and discussions with industry representatives from the National Cattlemen’s Beef Association, (NCBA) get their support. The initial pre-survey mailing to selected participants will include a letter of support from NCBA and other feedlot industry representatives.

2. NASS enumerators will use an established contact protocol (including up to 7 phone calls and 3 visits in person) followed by an on farm visit at a scheduled time before they are listed as a refused or inaccessible feedlot. NASS enumerators have gone through specific training to help them answer questions of reluctant producers so as to maximize response rates.

3. The APHIS data collectors will contact farms that have consented to continue in Phase II of the study and set up a convenient time for the producer to complete the questionnaire. Training for the APHIS data collectors will include specific suggestions from the NASS trainers based upon their experience in working with specific producers to maximize response.

4. We will provide training and informational materials to APHIS data collectors in order to guide them through study specifics, benefits, and commitments by the producer in order to maximize participant conversion.

Non-response adjustment:

0. Baseline expected response rates are taken from the NAHMS Feedlot 2011 and NAHMS Antimicrobial Use and Stewardship on U.S. Feedlots, 2017 and are shown in Table C.1 in Appendix C.

1. APHIS will adjust selection weights adjusted for nonresponse using NASS-supplied stratification variables. We will transfer weights of eligible non-respondents to responding feedlots that are most similar based on available data, including the State and size category stratification variables. The nonresponse adjustment will use the method of propensity scores, in which a logistic regression model is constructed to predict the probability of responding. The inverse of this probability is the nonresponse adjustment.

2. If the respondents differ substantially from the non-respondents, then there is potential for bias. NASS’ List Frame data may be available for both respondents and non-respondents to allow for examination of potential differences in type of responding and non-responding feedlots. If needed, APHIS will perform a nonresponse bias analysis to investigate unexpected response patterns to guide future sampling efforts. If significant nonresponse bias is found, the factors contributing to the bias will be incorporated into the nonresponse weight adjustment using post-stratification raking procedures.

Sampling and design strategies:

0. Sampling from the NASS List Frame after the NASS 2017 Census of Agriculture will help to maintain adequate response rates by avoiding feedlots that are out of scope, out of business, or otherwise do not have animals at the time of contact.

1. Because 11 of 12 States included in NASS’ Monthly Cattle on Feed Report are included in the large component, they will have the greatest likelihood of having been updated in the List Frame during the last month compared to other potential States.

2. Face-to-face enumerated questionnaires are planned for both phases. Face-to-face enumeration has historically been shown to produce higher response rates in NAHMS studies than those in studies implementing mail-out or phone-based surveys.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

APHIS will pretest questionnaires prior to field enumeration, involving fewer than 10 respondents. For the APHIS pretests, one component of the pretesting will focus on the formatting of the antibiotic use section of the Phase II Questionnaire. With regards to this specialized focus, original and alternative formatting for this question have been included in the OMB package submission to make the options available during pretesting transparent. A NASS survey methodologist will have federal employees with feedlot expertise review the Phase I questionnaire and perform pretests on the questionnaire on fewer than 20 feedlots, under a NASS generic OMB clearance.

APHIS will use the results of these pretests to refine the questionnaires in order to reduce respondent burden and improve the usefulness of the information. We will also use the pretested and revised questions from previous iterations of NAHMS Feedlot studies, where possible. The final questionnaires utilized by the Phase I and Phase II data collectors will have been reviewed by a variety of experts, including academic researchers, industry representatives, extension agents, veterinarians, health specialists, and epidemiologists.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The statistical aspects of the design were coordinated by:

- Mr. Matthew Branan, Mathematical Statistician, USDA, APHIS, VS, NAHMS, Fort Collins, CO (970-494-7349).

For questionnaire design and methodology, NAHMS will coordinate with:

- Mr. Kenneth Pick, USDA, NASS, Methodology Division Standards and Survey Development Methodology Branch, Washington, DC (202-720-7490).

NASS review of the OMB package submission will be coordinated with:

- Mr. David Hancock, USDA, NASS, Methodology Division Standards and Survey Development Methodology Branch, Washington, DC (202-690-2388).

The actual data collection will be conducted by NASS enumerators (Phase I) and APHIS-designated data collectors (Phase II). Contact persons for data collection are:

- Mr. Gerald Tillman, Chief, Survey Administration Branch, USDA, NASS, Washington, D.C. 20250, (202-720-3895).

- Dr. Bruce Wagner, Director, Centers for Epidemiology and Animal Health, USDA, APHIS, VS, Fort Collins, CO (970-494-7256).

- Dr. Burke Healey, Deputy Administrator, Veterinary Services, USDA, APHIS, Washington, D.C. (202-799-7147).

Analysis of the data will be accomplished by APHIS veterinarians, epidemiologists, and statisticians under the direction of:

- Dr. Amy Delgado, Director, Monitoring and Modeling, USDA APHIS, VS, CEAH, Fort Collins, CO (970-494-7302).

Appendix A: State selection

State selection was performed based on the percentage of feedlots and of inventory by State, for geographic representativeness, and expected response burden.

Table A.1. Number and percentage of feedlots and number and percentage of feedlot inventory by size class (as measured by inventory of cattle on feed).

Size class (inventory)	Percent Feedlots	Percent Inventory
Small (50-999 head)	91.5	21.5
Large (1,000 or more head)	8.5	78.5

* Population-level information is taken from the NASS 2017 Census of Agriculture.

Table A.2. Number and percentage of feedlots and of feedlot inventory on feedlots with 1,000 or more head inventories, by State. Note, columns may not sum to totals due to rounding.

State	Percent Feedlots	Percent Inventory
Texas	7.1	22.7
Nebraska	30.3	22.4
Kansas	8.6	19.9
Colorado	5.4	8.4
Iowa	20.1	5.9
California	1.4	4.3
Oklahoma	1.0	2.7
Idaho	1.3	2.2
South Dakota	6.8	2.2
Washington	0.9	1.8
Minnesota	5.7	1.5
Oregon	0.5	0.7
Illinois	2.5	0.5
Wyoming	0.9	0.5
New Mexico	0.3	0.4
Michigan	0.8	0.3
Montana	0.4	0.2
Missouri	0.5	0.1
Indiana	0.5	0.1
Utah	0.3	0.1
North Dakota	1.0	0.0
Ohio	0.8	0.0
Wisconsin	0.7	0.0
Arizona	0.4	0.0
Pennsylvania	0.4	0.0
Maryland	0.2	0.0
Nevada	0.1	0.0
Tennessee	0.1	0.0
Virginia	0.1	0.0
Delaware	0.1	0.0
Georgia	0.1	0.0
Kentucky	0.1	0.0
Maine	0.1	0.0
Mississippi	0.1	0.0
New York	0.1	0.0
U.S. Total	100.0	100.0

* Population-level information is taken from the NASS 2017 Census of Agriculture.

Table A.3. Number and percentage of feedlots and of feedlot inventory on feedlots with 50-999 head inventories, by State. Note, columns may not sum to totals due to rounding.

State	Percent Feedlots	Percent Inventory
Iowa	23.0	29.2
Minnesota	12.4	12.3
Nebraska	7.4	9.7
South Dakota	6.7	8.9
Wisconsin	11.0	7.3
Illinois	6.0	5.0
Ohio	5.2	4.4
Kansas	3.5	4.3
Michigan	3.9	3.4
Pennsylvania	4.6	3.0
Indiana	2.8	1.9
North Dakota	1.9	1.8
Missouri	2.5	1.7
Colorado	0.9	0.9
Texas	1.2	0.7
Wyoming	0.5	0.6
Montana	0.7	0.5
Utah	0.6	0.5
New York	0.9	0.5
Kentucky	0.8	0.4
Virginia	0.8	0.4
Idaho	0.4	0.4
California	0.2	0.3
Oklahoma	0.4	0.2
Oregon	0.4	0.2
Maryland	0.3	0.1
Washington	0.2	0.1
New Mexico	0.1	0.1
North Carolina	0.1	0.1
Nevada	0.1	0.1
West Virginia	0.1	0.0
Arizona	0.0	0.0
Tennessee	0.0	0.0
Vermont	0.0	0.0
New Jersey	0.1	0.0
Massachusetts	0.0	0.0
Maine	0.0	0.0
Delaware	0.0	0.0
Alaska	0.0	0.0
Louisiana	0.0	0.0
New Hampshire	0.0	0.0
U.S. Total	100.0	100.0

* Population-level information is taken from the NASS 2017 Census of Agriculture.

Appendix B: Precision of Estimates

Estimates of percent of feedlots and percent of animals will be reported at the national level and by size category.

Estimates of precision for proportions of .5 and .1 are shown in Table 1. As an example, for the size category 1000-7999 and an expected proportion of 0.5, the coefficient of variation (CV) is 5%. Only one of the estimated CV’s in the examples presented in Table 1 exceeds 20%. We believe these CV’s err on the high side because the effective sample sizes will likely be higher than those shown here for a design effect of 2.

Table B.1. Precision of estimates, by reporting class and by expected proportion, at 95% confidence.

Size of feedlot (capacity)	Estimated overall sample size*	Proportion estimate	Phase I CV estimate (%)	Phase II CV estimate (%)
Small (50-999 head)	3,193	0.50	4.7	6.6
		0.25	8.1	11.5
		0.10	14.0	19.9
Large (1,000 or more head)	2,200	0.50	4.9	6.8
		0.25	8.4	11.7
		0.10	14.6	20.3
Total	5,393	0.50	3.6	4.9
		0.25	6.2	8.6
		0.10	10.7	14.8

^* Sample size approximations presented here.

Appendix C: Estimated Response Rates

Note: There are many different methodologies for calculating and reporting response rates. The rate we choose to report here are the “raw” response rates. These are ratios of total numbers of subjects completing the respective Phase questionnaire to the total numbers of subjects selected from the NASS list frame for the given study. These “raw” rates give the best estimate of the real proportion of sampled subjects giving complete data for the study as they take into account both response propensity of eligible subjects and list frame covereage quality since it does not filter out ineligible subjects.

The response rates from the Antimicrobial Use (AMU) and Stewardship on U.S. Feedlots, 2017, are presented below. The response rates to this study were lower than are typical for NAHMS studies in general, and for NAHMS studies of cattle on feed (CoF). For this reason, the response rates from the NAHMS Feedlot 2011 study are included in the table as well. We have reason to believe, since the study design has changed positively since the 2017 study, and because the list frame from which the final sample will be taken will have been updated using the Census of Agriculture 2017 data, that the expected response rates for the 2020 study will not be as they were in the 2017 study. For this reason, the expected response rates for small capacity feedlots and large capacity feedlots at both phases of the study will be closer to the average of the 2017 and 2011 response rates, if not greater.

Table C.1. Response rates from the Cattle on Feed 2017 study and Feedlot 2011 studies.

Study - Study Component	Phase I response rate (%)1		Phase II response rate (%)
Feedlot 20112 ( and , respectively)	40.5	65.1
AMU CoF 2017 – Large ( and , respectively)	23.2	47.1
AMU CoF 2017 – Small ( and , respectively)	16.5	35.0
Health Management on U.S. Feedlots, 2020 – Large (estimated, see calculations below)	31.8	56.1
Health Management on U.S. Feedlots, 2020 – Small (estimated, see calculations below)	28.5	50.1

¹ Phase I response rates for AMU 2017 study components is similar to the product of the Phase I complete rate and the rate of feedlots that consent to participate in Phase II of a typical NAHMS study, since there was no Phase I questionnaire in 2017. That means that, on average, we expect percent of large feedlots to complete Phase I and consent to Phase II. This means that approximately percent of large feedlots will complete Phase I and 54.9 percent of those will consent to participating in Phase II.

² In-scope feedlots for this study included feedlots with 1,000 or more cattle on feed capacity.

Health Management on U.S. Feedlots, 2020 – Small estimates response rates for Phase I ( ) and Phase II ( )

Health Management on U.S. Feedlots, 2020 – Large estimates response rates for Phase I ( ) and Phase II ( )

Appendix D: Estimated Time Burden

Table D.1. Response burden estimates from the NAHMS Antimicrobial Use and Stewardship on U.S. Feedlots, 2017 Study (in minutes).

Survey	Lower Quartile	Median	Upper Quartile
Completion time	30	40	60
Travel time (round trip)	60	120	180

1 California, Colorado, Idaho, Illinois, Iowa, Kansas, Minnesota, Missouri, Montana, North Dakota, Nebraska, Oklahoma, South Dakota, Texas, Utah, Washington, and Wyoming

2 California, Colorado, Idaho, Illinois, Indiana, Iowa, Kansas, Michigan, Minnesota, Missouri, North Dakota, Nebraska, Ohio, Pennsylvania, South Dakota, Texas, Wisconsin, and Wyoming

3 The study was designed using population counts from NASS detailing feedlots and feedlot inventory by feedlot capacity. These estimates are suppressed in this justification and replaced with design statistics based on NASS data on feedlot inventory from the 2017 Census of Agriculture, where appropriate because population counts by feedlot capacity are not published NASS statistics. Similar results hold for both methods. We make this clear in the justification by using the terms “capacity” and “inventory” to denote which classification variable of feedlots is being used.

24