B. Collections of Information Employing Statistical Methods
Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, state and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.
The potential respondent universe for the National Animal Health Monitoring System (NAHMS) Bison 2022 Study is all U.S. farms with 1 or more bison at the time of the Study in all 50 states. The National Agricultural Statistics Service (NASS) will select all U.S. farms with 1 or more bison in all 50 States on their list frame to be in the Study. Thus, this Study will be a census of the NASS list frame. The number of operations NASS will select is expected to be approximately 2,000 operations (NASS 2017 Census of Agriculture and unpublished NASS data procured to produce this justification). This number may be revised when final population counts are requested closer to the enumeration timeline.
This will be a cross-sectional Study with two phases. The primary data collection instrument of the “NASS Phase” is the NAHMS Bison 2022 Study Survey (VS Form 21-320) that will be implemented by NASS via paper-assisted self-interviews (PASI), computer-assisted self-interviews (CASI), and computer-assisted telephone interviews (CATI). For producers who indicate their willingness to participate in the “Biologics Phase” on the NAHMS Bison 2022 Producer Informed Consent Form (VS Form 21-321), their contact information will be turned over to United States Department of Agriculture (USDA)-Animal and Plant Health Inspection Service (APHIS)-NAHMS in order to allow the producer to participate in a biological sample collection opportunity (NAHMS Bison 2022 Enteric Microbe Collection Record, NAHMS Bison 2022 Pre-Deworming Fecal Parasite Kit A Collection Record, NAHMS Bison 2022 Post-Deworming Fecal Parasite Kit B Collection Record, and NAHMS Bison 2022 Forage Collection Record; VS Form 21-322, VS Form 21-323, VS Form 21-324, and VS Form 21-325, respectively) where producers will self-collect biologics on their farms and receive biological testing results for bison and forage on their farm.
There will be two reporting strata for the Study: size (based on bison inventory) and region of the country. Due to the Study being a census rather than a sample, sample sizes are equal to the population within each individual stratum level and thus have been suppressed from this form since this is unpublished data provided to APHIS from NASS for the purposes of designing the study.
The expected response rates over the course of the Study and response rates from previous related studies are presented in Appendix B.
2. Describe the procedures for the collection of information including:
There is no sample selection because all of the approximately 2,000 U.S. farms with 1 or more bison in all 50 states at the time of the Study will be selected from the NASS list frame during the NASS Phase.
Operations will be post-stratified in order to properly account for non-response among like operations and will take place along regional and size of operation (inventory of bison) boundaries, of which there will be four levels of each. Regions will be defined as:
Northeast (CT, DE, ME, MD, MA, NH, NJ, NY, OH, PA, RI, VT, WV)
Southeast (AL, AR, FL, GA, KY, LA, MS, NC, SC, TN, VA)
North Central (IL, IN, IA, MI, MN, MO, WI)
West (AK, AZ, CA, CO, HI, ID, KS, MT, NE, NV, NM, ND, OK, OR, SD,
TX, UT, WA, WY)
Size categories will be defined as:
Very Small (1-9 head)
Small (10-24 head)
Medium (25-99 head)
Large (100 or more head)
If the data supports it, a Very Large size category will be added.
The Study is two-phase with probability of selection equal to 1 in the NASS Phase. All producers who complete the survey in the NASS Phase will be invited to participate in the Biologics Phase, if willing. APHIS will construct weights using this information combined with response information in order to adjust selection probabilities by non-response observed in the survey phases. We will perform the statistical estimation using SAS survey procedures, SUDAAN, and/or R. All software packages use a Taylor series expansion to estimate variances appropriate to the survey design, including the non-response adjusted sampling weights.
The APHIS goal is to develop descriptive statistics (proportions or means) with a coefficient of variation (CV) of 20 percent or less. If possible given adequate response rates, APHIS will produce estimates by region (Northeast, Southeast, North Central, and West) and by size categories (Very Small, Small, Medium, and Large).
In order to meet the precision criteria within each of the given reporting stratification categories, APHIS requires approximately 589 bison operations assuming that a sample with a perfect response rate is taken. However, due to practical considerations, we must account for the expected response of approximately 40 percent at the NASS Phase, an expected turnover response of approximately 55 percent, and approximately 50 percent response at the Biologics Phase to obtain estimates with appropriate standard errors (the design effect is assumed to be 1). An overall sample size exceeding the approximately 2,000 bison operations in the population would be required after adjusting for these factors to produce estimates with CV not exceeding 20 percent within each of the reporting stratification cells. Therefore, a census of bison operations is required to meet the statistical needs of the Study.
Tables A.1 and A.2 in Appendix A show estimates of precision in each size and region category levels, as well as nationally, assuming a census of the 2,000 bison operations on the NASS list frame. Coefficients of variation generally meet the design CV of 20 percent in the NASS Phase. Many of the coefficients of variation exceed the 20 percent designed limit and so estimates created at the Biologics Phase will need to be reported at the National level (for which coefficients of variation are expected to be below 20 percent) or, even if response rates are greater than expected, reporting strata may need to be collapsed.
APHIS will report estimates at the national level, by region strata, and by size strata where possible and appropriate. APHIS will not report any cross strata (Region-Size) categories. Reporting strata may be adjusted depending on the number of respondents. In general, if sample sizes are too small or CVs too large for any estimates, those estimates are not published or are reported at a more aggregate level.
There are no unusual problems requiring specialized sampling procedures and data collection cycles.
Any use of periodic (less frequent than annual) data collection cycles to reduce burden.
The data collection described is not planned to be carried out on an annual or less than annual frequency basis.
Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.
The Study minimizes collection of data to that which is absolutely necessary to meet the stated objectives. Surveys are extensively reviewed by APHIS staff, NASS staff, and experts both in industry and in academia.
For the NASS Phase data collection, APHIS staff will develop training materials (including a NASS training manual) for NASS enumerators and customer service that explain the purpose of the Study and address anticipated difficulties with questions. APHIS representatives will participate in the NASS trainings.
For the biologics collection, the participating bison producers will be sent detailed instructions on how to collect samples on their farms using the provided test kits, and shipping instructions will be provided for routing their samples to the appropriate testing laboratory. These instructions, as well as links to other biological sampling aids, will be available on the NAHMS Bison Studies web page.
The survey mailer sent to bison producers during the NASS Phase will include information on the Biologics Phase and its benefits to ensure the participant is aware of what will be covered in the Biologics Phase. The same information will be covered by NASS enumerators administering the CATI to bison producers who didn’t answer the PASI or CASI. Producers will be informed as to how the data they supply will be protected and used.
Data collectors and data handlers will have been trained on data and information security guidelines.
APHIS staff have made numerous contacts and has been involved in collaborative efforts to identify the information needs of the industry and the best way to ask for and incentivize the information collection via survey.
NASS will first send a producer selection letter to bison producers selected to participate in the Study. After notifying producers that they have been selected to participate in the study, NASS will send a survey packet to bison producers that includes a survey launch sheet with study background and benefits information, a description of the study timeline and biologics benefits, a paper survey with a link to the CASI survey on the web, forms describing participation in the Biologics Phase of the Study, educational materials (an informational incentive), a field flyer, VS Form 21-326 - NAHMS Bison 2022 Producer Evaluation, and a guide to assist producers in answering the CASI survey.
A reminder card will be mailed out in the middle of the self-enumeration period to remind selected producers of their opportunities for responding.
A thank you card (considered an incentive) will be sent to all producers who complete the NASS Phase.
APHIS conducted meetings and discussions with industry representatives from the summer of 2020 through present to get their support. The survey mail-out to selected participants will include a launch sheet expressing support from the National Bison Association and associated industry associations, and other bison industry representatives.
During the CATI followup to non-respondents of the PASI and CASI, NASS enumerators will use an established contact protocol before an operation is listed as a refusal or inaccessible bison operation. NASS enumerators have gone through specific training to help them answer questions of reluctant producers to maximize response rates.
Baseline expected response rates are taken from the NAHMS Bison 2014, Cervid 2014, Beef 2017, and Goat 2019 studies and are shown in Table B.1 in Appendix B.
APHIS will adjust selection weights adjusted for non-response using NASS-supplied stratification variables. Weights of eligible non-respondents will be transferred to responding bison operations that are most similar based on available data, including the State and size category stratification variables. The non-response adjustment will use the method of propensity scores, in which a logistic regression model is constructed to predict the probability of responding. The inverse of this probability is the nonresponse adjustment.
If the respondents differ substantially from the non-respondents, then there is potential for bias. NASS’ List Frame data may be available for both respondents and non-respondents to allow for examination of potential differences in type of responding and non-responding bison operations. If needed, APHIS will perform a non-response bias analysis to investigate unexpected response patterns to guide future sampling efforts. If significant nonresponse bias is found, the factors contributing to the bias will be incorporated into the nonresponse weight adjustment using post-stratification raking procedures.
Sampling from the NASS List Frame after the NASS 2017 Census of Agriculture will help to maintain adequate response rates by avoiding bison operations that are out of scope, out of business, or otherwise do not have animals at the time of contact. Bison operations are not surveyed often, and this was the last time NASS conducted a survey specifically for bison operations.
Due to the statistical design requiring a census of the entire U.S., the modes of administration through PASI, CATI, and CASI were chosen to meet the varied response mode preferences of producers. Preferred response modes vary across the population, and so making available a selection of response opportunities to fit producer’s schedules and preferences has been important to maximizing response in a COVID-19 environment and will continue to be important going forward.
APHIS and NASS will pretest the survey prior to field enumeration, involving fewer than 10 respondents. APHIS will use the results of these pretests to refine the surveys in order to reduce respondent burden and improve the accuracy and usefulness of the information. The pretested and revised questions from the NAHMS Bison 2014 Study will be used as a baseline, where possible, in order to borrow from the work performed during that study and to ensure that trends on particular topics can be drawn between the two studies. The final survey utilized in the NASS Phase will have been reviewed by a variety of experts, including academic researchers, industry representatives, extension agents, veterinarians, health specialists, and epidemiologists.
5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.
The statistical aspects of the design were coordinated by:
Mr. Matthew Branan, Mathematical Statistician, USDA, APHIS, VS, NAHMS, Fort Collins, CO (970-494-7349).
For survey design and methodology and for a NASS review of the OMB package submission, NAHMS will coordinate with survey methodologists reporting to:
Mr. Daniel Beckler, Chief, Standards and Survey Development Methodology Branch, USDA, NASS, Washington, DC (202-720-8858).
The actual enumerated data collection will be conducted by NASS enumerators (the NASS Phase) and the Biologics Phase biological samples will be producer-collected. Contact person for the NASS Phase data collection is:
Mr. Gerald Tillman, Chief, Survey Administration Branch, USDA, NASS, Washington, D.C. 20250, (202-720-3895).
Analysis of the data will be accomplished by APHIS veterinarians, epidemiologists, and statisticians under the direction of:
Dr. Amy Delgado, Associate Director, Monitoring and Surveillance, USDA APHIS, VS, CEAH, Fort Collins, CO (970-494-7302).
Appendix A: Precision of Estimates
Estimates of percent of operations and percent of animals will be reported at the national level and for region and size categories.
Estimates of precision for proportions of 0.5, 0.25, and 0.1 are shown in Table A.1 for size and Table A.2 for region. As an example, for the small (10-24 head) size category and an expected proportion of 0.5, the coefficient of variation (CV) is 5.3% in the NASS Phase. All but three estimates meet the precision criteria for the NASS Phase. Many estimates for the Biologics Phase components exceed the precision criteria and so collapsing of reporting strata may need to be done or reporting may need to be restricted to the National level in order to maintain average CV estimates.
Table A.1. Precision estimates at the NASS and Biologics Phases for the Bison 2022 Study, by size of operation.
Size of operation (number of bison) |
Proportion estimate |
NASS Phase CV estimate (%) |
Biologics Phase - Fecal CV estimate (%) |
Biologics Phase – Parasites A CV estimate (%) |
Biologics Phase - Forage CV estimate (%) |
Very Small (1-9 head) |
0.50 |
4.1 |
9.9 |
9.0 |
10.3 |
0.25 |
7.1 |
17.2 |
15.7 |
17.8 |
|
0.10 |
12.3 |
29.8 |
27.1 |
30.8 |
|
Small (10-24 head) |
0.50 |
5.3 |
12.8 |
11.6 |
13.2 |
0.25 |
9.1 |
22.2 |
20.2 |
22.9 |
|
0.10 |
15.8 |
38.4 |
34.9 |
39.6 |
|
Medium (25-99 head) |
0.50 |
5.6 |
13.7 |
12.5 |
14.1 |
0.25 |
9.7 |
23.8 |
21.6 |
24.5 |
|
0.10 |
16.9 |
41.2 |
37.4 |
42.4 |
|
Large (100 or more head) |
0.50 |
8.1 |
19.8 |
18.0 |
20.4 |
0.25 |
14.1 |
34.2 |
31.1 |
35.3 |
|
0.10 |
24.3 |
59.3 |
53.9 |
61.1 |
|
Total |
0.50 |
2.6 |
6.4 |
5.9 |
6.6 |
0.25 |
4.6 |
11.2 |
10.1 |
11.5 |
|
0.10 |
7.9 |
19.3 |
17.6 |
19.9 |
Table A.2. Precision estimates at the NASS and Biologics Phases for the Bison 2022 Study, by region of operation.
Region |
Proportion estimate |
NASS Phase CV estimate (%) |
Biologics Phase - Fecal CV estimate (%) |
Biologics Phase – Parasites A CV estimate (%) |
Biologics Phase - Forage CV estimate (%) |
0.50 |
5.5 |
13.4 |
12.2 |
13.8 |
|
0.25 |
9.5 |
23.2 |
21.1 |
23.9 |
|
0.10 |
16.5 |
40.2 |
36.5 |
41.4 |
|
Southeast (AL, AR, FL, GA, KY, LA, MS, NC, SC, TN, VA, WV) |
0.50 |
8.8 |
21.4 |
19.4 |
22.0 |
0.25 |
15.2 |
37.0 |
33.7 |
38.2 |
|
0.10 |
26.4 |
64.2 |
58.3 |
66.1 |
|
North Central (IL, IN, IA, MI, MN, MO, WI) |
0.50 |
8.3 |
20.3 |
18.4 |
20.9 |
0.25 |
14.4 |
35.1 |
31.9 |
36.2 |
|
0.10 |
25.0 |
60.8 |
55.2 |
62.7 |
|
West (AK, AZ, CA, CO, HI, ID, KS, MT, NE, NV, NM, ND, OK, OR, SD, TX, UT, WA, WY) |
0.50 |
3.5 |
8.5 |
7.7 |
8.8 |
0.25 |
6.0 |
14.7 |
13.4 |
15.2 |
|
0.10 |
10.5 |
25.5 |
23.1 |
26.3 |
|
Total |
0.50 |
2.6 |
6.4 |
5.9 |
6.6 |
0.25 |
4.6 |
11.2 |
10.1 |
11.5 |
|
0.10 |
7.9 |
19.3 |
17.6 |
19.9 |
Note that the term “response ratio” is, here, used as a description of the raw ratio of the count of producers who submitted complete (non-zero, in-scope) surveys to the count of producers selected from the NASS list frame for the given Study. This provides the best estimates of the number of producers who will supply complete data to be included in Study report tables compared to alternate definitions of “response ratio,” which may have alternate goals such as serving as a coverage metric or demonstrating willingness of the Study population to participate in the survey.
To estimate the response ratios expected for the Bison 2022 Study, response ratios observed from previous, similar NAHMS studies are presented below.
Table B.1. Response ratio estimates from (Phase I or the NASS Phase of) the Bison 2014 (OMB number 0579-0420) and Cervid 2014 studies (OMB number 0579-0417).
Study |
Study mode(s) |
Response ratio |
Bison 2014 |
PASI |
21.9 |
Cervid 2014 |
PASI/CATI |
42.5 |
Even though the population of interest in the Bison 2022 Study is more similar to that from the Bison 2014 Study, the modes included in the Study are more similar to those of the Cervid 2014 Study, which consisted of a mail-out survey that was completed by PASI, for which non-respondents were followed up with CATI.
Upon further inspection, the PASI-only responses were similar in both studies, being approximately 21.9 percent in Bison 2014 (which had two mailing events) and approximately 17.3 percent in Cervid 2014, indicating that the populations of interest appeared to be similar with regard to response via PASI.
Therefore, the estimated response ratio for the Bison 2022 Study will be assumed to be close to the Cervid 2014 response ratio, or approximately 42.5 percent. This estimate is similar to the response ratio expected for the NAHMS Swine 2021 Small Enterprise Study (OMB number 0579-0315), which consists of the same implementation of survey modes (PASI, CASI, with CATI follow-up). In the response ratio estimation for that Study, it was estimated that approximately 57.4 percent of sampled producers would be eligible and that 71.8 percent of those producers would submit complete (non-zero, in-scope) surveys, resulting in an estimated 41.2 percent of producers sampled expected to submit complete surveys.
Estimates for the Biologics Phase for the Bison 2022 Study have more uncertainty than those for the NASS Phase due to the fact that this population hasn’t participated in a biological sampling component during a NAHMS Study. Therefore, the estimates for turnover and participation in the Biologics Phase of the Bison 2022 Study, which will be a biological sampling component, will be based on results from the Goat 2019 Study (OMB number 0579-0354) and the Beef 2017 Study (OMB number 0579-0326), which are the two most recently completed NAHMS studies with similar biologics testing to the Bison 2022 Study. Again, any use of the term “response ratio” will be the raw ratio of the count of producers who submitted complete (non-zero) surveys to the count of producers eligible for that phase of the study.
Table B.2. Response ratio estimates from turnover and the biologics phases of the Beef 2017 and Goat 2019 studies.
Study |
Phase |
Response ratio |
Goat 2019 |
Turnover to be contacted for Phase II |
72 |
Biologics Phase – Fecal |
45 |
|
Biologics Phase – Parasites A |
85 |
|
Biologics Phase – Parasites B (as a ratio to those who completed Parasites A) |
50 |
|
Beef 2017 |
Turnover to be contacted for Phase II |
38 |
Biologics Phase – Fecal |
50 |
|
Biologics Phase – Forage |
45 |
Using averages between the response ratios to Beef 2017 and Goat 2019, the expected response ratios for each of the Biologics phases of Bison 2022 are as follows.
Table B.3. Response ratio estimates for turnover and the biologics phases of the Bison 2022 study.
Phase |
Computation |
Response ratio |
Turnover to be contacted for Biologics Phase |
Average response ratios from Beef 2017 and Goat 2017 turnover |
55.00 |
Biologics Phase – Fecal |
Average response ratios from Beef 2017 and Goat 2017 Biologics Phase - Fecal |
47.50 |
Biologics Phase – Parasites A |
Average response ratios from Beef 2017 Biologics Phase – Fecal and Forage and Goat 2017 Biologics Phase – Fecal and Parasites A |
56.25 |
Biologics Phase – Parasites B (as a ratio to those who completed Parasites A) |
Goat 2019 Biologics Phase – Parasites B |
50.00 |
Biologics Phase – Forage |
Beef 2017 Biologics Phase - Forage |
45.00 |
Appendix C: Estimated Time Burden
Table C.1. Response burden estimates from the NAHMS Bison 2014 Study and the NAHMS Cervid 2014 Study (in minutes).
Survey |
Average burden for respondents (min.) |
Average burden for nonrespondents (min.) |
NAHMS Bison 2014 Questionnaire |
31.6 |
2.4 |
NAHMS Cervid 2014 Industry Study Questionnaire |
32.3 |
9.6 |
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Fields, Victoria - MRP-APHIS-VS, Fort Collins, CO |
| File Modified | 0000-00-00 |
| File Created | 2022-02-09 |