Supporting Statement Part B Antiparasitic Survey

Supporting Statement

Antiparasitic Drug Use and Antiparasitic Resistance Survey

0910-NEW

Part B. Statistical Methods (used for collection of information employing statistical methods)

1. Respondent universe and sampling

The target population for this survey is the subset of veterinarians and parasitologists who have a direct opportunity to observe and assess antiparasitic drug resistance issues in the field. The sampling frame to reach the respondent universe will therefore be composed of veterinarians and experts who are involved directly with veterinary parasitology, who work in private practice, academia, research, and industry, who are members of a select group of professional organizations, and who have included their email addresses in the organizational database. The professional organizations targeted are the: American Veterinary Medical Association (AVMA), American Association of Bovine Practitioners (AABP), American Association of Small Ruminant Practitioners (AASRP), American Association of Equine Practitioners (AAEP), and American Association of Veterinary Parasitologists (AAVP). CVM will contact these professional organizations to inform them about the survey and to request their members’ participation in the survey. These organizations were selected because they are the major professional organizations associated with the target population of interest. Over 88% percent of all US veterinarians born after 1942 and who received their veterinary degree after 1967 are active AVMA members. The addition of species specific practice organizations such as AASRP or AABP will allow access to members who may not be AVMA members or who may have more identity with a specific practice type than with the general veterinary organization. These individuals may be more likely to follow emailed postings from specific practice group organizations such as AABP. All members with emails in the database will be contacted and invited to participate in the survey. Online panels have been utilized in other internet surveys (Maurer et al 2009, Bell et al 2011). While these panels have the benefit of allowing for selected sampling using study criteria, to our knowledge, an online panel of veterinarians or veterinary parasitologists does not exist. The use of a professional organization’s membership directory as a sampling frame is a common practice for surveys of health professionals in human and veterinary medicine (Parsons et al 2006; Peacock et al 2008; Cattaneo et al 2009; Coen et al 2011; Fajt, Wagner and Norby 2011; Hauck et al 2011).

Organization	Number of members
AVMA	82,686 (13,079 identify as food animal, equine, or mixed animal practitioners)
AABP	Approx. 6000
AAEP	Approx. 10,000
American Association of Small Ruminants	Approx. 1000
American Association of Veterinary Parasitologists	Approx. 450

The sampling unit will be the individual e-mail addresses available in the database. Members who do not have email addresses in the professional organization databases will not receive an invitation to participate in the survey directly but may receive links in the form of forwarded emails from other practitioners that received the invitation. To our knowledge, there is no evidence that an individual’s email directory status is related to his or her experience or opinion regarding antiparasitic drug use or antiparasitic drug resistance. Individuals who do not have experience with equine, bovine, or small ruminant medicine or veterinary parasitology in the United States will be excluded during the survey completion process through the use of targeted exclusion questions.

Survey responses will be collected by a URL link specific to each organization which will allow for determination of response rate per organization. Each organization will be contacted to provide information on the number of individuals, the number of individual email addresses, and the number of individuals per practice type, if applicable, and state in the organization’s database. This information will allow for comparisons of respondent characteristics (practice type, region) with organizational information. Selection bias may occur if individuals who are more interested in the topic are more likely to complete the survey. The reported level of client inclusion of respondents in routine antiparasitic drug use decision can be compared with results from prior studies (e.g., Stowe 2010; USDA 2007) to determine if the population of respondents is more actively approached by clients for routine antiparasitic drug use decisions. Again, involvement is not expected to vary by email status, but may inflate the proportion of individuals who report awareness of antiparasitic drug resistance. This study is not designed to estimate prevalence of antiparasitic drug resistance or frequency of antiparasitic drug use, but to elicit the opinions and experiences of practicing veterinarians and parasitologists. A benefit of potentially targeting individuals who are more actively involved in antiparasitic drug decision making is that CVM can consider the opinions of those who are most likely to take advantage of policies promoting safe and effective use of antiparasitic drugs and who could disseminate information to other treatment decision makers.

The survey instrument is designed to elicit a high response rate. The questions have been written using current recommendations for wording, have been screened for clarity, and allow for respondents to choose “non-applicable” or to enter an individual response. The survey format is uniformly designed to allow for easy viewing. Previously published studies in similar veterinary populations including web-based surveys of bovine practitioners identified through association e-mail directories have reported response rates that range from 8% to 26% (Cattaneo et al. 2009; Fajt, Wagner and Norby 2011 ; Coetzee et al. 2010) which are similar to response rates found for surveys of physicians (Swaminath et al. 2011). Using a conservative mid-range response rate based on the above studies of 11% from food animal, equine, and small ruminant practitioners and assuming 50% of individuals had emails in the professional organization database would result in 650 collected surveys. On average, response rates are about 10% lower in internet surveys compared with traditional postal or random digit dial (RDD) telephone surveys (Manfreda et al 2008, Shih and Fan 2008). Response rates for multiple modes and types of surveys, including paper and random-digit-dial telephone surveys have continually declined over the past 20 to 30 years (Curtin, Presser and Singer 2005; Newell et al 2004; Tuckel and O’Neill 2001). The 2010 Census received a 74% national mailed response rate despite a large allocation of resources to promote awareness and support of the survey (2010census.gov). Other established longitudinal surveys have also suffered large declines. The 2007 Health Information National Trends Survey (HINTS), a biennial cross sectional RDD survey supported by the National Cancer Institute, had an overall RDD telephone survey response rate of 24.23% and a 30.99% mailed survey response rate (Cantor et al 2009). Curtin, Presser, and Singer (2005) reported that response rates for the University of Michigan’s Survey of Consumer Attitudes (SCA) declined from 72% in 1979 to 48% in 2003 driven by the increase in noncontact rates and a recent increase in refusal rate. Monthly SCA response rates are now reported to vary from 42 to 45% (Wagner 2010). Lower response rates have been reported in other similar polls and surveys (Newell et al 2004; Tuckel and O’Neill 2002).

This survey is an exploratory study designed to elicit mainly descriptive responses. The a priori hypotheses for the true proportions of the individuals who will hold certain opinions or perform certain diagnostics are unknown and unavailable in the literature. While a low response rate may suggest bias and increase the possibility that respondents differ from non-respondents, this potential bias can be addressed in the discussion of the results. The value of examining multivariate relationships and comparisons between different subsets of our respondents, including those who are highly involved or interested in antiparasitic drug use and antiparasitic drug resistance, in order to acquire information that is presently unavailable, is significant to CVM.

2. Procedures for the Collection of Information

A probability sampling method was considered. However, this would require identification and contact of association members to obtain e-mail addresses for an internet survey or would incur printing costs associated with a mailed survey to listed practice addresses. Either method would require extensive financial and human resources.

Due to these considerations, a survey of a complete sampling frame was determined to be the best feasible option. The AVMA membership email database, the most complete sampling frame of veterinarians available, supplemented by other associations provides excellent coverage. A limitation of this method is that inferences for a generalized population cannot be given. The use of the entire sampling frame without knowledge of non-respondent characteristics hinders our ability to judge the accuracy of our predictions of univariate estimates for the entire population of US veterinarians and veterinary parasitologists. This data collection is more interested in examining multivariate relationships and comparisons between different subsets of the veterinary population, including credential type (respondents with advanced parasitology degrees or respondents with DVM degrees only), previous experience of antiparasitic drug resistance, or types of diagnostic testing performed. We have no evidence that non-coverage bias resulting from not including members who do not have email addresses is in any way associated with any of our respondent characteristics, opinions, experiences, or practices. Therefore, our comparisons among the groups will be valid and remains our main research interest.

The agency has developed a detailed survey questionnaire which will use a third-party internet survey hosting service. The survey was provided an exemption approval by the Research in Human Subject Committee (Exemption approval #11-020V, dated April 11, 2011). The statistical classification for the survey is the “Fields of Science and Engineering.”

The detailed survey instrument, executive summary, and survey question mapping and analysis plan are attached as separate documents. The data will be collected one time using the third-party internet survey hosting service SurveyMonkey, LLC (“SurveyMonkey”). SurveyMonkey is an internet survey hosting platform used in many peer-reviewed research studies, including studies that elicit expert specialist and physician opinions and practice methods (Coen et al 2011; Freedman et al 2011; Hauk and Nogan 2011; Parsons et al 2006; Stefanidis, Richardson and Fanelli 2010; Swaminath et al 2011). SurveyMonkey was also used by the National Animal Health Monitoring System (NAHMS) for needs assessments to help determine scope and focus of future animal health surveys (USDA 2007, 2011). The use of a third-party platform allows for the use of a hosting platform that is recognized by many internet users, is functional on many different computer hardware configurations, including tablet and netbook (small laptop) computers, has validated cross-browser functionality, and is compliant with section 508 of the Rehabilitation Act.

The survey instrument was designed using the Dillman Tailored Design Method customized for internet surveys (Dillman, Smyth and Christian 2009). The multi-page survey instrument has 37 unique questions. The survey is formatted as 1 or 2 questions per sequential screen to allow for a common visual stimulus, ease of visual access and response, and to allow for immediate recording of data after each question submission. The restriction on page size limits potential differences in on-screen display that may occur during use of currently popular tablet or netbook devices. Skip-logic for question advancement is utilized for some questions; individuals will not be required to answer questions that are not relevant to them. The maximum number of questions that an individual will be required to answer is 37, though most respondents will answer 19 to 33 questions depending on answers to prior questions. Individuals will initially be directed to answer questions corresponding to the target animal class represented by their stated practice or focus area. Respondents will be able to repeat two of the sections for another target animal class if desired, resulting in an additional 5 to 18 questions for each additional target animal. Completing the survey for additional animals is entirely voluntary.

The questions are predominately closed-ended, many with dichotomous yes/no responses or simple multiple choice answers. All responses are either tailored to the response to a previous question through skip-logic or allow an “other” or “not applicable” response to limit the frequency of item non-responses. The opportunity for respondents to include an open-ended answer choice is offered for some of the questions to allow inclusion of different responses not otherwise expressed in the answer choices. The collection period will occur over 5 weeks, allowing multiple points of contact and time for individuals to complete the survey at their convenience. Respondents will be given clear instructions for the purpose of the survey, how they were chosen for the survey by nature of inclusion in the professional organization, how long we expect the survey to take to complete, and contact information for the agency on the first page of the survey.

The survey is conceptually divided into 5 sections:

1. Part I contains 9 questions and will provide background information on the respondents and are required to determine eligibility to complete the survey and to determine categorical placement of observations during data analysis.

2. Part II contains 5 questions related to awareness of, concern about, and experience with antiparasitic drug resistance in cattle, horses, and/or small ruminants.

3. Part III contains 8 questions related to respondents’ decision criteria for use of antiparasitic drugs in cattle, horses, and/or small ruminants.

4. Part IV contains 10 questions for cattle and horses and 11 questions for small ruminants related to methods that respondents use to detect and manage antiparasitic drug resistance.

5. Part V contains 4 questions to elicit the types of information that would best assist end users to use antiparasitic drugs safely and effectively.

Responses from individuals will be automatically coded by response choice and entered into a downloadable spreadsheet by SurveyMonkey. Data will be downloaded from SurveyMonkey along with information about which professional organization supplied the link and saved as a Microsoft Excel file and/or other statistical software data files (e.g., SAS, Cary, NC) on FDA’s secure server. The data will be sorted and appropriately examined for possible duplicates and values that are outside expected responses.

Imputation will not be utilized to assign values for missing data; missing data points will be included as item non-responses. Data will be kept on the FDA’s secure environment with access limited to approved individuals. No individually identifiable data will be included. Coding will be objective and limited to the question responses. Additional responses not listed in the question elicited through comment boxes will be examined and reported, but will be included as an “other” category and not included in other listed categories.

3. Methods to Maximize Response Rates and Deal with Non-response

Response rates will be calculated using the number of email invitations sent per association to determine the sample size. The number of non-delivered email invitations will be reported and included in the response rate calculation. The entire sampling frame will be used and therefore, sample weights will not be used. Item non-response will be calculated using established methods and applicable analysis required as determined by unit and item response rate. CVM will use existing data on percentages of types of veterinarians and animal scientists who are members of the practice organizations to compare our response percentages. Additionally, organizations will be asked to provide information regarding the percentage of their members who reside in each state. This information can be aggregated by region for each organization and compared to the distribution of our respondents per region by organization. AVMA has also provided publically available information on the number of food animal veterinarians per state (defined as: food animal predominant or mixed animal, which consisted of all those who had 30% or more involvement with one or more of the following species: Bovine, Porcine, Ovine/Camelid, Cervid, and Poultry¹). This information will be aggregated by region as defined in our survey and compared to the regional distribution of the total number of practitioner respondents with greater than or equal to 30% of their practice area in one of the appropriate practice areas. This distributional comparison will be used to discuss the validity and ability to generalize the data. For respondents who do not choose a specific practice group directed at our targeted animals of interest, further additional questions are asked to determine eligibility and direction of the remainder of the survey.

Non-response error is determined by interactions of the non-response rate and differential responses of respondents and non-respondents for a particular variable of interest (Hox 2012). It follows that by decreasing the number on non-responses, one would decrease non-response error. In contrast, Groves and Peytcheva (2008) reported only a weak relationship between response rate achieved and the non-response bias in their meta-analysis of 59 methodological studies. Additionally, increasing non-responses from individuals who were at first unwilling to participate may reduce the non-response error but have little effect on the total survey error (Hox, de Leeuw and Chang 2012) due to the potential increase in measurement error (Groves and Couper, 1998). While lower response rates increase the probability of nonresponse bias, the actual level of bias depends on the relationship between factors that may promote nonresponse and the variables of interest and higher response rates may not necessarily result in a lower nonresponse bias (Massey and Tourangeau 2013; Fuchs, Bossert and Stukowski 2013).

Methods to increase response rates

1. The use of the internet based survey will improve response rates because the sampling population (veterinarians and other scientists) frequently uses the internet for research and collaboration.

2. The survey questions are designed with closed questions and utilize skip-logic to decrease the time commitment of the respondent and encourage participation. An estimate of the time commitment is given in Table 1 and will be communicated to respondents on the survey introduction page.

3. An initial introduction to the survey will be disseminated to the members of the target professional associations. This initial introduction will include a link to the survey.

4. One week after the distribution period opens, a reminder/thank you e-mail message will be sent to members of the target professional associations to which the survey was disseminated.

5. Three weeks after the distribution period opens, a final email containing the link to the survey will be sent to the members of the target professional associations.

6. Contact information (telephone and email) for a veterinarian at CVM will be provided so that individuals may contact CVM with any questions.

Justification for the use of an online survey platform as a survey tool

For our target population within the veterinary community, we chose an internet-based survey because we expect it will increase the coverage of the target population, decrease non-response bias, improve the quality and completeness of responses, increase timeliness of data collection, and decrease cost (Dillman, Smyth and Christian 2009). The target population has a high degree of involvement in veterinary professional organizations; therefore, the use of email databases associated with these organizations will ensure that the target population is reached in an efficient manner. The agency intends to use an internet-based survey comprised of an email invitation to an online survey tool hosted rather than telephone or mail survey for the following reasons:

1. Manpower and time commitment: Administration of an internet-based survey requires less time and manpower. The agency does not have sufficient manpower to conduct telephone and mail surveys.

2. Funding: The agency believes that use of a low-cost internet-based survey will reduce the cost to taxpayers.

3. Data quality: The agency expects to obtain quality information from the proposed internet-based survey comprised of email and online survey tool. The survey will be sent to members of selected professional groups by email so that responses obtained will be from targeted participants rather than from random individuals. Use of internet-based surveys have been successfully used to gather high quality data in several peer-reviewed scientific studies (Coen et al 2011; Freedman et al 2011; Hauk and Nogan 2011; Hussain, Alleyne and Jenkins 2009, Morgan and Worsley, 2011; Parsons et al. 2006; Stefandis, Richardson and Fanelli 2010; Swaminath et al 2011). Some studies have suggested that measurement error may be less in internet mode surveys since respondents can complete the survey at their leisure when it is convenient, take as much time as needed, and re-read questions if necessary (Chang and Krosnick 2009). Studies have also shown that item non-response is less for internet surveys (Shin, Johnson and Rao 2012). Additionally, the absence of an interviewer eliminates interviewer and normative bias. Skip logic allows respondents to skip over questions without being burdened with a paper version. These benefits may increase question completion and accuracy of responses. While paper and internet mediums have different strengths and weaknesses, studies have shown that the two media are basically equivalent or that surveys delivered via the internet are better (de Leeuw 2012).

4. Convenience: The agency believes that the proposed internet-based survey is more convenient for responders. A majority of the general U.S. population (82%) uses the internet including 91% of 30-49 year olds, 77% of 50-64 year olds, and 53% of adults 65 and over (Pew Internet 2012). Most veterinarians and veterinary parasitologists are computer literate, have access to the internet, have email addresses, and will be comfortable with internet surveys. This includes access to high-speed internet. Once recent study found that in a sample of veterinarians who used the internet, 83% of respondents had broadband internet service at home and 87% had access to broadband, university, school, or a business network at work (Fleishman-Hillard International Communications 2008). Increasingly, email is the preferred method for distributing communication from the USDA and state veterinary boards and health departments increasing the likelihood that veterinarians will include internet access as an important business tool. No specific hardware or software configuration is required in order to participate in the survey.

4. Test of Procedures or Methods to be Undertaken

The survey was pre-tested by 7 individuals who are presently practicing in equine, food animal, or small ruminant medicine or parasitology. Pre-testing utilized a web URL link and survey identical to the final survey to allow for testing on various hardware and internet browsers configurations as well as to validate question and response choices for clarity and understanding. Comments received during the pre-testing resulted in minor question modification for increased clarity and validity.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Data will be collated by SurveyMonkey and responses will be downloaded both as summaries and detailed Excel spreadsheets. CVM employees listed in Table 2 will be responsible for collecting and analyzing the data. CVM biostatisticians will analyze the results of the survey using statistical methods appropriate to addressing our research objectives as described in the attached document. The statistical methods used will include but not be limited to descriptive, graphical, cross-tabulation, correlation, measurements of association, and logistic regression. CVM biostatisticians will collaborate with CVM veterinarians in writing the final report.

References

Bell, J., Huber, J., and Viscusi, W. K. Survey mode effects on valuation of environmental goods. Int. J. Environ. Res. Public Health 2011, 8, 1222-1243; doi:10.3390/ijerph8041222

Cantor, D., Coa, K., Crystal-Mansour, S., Davis, T., Dipko, S. and Sigman, R. Health Information National Trends Survey (HINTS) 2007 Final Report. Rockville, MD: Westat; 2009.

Cattaneo, A.A, Wilson, R., Doohan, D. and LeJuene, J.T. 2009. Bovine veterinarians’ knowledge, beliefs, and practices regarding antibiotic resistance on Ohio dairy farms. J. Dairy Sci. 92: 3494-3502

Chang, L., and Krosnick, J. A. National surveys via RDD telephone interviewing versus the internet comparing sample representativeness and response quality. Pub. Opin. Q 73 (4): 641–678

Coen, J.R., Carpenter, A.M., Shupp, J.W., Matt, S.E., Shaw, J.D., et al. 2011. The results of a national survey regarding nutritional care of obese burn patients. J. Burn Care & Res. 32(5):561-565.

Coetzee, J.F., Nutsch, A.L., Barbur, L.A. and Bradburn, J.M. 2010. A survey of castration methods and associated livestock management practices performed by bovine veterinarians in the United States. BMC Vet. Res. 6:12.

Curtin, R. Presser, S. and Singer, E. 2005. Changes in telephone survey nonresponse over the past quarter century. Pub. Opin. Quart. 69 (1): 87–98.

de Leeuw, E. D. 2012. Counting and Measuring Online: The Quality of Internet Surveys. Bull. Méth. Soc. 114:68-78.

Dillman, DA.; Smyth, JD and Christian, LM. 2009. Internet, Mail, and Mixed-Mode Surveys. The Tailored Design Method. Third Edition. Hoboken, New Jersey: John Wile & Sons, Inc. 499pp.

Fajt, V.R., Wagner, S.A., and Norby, B. 2011. Analgesic drug administration and attitudes about analgesia in cattle among bovine practitioners in the United States. JAVMA 238 (6): 755-767.

Fleishman-Hillard International Communications. 2008. Digital Clinic Survey. February 18, 2008. Available at: http://www.avma.org/reference/digital_clinic_survey_report.pdf. Accessed 7/9/2012.

Freedman, S.B., Sivabalasundaram, V., Bohn, V., Powell, E.C., Johnson, D.W., and Boutis, K. 2010. The treatment of pediatric gastroenteritis: A comparative analysis of pediatric emergency pyshicians’ practice patterns. Acad. Emerg. Med. 18(1):3 8-44.

Fuchs, M., Bossert, D. and Stukowski, S. 2013. Response rate and nonresponse bias – Impact of the number of contact attempts on data quality in the European Social Survey. Bull. Meth. Soc. 117:26-45.

Groves R.M. and Couper M.P. 1998. Non-response in household interview surveys. New York: Wiley.

Groves, R.M. and Peytcheva, E. 2008. The impact of nonresponse rates on nonresponse bias. A meta-analysis. Public Opin Q 72 (2): 167-189. doi: 10.1093/poq/nfn011

Hauck, R.M. and Nogan, S. 2011. The use of prophylactic antibiotics in plastic surgery. Ann. Plast. Surg. ePub ahead of print. Doi: 10.1097/SAP.0b013e31821e8f9a. Accessed September 6, 2012.

Hox, J.J., de Leeuw, E. D. and Chang, H-T. 2012. Nonresponse versus measurement error: are reluctant respondents worth pursuing? Bull. Méth. Soc. 113: 5–19. doi: 10.1177/0759106311426987

Hussain, A., S. Alleyne, and Jenkins, D. 2010. A survey of attitudes towards methicillin-resistant Staphylococcus aureus bacteraemias amongst United Kingdom microbiologists. J. Antimicrob. Chemother. 65:347-349.

Manfreda, K. L., Bosnjak, M., Berzelak, J., Haas, I., and Vehovar, V. 2008. Web surveys versus other survey modes: a meta-analysis comparing response rates. Int. J. of Market Res. 50 (1): 79-104.

Massey, D.S. and Tourangeau, R. 2013. Where do we go from here? Nonresponse and social measurement. Ann. Am. Acad. Pol. Soc. Sci. 645:222. Available at http://ann.sagepub.com/content/645/1/222.

Maurer, J., Harris, K. M., Parker, A., and N. Lurie. 2009. Does receipt of seasonal influenza vaccine predict intention to receive novel H1N1 vaccine: Evidence from a nationally representative survey of U.S. adults. Vaccine 27: 5732-5734.

Morgan, E. H. and T. Worsley. 2011. Expert perspectives on fruit and vegetable consumption in Australia. Am. J. Health Promot. 26:10-13.

Newell, C.E., Rosenfeld, P., Harris, R.N . and Hindelang, RL. 2004. Reasons for nonresponse on US Navy surveys: A closer look. Military Psychology. 16 (4): 265-276. doi:10.1207/s15327876mp1604_4

Parsons, J.P., O’Brien, J., Lucarelli, M.R., and Mastronarde, J.G. 2006. Differences in the evaluation and management of exercise-induced bronchospasm between family physicians and pulmonologists. J. Asthma 43:379-384.

Peacock, W.F., Bhatt, D.L., Diercks, D., Amsterdam, et al. 2008. Cardiologists’ and emergency physicians perspectives on and knowledge of reperfusions guidelines pertaining to the ST-segment-elevation myocardial infarction. Tex. Heart. Inst. J. 35(2):152-161.

Pew Internet. 2012. Demographics of who’s online. Available at http://pewinternet.org/Static-Pages/Trend-Data-(Adults)/Whos-Online.aspx . Accessed 7/9/2012.

Shih, T-H. and Fan, X. 2008. Comparing response rates from web and mail surveys: a meta-analysis. Field Meth. 20 (3): 249-271.

Shin, E., Johnson, T.P., and Rao, K. 2012. Survey mode effects on data quality: comparison of web and mail modes in a U.S. national panel survey. Soc. Sci. Comp. Rev. 30(2): 212-228.

Stefandis, D., Richardson, and W.S., Fanelli, R.D. 2010. What is the utilization of the SAGES guidelines by its members? Surg. Endosc. 24:3210-3215.

Stowe, C. J. 2010. Results from 2009-2010 AHP Equine Industry Survey. American Horse Publications. Available at http://www.americanhorsepubs.org/resources/AHP_FinalReport.doc. Accessed July 10, 2012.

Swaminath, A., Lebwohl, B., Capiak, K.M., and Present, D.H. 2011. Practice patterns in the use of anti-tumor necrosis factor alpha agents in the management of Crohn’s disease: a US national practice survey comparing experts and non-experts. Dig. Dis. Sci. 56:1160–1164

Tuckel, P. and O’Neill, H. 2002. The vanishing respondent in telephone surveys. J. Adv. Res. 42(5): 26-48.

USDA. 2007. Dairy 2007, Part I: Reference of Dairy Cattle Health and Management Practices in the United States, 2007 USDA:APHIS:VS, CEAH. Fort Collins, CO, #N480.1007

USDA. 2011. NAHMS Feedlot 2011 study. Available online at http://aavld.memberclicks.net/assets/documents/feedlot%20announcement%2011%209%2010%20final.pdf. Accessed September 6, 2012.

Wagner, J. 2010. The fraction of missing information as a tool for monitoring the quality of survey data. Pub. Opin. Quart. 74(2): 223–243

1 https://www.avma.org/KB/Resources/Reference/Pages/Food-Supply-Veterinary-Medicine-data-sources-and-terminology.aspx

Supporting Statement Part B 1