State, Local, Tribal Government

According to the Paperwork Reduction Act of 1995, an agency may not conduct or sponsor, and a person is not required to respond to a collection of information, unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0579-XXXX. The time required to complete this information collection is estimated to average 30 minutes per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collected.		OMB Approved 0579-XXXX EXP: XX/20XX
UNITED STATES DEPARTMENT OF AGRICULTURE ANIMAL AND PLANT HEALTH INSPECTION SERVICE VETERINARY SERVICES NATIONAL ANIMAL HEALTH MONITORING SYSTEM 2150 CENTRE AVE, BLDG B FORT COLLINS, CO 80526	NAHMS Sheep 2024 Study – After Action Survey

To be completed by all individuals involved in the NAHMS Sheep 2024 Study (*required)

.

Study Role

1. Which state did you represent during the Sheep 2024 Study? Pick from drop down list

2. What was your role in the Sheep 2024 Study? *

1. Study coordinator or co-coordinator

2. Federal VMO, not a coordinator

3. Federal AHT, not a coordinator

4. State collaborator (veterinarian or AHT)

5. Other (please specify: __________)

Study Coordinator Only Questions- Only complete if Q2= A

3. In how many NAHMS Studies, including the Sheep 2024 Study, have you been a coordinator?

   1. 1

   2. 2

   3. 3

   4. 4

   5. 5

   6. 6

   7. 7 or more

4. Would you be willing to be a NAHMS coordinator in future NAHMS studies?

   1. Yes

   2. No

If no –

5. Please describe why you would not like to be a NAHMS coordinator in the future:

6. Did you attend the NASS field interviewer training (NASS calls these “schools”) in your state?

   1. Yes

   2. No

If yes –

7. Please rate how valuable the training was to prepare field interviewers to administer the NASS questionnaire. The NASS training was:

   1. Not at all valuable

   2. Slightly valuable

   3. Moderately valuable

   4. Very valuable

   5. Extremely valuable

8. Did you experience any difficulty with NASS in having the consent forms turned over to you? (e.g., difficulty meeting the NASS person, some missing forms, more time spent communicating back-and-forth w/NASS than anticipated, etc.)

   1. Yes

   2. No

If yes –

9. What difficulties did you experience during the turnover of consent forms?

10. Did you use the electronic spreadsheet of producer NAHMS IDs and counties (for those producers that signed consent forms) to plan workloads, make assignments, etc.?

    1. Yes

    2. No

If no –

11. Is there a way that this spreadsheet could be improved to help this be a more useful tool as a coordinator (NOTE: we can’t include full contact information such as street address, phone number, email address, as that information must come from the consent form)?

12. Did you experience any other problems or have any notable successes interacting with NASS? If so, please describe them below:

13. Did you have adequate personnel to contact and visit the sheep operations that consented to the study?

    1. Yes

    2. No

14. Besides VS field staff, please check any of the individuals that you enlisted to help in the sheep study. [Check all that apply.]

    1. State veterinarians or AHTs

    2. Extension or university personnel

    3. University students

    4. Other (please specify:__________)

15. For the Sheep 2024 Study we provided two shipments of kits, based on the estimated biologic response rates. The first shipment of kits were sent to the field office and the second shipment was sent to your specified location. If you needed more kits throughout the study you could order them. Did this method work okay?

    1. Yes

    2. No

If no –

16. Please provide feedback on why this did not work, and insights on how we can improve biologic kit shipments for future studies:

Study Communication

17. Please rate your level of satisfaction with the communication you received from NAHMS during the Sheep 2024 Study. *

    1. Not at all satisfied

    2. Slightly satisfied

    3. Moderately satisfied

    4. Very satisfied

    5. Extremely satisfied

If slightly satisfied or not at all satisfied –

18. Please explain how NAHMS could improve its communication with you:

19. Did you participate in the monthly check in calls?

    1. Yes

    2. No

If yes –

20. Were the monthly check in calls:

    1. Too frequent

    2. Not frequent enough

    3. The right frequency

21. Would you like to have monthly check in calls for future NAHMS studies?

    1. Yes

    2. No

22. Please provide any additional feedback you have regarding the monthly calls: aa122

23. Please provide any additional feedback regarding study communication:

Tableau Dashboard

24. Did you use the Tableau dashboard for the Sheep 2024 Study?*

    1. Yes

    2. No

If yes –

25. Approximately how many times did you log on to use the Tableau dashboard?

    1. Daily

    2. Weekly

    3. Monthly

    4. Less Frequently

26. Please rank each Tableau dashboard page by their usefulness to you during the study. (Skip this question if you did not use the dashboard.)

    1. Welcome Page

    2. Study Timeline and Info

    3. Contact Information

    4. Field Staff Paperwork

    5. Questionnaire Tracking

    6. Biologics Tracking

    7. Response and Completion Rates (Biologics and Questionnaires)

27. If there is other information you would like to see on future dashboards, please describe below:

28. Would you like Tableau dashboards to be used in future NAHMS studies? *

    1. Yes

    2. No

29. If no –

Please tell us why you would not like future dashboards and provide alternative suggestions to help with data tracking.

Consent Forms

30. For the sheep study, we changed the consent form to include the number of sheeps, the type of operation, a clearly marked notes section, and a producer email address. Which of this information did you find useful? [Check all that apply.] aa130

    1. Number of sheeps

    2. Type of operation

    3. Notes section

    4. Producer email address

31. Was there any other information you wish was included on the consent form?

    1. Yes

    2. No

If yes –

32. Please specify what information you would like included:

    1. Write in

33. Were the consent forms filled out adequately with information about the operation, and clear notes regarding how to contact the producer, travel to the operation, or the location of the sheep if they were in a different spot than the address?

    1. Yes

    2. No

If no –

34. Please explain any issues you had with information provided on the consent form:

NAHMS Website

35. Did you use the NAHMS website or cloud during the Sheep 2024 study to obtain study information, outreach material, forms, or to watch webinars or training videos? *

    1. Yes

    2. No

If yes –

36. Approximately how many times did you access the NAHMS website to obtain study information? *

    1. I did not access the website

    2. 1 to 5 times

    3. 6 to 10 times

    4. 11 to 15 times

    5. 16 or more times

37. In previous NAHMS studies we used both an internal SharePoint site and our website. Due to previous feedback we chose to post all material on the website. Did this communication method work okay for you? *

    1. Yes

    2. No

Study Outreach

38. As part of the Sheep 2024 Study, NAHMS tried to increase outreach prior to the study launch. One part of this included providing a list of events for field staff to attend. Did you attend any of these events? aa142

    1. Yes

    2. No

If no –

39. Why did you not attend any events?

If yes --

40. How many different sheep events did you attend?

41. Did you find these events helpful for promoting the study in your state?

    1. Yes

    2. No

42. Would you like NAHMS to include lists of events for future studies?

    1. Yes

    2. No

43. NAHMS posted various outreach material on the website that could be used for conferences, meetings, or events. Did you ever refer to the outreach material on the website?

    1. Yes

    2. No

44. Please rank the following outreach materials by usefulness. (Keep each one individually)

    1. Sheep 2024 Launch Sheet

    2. Biologics Benefits and Timeline Sheet

    3. Sheep 2024 Flyer

    4. Sheep 2024 Presentations

45. Is there other specific outreach material you would like to see made available?

    1. Yes

    2. No

If yes –

46. Please specify what type of outreach materials you would like to see

47. Calling producers to get them to participate in a NAHMS study can involve "selling" the study to the producer. Did you feel comfortable doing this?

    1. Yes

    2. No

48. Did you find the training from NASS about getting producers to participate useful?

    1. Yesa

    2. No

49. Did you use the script provided when contacting producers?

    1. Yes

    2. No

Training

50. Did you participate in or listen to any of the webinars leading up to the study launch date? *

    1. Yes

    2. No

51. Of the webinars leading up to the study launch date, please select the usefulness of the webinar. * Ranking options for each choice include Very useful, moderately useful, useful, not useful, or I did not participate in this webinar.

    1. Study overview webinar held on 2/27/19

    2. Introduction to NASS webinar held on 3/27/19

    3. Questionnaire Overview webinar held on 4/24/19

52. Would you like to see monthly webinars included in future NAHMS studies? *

    1. Yes

    2. No

53. Please list any topics that were not discussed on these monthly webinars that you would like to see included in future studies. *

54. Please rate the value of the NAHMS Sheep 2024 training materials. *

Ranking options for each choice include: Most important and useful, moderately important and useful, slightly important and useful, not important or useful, or did not use at all. (keep the way we had discussed previously)

1. NAHMS Sheep 2024 Training Manual

2. Biologics collection deworming parasites video

3. Biologics collection enteric microbes video

4. Biologics collection sheep restraint video

5. Interdigital swab collection video

6. Sheep industry overview: slides and video

7. Enteric microbess presentation: slides and video

8. Parasites presentation: slides and video Lameness presentation: slides and video

9. Qualtrics presentation: slides and video

10. Questionnaire review presentation: slides and videos

11. Survey methodology presentation: slides and video

12. Tableau dashboard presentation: slides and video

55. NAHMS provided the following resource materials for the Sheep 2024 Study. Please rate the usefulness of each of the documents. Most important and useful, moderately important and useful, slightly important and useful, not important or useful, or did not use at all.

    1. Difficult contacts letter

    2. Phone script for initial call

    3. Participant cheat sheets

    4. Review of double checking the questionnaire

    5. Study coordinator responsibilities sheet

56. Did you receive a NAHMS Sheep 2024 training manual? *

    1. Yes

    2. No

If yes --

57. How many times did you refer to the NAHMS Sheep 2024 training manual during the study?

    1. 0 (not at all)

    2. 1 to 5 times

    3. 6 to 10 times

    4. 11 or more times

58. What changes would make the NAHMS Sheep 2024 Training Manual more useful?

59. What additional training would have been helpful to prepare you for the NAHMS Sheep 2024 study?

Biologics

60. Which of the following was the best incentive for producer participation? (change to ranking)

    1. Parasite resistance testing

    2. Fecal testing for Salmonella, E. coli, Campylobacter, Giardia, and Cryptosporidium

    3. Lameness pathogen testing

61. Did you experience problems at the beginning of the study in obtaining kits for biologic sampling?

    1. Yes

    2. No

62. We could only accept 300 operations for fecal sampling. Did you have operations that would have participated in the fecal sampling if we could have collected more?

    1. Yes

    2. No

63. In previous NAHMS studies we provided an enteric collection schedule, for example we would only allow certain states to collect enteric samples during certain weeks of the study. For future studies, would you prefer a collection schedule, or would you prefer that we limit the number of kits in the field, like we did for the NAHMS Sheep 2024 study?

    1. Prefer an enteric collection schedule

    2. Prefer limiting the kit numbers in the field

64. Please provide any suggestions or ideas on how to limit the number of enteric samples submitted per week.

65. On average, were you able to collect enough fecal material to submit samples for both parasites and fecal pathogens?”?

    1. Yes

    2. No

66. Did you experience problems with the logistics of any of biologic kits?

    1. Yes

    2. No

If yes—

67. Please describe:

68. Did you experience any logistical issues with sample collection and shipping?

    1. Yes

    2. No

If yes –

69. please describe:

70. Please describe any other problems/issues with biologics you had during the Sheep 2024 study:

71. Did you have any issues with finding a FedEx to ship samples?

    1. Yes

    2. No

72. Did you use the pickup feature of FedEx to ship samples?

    1. Yes

    2. No

73. Would you prefer FedEx or UPS for shipping in future studies?

    1. FedEx

    2. UPS

74. Please provide any additional comments or thoughts regarding shipping of biologic samples. Aa202

Visit

75. Please rate the length of the VS Visit questionnaire

    1. Too long

    2. About right

    3. Too short

76. Please rate the length of the VS agritourism questionnaire.

    1. Too long

    2. About right

    3. Too short

    4. Never administered this questionnaire

77. If you had difficulty understanding or administering any of the questions in the VS Visit questionnaire, please list the section letter and question number below and briefly describe why you had trouble with the question(s):

78. What did not go well for you in the Sheep 2024 study? What could NAHMS do to improve your experience in participating in NAHMS studies in the future?

79. What went well for you in the Sheep 2024 study? Could NAHMS have made this any better?

80. If you have any other feedback on the Sheep 2024 study, please describe it below: