NSSP BioSense Platform User Assessment: Functionality and Utility

Attachment B—Instrument: Word Version
Form Approved
OMB No. 0920-0879
Expiration Date 01/31/2021

NSSP BioSense Platform User Assessment
This information collection is for registered users of the National Syndromic Surveillance Program’s
BioSense Platform. The questions ask about user experiences with the BioSense Platform, associated
applications (Access & Management Center [AMC], ESSENCE, Adminer, SAS Studio, and RStudio), the
platform-related quick start guides, and the NSSP Support Desk.

Section 1 – General information about the respondent
1)

Which of the following best describes the work you do?
a. Epidemiology, surveillance, data analytics, or other similar types of work
b. Public health program and operations support, coordination, or management
c. Public health informatics, hardware/software, or other related work
d. Other (If other, please go to question #1a, else go to question #2)
1a)

Please describe the work you do (optional – For those who chose “Other” when responding
to question 1) ______________________

2)

How would you describe your level of expertise with using the BioSense Platform and its
applications to view and analyze data?
a. Advanced
b. Intermediate
c. Novice
d. Unsure of the level

3)

Please indicate your level of agreement with the following statements:

a)

I find the BioSense Platform useful:

Strongly
Agree

Agree

Neutral

Disagree

Strongly
Disagree

O

O

O

O

O

CDC estimates the average public reporting burden for this collection of information as 20 minutes per
response, including the time for reviewing instructions, searching existing data/information sources,
gathering and maintaining the data/information needed, and completing and reviewing the collection of
information. An agency may not conduct or sponsor, and a person is not required to respond to a
collection of information unless it displays a currently valid OMB control number. Send comments
regarding this burden estimate or any other aspect of this collection of information, including
suggestions for reducing burden to CDC/ATSDR Information Collection Review Office, 1600 Clifton Road
NE, MS D-74, Atlanta, Georgia 30333; ATTN: PRA (0920-0879).

Page |2
b)

I find the BioSense Platform easy to
use:

O

O

O

O

O

Section 2 – Utility and functionality of the BioSense Platform’s AMC application

4)

Do you have a site administrator account on the BioSense Platform’s AMC (this account enables
you to create user accounts for others, create user groups, and create data access rules)?
o Yes (If Yes, please go to question 5)
o No (If No, please skip questions 5 to 9 and go to question 10)
o Do not know (If Do not know, please skip questions 5 to 9 and go to question 10)

5)

In the past 6 months, have you used the AMC to create or modify a user account?
o Yes – I have done this activity and did not encounter difficulty
o Yes – I have done this activity (or tried to do this activity) but encountered difficulty (If you
select this answer go to question #5a, else go to question #6)
o No – I have not done this activity
o Do not know
5a)

6)

In the past 6 months, have you used the AMC to create or modify a user group?
o Yes – I have done this activity and did not encounter difficulty
o Yes – I have done this activity (or tried to do this activity) but encountered difficulty (If you
select this answer go to question #6a, else go to question #7)
o No – I have not done this activity
o Do not know.
6a)

7)

Please briefly describe the difficulty (optional - if you indicated encountering difficulty with
the activity) ______________________________

Please briefly describe the difficulty (optional - if you indicated encountering difficulty with
the activity) ______________________________
In the past 6 months, have you used the AMC to create or modify a data access rule?
o Yes – I have done this activity and did not encounter difficulty
o Yes – I have done this activity (or tried to do this activity) but encountered difficulty (If you
select this answer go to question #7a, else go to question #8)
o No – I have not done this activity
o Do not know

Page |3
7a)

8)

Please briefly describe the difficulty (optional - if you indicated encountering difficulty with
the activity) ______________________________
In the past 6 months, have you used the AMC to view or update your site’s Master Facility
Table?
o Yes – I have done this activity and did not encounter difficulty
o Yes – I have done this activity (or tried to do this activity) but encountered difficulty (If you
select this answer go to question #8a, else go to question #9)
o No – I have not done this activity
o Do not know

8a)

9)

Please briefly describe the difficulty (optional - if you indicated encountering difficulty with
the activity) ______________________________
In the past 6 months, have you used the AMC Reports Tab to view the data your site users can
access?
o Yes – I have done this activity and did not encounter difficulty
o Yes – I have done this activity (or tried to do this activity) but encountered difficulty (If you
select this answer go to question #9a, else go to question #10)
o No – I have not done this activity
o Do not know

9a)

10)

In the past 6 months, have you used the AMC to create or change your password?
o Yes – I have done this activity and did not encounter difficulty
o Yes – I have done this activity (or tried to do this activity) but encountered difficulty (If you
select this answer go to question #10a, else go to question #11)
o No – I have not done this activity
o Do not know
10a)

11)

Please briefly describe the difficulty (optional - if you indicated encountering difficulty with
the activity) ______________________________

Please briefly describe the difficulty (optional - if you indicated encountering difficulty with
the activity) ______________________________

Please indicate your level of agreement with the following statements:

a) I find the AMC application useful:

Strongly
Agree

Agree

Neutral

Disagree

Strongly
Disagree

O

O

O

O

O

Page |4
b) I find the AMC application easy to
use:

12)

O

O

O

O

O

Please provide any suggestions you have on how the AMC application can be improved
(optional).

Section 3 – Utility and functionality of the NSSP BioSense Platform’s ESSENCE application

13)

On average, how often do you use the NSSP BioSense Platform’s ESSENCE application?
o Daily
o Once a week or more - but less than daily
o Once a month or more - but less than once a week
o Less than once a month
o I don’t use the NSSP BioSense Platform’s ESSENCE application (if you choose this response,
skip the remaining questions in this section and go to question #28)

14)

In the past 30 days, have you used the Query Portal in ESSENCE to generate/explore a time
series query for a syndrome, sub-syndrome, or any query defined using CC and/or DD?
o Yes – I have done this activity and did not encounter difficulty
o Yes – I have done this activity (or tried to do this activity) but encountered difficulty (If you
select this answer go to question #14a, else go to question #15)
o No – I have not done this activity
o Do not know
14a)

15)

Please briefly describe the difficulty (optional - if you indicated encountering difficulty with
the activity) ______________________________

In the past 30 days, have you used the Query Portal in ESSENCE to run other types of queries
(i.e. other than those mentioned in question 14 above)?
o Yes – I have done this activity and did not encounter difficulty
o Yes – I have done this activity (or tried to do this activity) but encountered difficulty (If you
select this answer go to question #15a, else go to question #16)
o No – I have not done this activity
o Do not know

Page |5
15a)

16)

In the past 30 days, have you used the Data Quality Tab in ESSENCE to generate data quality
metrics such as percent completeness, percent mapped to known values, or percent received
within 24 hours?
o Yes – I have done this activity and did not encounter difficulty
o Yes – I have done this activity (or tried to do this activity) but encountered difficulty (If you
select this answer go to question #16a, else go to question #17)
o No – I have not done this activity
o Do not know
16a)

17)

Please briefly describe the difficulty (optional - if you indicated encountering difficulty with
the activity) ______________________________

In the past 30 days, have you used the myESSENCE Tab in ESSENCE?
o Yes – I have done this activity and did not encounter difficulty
o Yes – I have done this activity (or tried to do this activity) but encountered difficulty (If you
select this answer, go to question #18a, else go to question #19)
o No – I have not done this activity
o Do not know
18a)

19)

Please briefly describe the difficulty (optional - if you indicated encountering difficulty with
the activity) ______________________________

In the past 30 days, have you used the Alerts List Tab in ESSENCE to view or explore alerts
potentially indicating public health events?
o Yes – I have done this activity and did not encounter difficulty
o Yes – I have done this activity (or tried to do this activity) but encountered difficulty (If you
select this answer go to question #17a, else go to question #18)
o No – I have not done this activity
o Do not know
17a)

18)

Please briefly describe the difficulty (optional - if you indicated encountering difficulty with
the activity) ______________________________

Please briefly describe the difficulty (optional - if you indicated encountering difficulty with
the activity) ______________________________

In the past 30 days, have you used the data details function in ESSENCE to review a line listing of
query results?
o
o

Yes – I have done this activity and did not encounter difficulty
Yes – I have done this activity (or tried to do this activity) but encountered difficulty (If you
select this answer go to question #19a, else go to question #20)

Page |6
o
o

19a)

20)

Please briefly describe the difficulty (optional - if you indicated encountering difficulty with
the activity) ______________________________

In the past 30 days, have you used the Map Portal Tab in ESSENCE to view or examine data?
o Yes – I have done this activity and did not encounter difficulty
o Yes – I have done this activity (or tried to do this activity) but encountered difficulty (If you
select this answer go to question #22a, else go to question #23)
o No – I have not done this activity
o Do not know
22a)

23)

Please briefly describe the difficulty (optional - if you indicated encountering difficulty with
the activity) ______________________________

In the past 30 days, have you used the myESSENCE or Query Manager Tab in ESSENCE to share
your data with other users?
o Yes – I have done this activity and did not encounter difficulty
o Yes – I have done this activity (or tried to do this activity) but encountered difficulty (If you
select this answer go to question #21a, else go to question #22)
o No – I have not done this activity
o Do not know.
21a)

22)

Please briefly describe the difficulty (optional - if you indicated encountering difficulty with
the activity) ______________________________

In the past 30 days, have you used the Query Manager Tab in ESSENCE to view or modify saved
queries?
o Yes – I have done this activity and did not encounter difficulty
o Yes – I have done this activity (or tried to do this activity) but encountered difficulty (If you
select this answer go to question #20a, else go to question #21)
o No – I have not done this activity
o Do not know
20a)

21)

No – I have not done this activity
Do not know

Please briefly describe the difficulty (optional - if you indicated encountering difficulty with
the activity) ______________________________

In the past 30 days, have you used the Reports Tab in ESSENCE to generate reports?
o Yes – I have done this activity and did not encounter difficulty

Page |7
o
o
o
23a)

24)

Yes – I have done this activity (or tried to do this activity) but encountered difficulty (If you
select this answer go to question #23a, else go to question #24)
No – I have not done this activity
Do not know
Please briefly describe the difficulty (optional - if you indicated encountering difficulty with
the activity) ______________________________

Please indicate your level of agreement with the following statements:
Strongly
Agree

Agree

Neutral

Disagree

Strongly
Disagree

a) I find ESSENCE useful:

O

O

O

O

O

b) I find ESSENCE easy to use:

O

O

O

O

O

25)

Please identify the top three activities you commonly perform with ESSENCE (or other
applications on the BioSense Platform)
(e.g., routinely monitor syndromes/sub-syndromes, analyze opioid crisis-related data, support
response to local public health issues)

26)

In addition to emergency department (ED) data, what other data do you commonly use (or
would most like to use) in your syndromic surveillance activities?
Please identify up to three types or sources of data:

27)

Please provide any suggestions you have on how the ESSENCE can be improved (optional):

Section 4 – Utility and functionality of the NSSP BioSense Platform’s Adminer, RStudio, and SAS
Studio applications
28)

Do you have access to the Master Facility Table (MFT) data or pre-ESSENCE data (e.g., the data
in Raw Messages Table, Exceptions Table, Processed Messages Table) for your site in the
BioSense Platform?
o Yes (If Yes, please go to question number 29)
o No (If No, please skip the remaining questions in this section and go to question number
36)

Page |8
o

Do not know (If Do not know, please skip the remaining questions in this section and go to
question number 36)

29)

Do you use (or have you used) any of the following applications to view, verify, or analyze the
Master Facility Table (MFT) data or pre-ESSENCE data for your site (e.g., the data in Raw
Messages Table, Exceptions Table, Processed Messages Table)? (Please check all that that
apply.)
o Adminer
o RStudio
o SAS Studio
o No, I have not used these applications for MFT or pre-ESSENCE data (If No, please skip
the remaining questions in this section and go to question n#36)
o Do not know (If Do not know, please skip the remaining questions in this section and go
to question #36)

30)

In the past 30 days, have you viewed or verified your site’s facilities in the Master Facility Table
(MFT) in the BioSense Platform?
o Yes – I have done this activity and did not encounter difficulty
o Yes – I have done this activity (or tried to do this activity) but encountered difficulty (If you
select this answer go to question #30a, else go to question #31)
o No – I have not done this activity
o Do not know
30a)

31)

In the past 30 days, have you used Adminer to view or analyze pre-ESSENCE data for your site
(e.g., the data in Raw Messages Table, Exceptions Table, Processed Messages Table) in the
BioSense Platform?
o Yes – I have done this activity and did not encounter difficulty
o Yes – I have done this activity (or tried to do this activity) but encountered difficulty (If you
select this answer go to question #31a, else go to question #32)
o No – I have not done this activity
o Do not know
31a)

32)

Please briefly describe the difficulty (optional - if you indicated encountering difficulty with
the activity) ______________________________

Please briefly describe the difficulty (optional - if you indicated encountering difficulty with
the activity) ______________________________

In the past 30 days, have you used RStudio to view or analyze pre-ESSENCE data for your site
(e.g., the data in Raw Messages Table, Exceptions Table, Processed Messages Table) in the
BioSense Platform?

Page |9
o
o

Yes – I have done this activity and did not encounter difficulty
Yes – I have done this activity (or tried to do this activity) but encountered difficulty (If you
select this answer go to question #32a, else go to question #33)
o No – I have not done this activity
o Do not know
32a) Please briefly describe the difficulty (optional - if you indicated encountering difficulty with the
activity) ______________________________

33)

Did you know SAS Studio is now available to users on the BioSense Platform?
o Yes.
o No.

34)

In the past 30 days, have you used SAS Studio to view or analyze pre-ESSENCE data for your site
(e.g., the data in Raw Messages Table, Exceptions Table, Processed Messages Table) in the
BioSense Platform?
o Yes – I have done this activity and did not encounter difficulty
o Yes – I have done this activity (or tried to do this activity) but encountered difficulty (If you
select this answer go to question #34a, else go to question #35)
o No – I have not done this activity
o Do not know
34a)

35)

Please briefly describe the difficulty (optional - if you indicated encountering difficulty with
the activity) ______________________________

Please provide any suggestions you have on how the Adminer, RStudio, and/or SAS Studio
applications can be improved (optional):

Section 5 – Utility and functionality of the quick start guides and the NSSP Support Desk and training
preferences

36)

On a scale of 1 to 5 (with 1 being only a little useful and 5 being extremely useful), please
indicate how useful you have found the following quick start guides for helping you to begin
using (or as a quick reference about the features and functions of) the BioSense Platform
applications.
Not at all
useful

The AMC Quick Start Guide

1

Extremely
useful

2

3

4

5

P a g e | 10
The ESSENCE Quick Start Guide

1

2

3

4

5

The Adminer Quick Start Guide

1

2

3

4

5

The SAS Studio Quick Start Guide

1

2

3

4

5

The RStudio Quick Start Guide

1

2

3

4

5

The Master Facility Table Quick Start
Guide

1

2

3

4

5

37)

In the past 30 days, have you used the NSSP Service Desk for any BioSense Platform-related
support needs?
o Yes (If Yes, go to question number 38)
o No (If No, go to question number 39)
o Do not know (If Do not know, go to question number 39)

38)

Please indicate your level of agreement with the following statement:

The NSSP Service Desk has been
helpful in addressing my BioSense
Platform-related support needs

Strongly
Agree

Agree

Neutral

Disagree

Strongly
Disagree

O

O

O

O

O

39)

Please provide any suggestions you have on how the NSSP Service Desk can be improved.
(Optional)

40)

Please provide comments or suggestions you have about information materials (e.g., quick start
guides) or trainings you would like on how to use the BioSense Platform and its applications
(e.g., AMC, ESSENCE, Adminer, SAS Studio, and RStudio). (Optional)

Thank you for participating in this assessment.