SVIS - G:Online Assessment - word

Form Approved

OMB No. 0920-0879

Exp. Date 04/30/2017

ATTACHMENT – G: Online Instrument Word

Examination of IIS Services and Value in a Changing Landscape

Introduction and Assessment Context

This assessment is being conducted as part of the NCIRD-sponsored project, Examination of IIS Services and Value in a Changing Landscape (IIS-ESV).

The purpose of this assessment is to assess the major functions and services of IIS grantees and the associated costs, as well as identify opportunities for IIS to share those services. The responses will ultimately help CDC to better understand the IIS current state and further the success of IIS. The assessment consists of 3 sections and has a total of 27 questions. The assessment should take no more than 45 minutes to complete. You can exit the assessment and your progress will be saved for later, but responses on the current section will be lost. You may find it helpful to work collaboratively with other program staff to complete the assessment.

If you have any questions regarding this assessment or would like to provide any additional information, you can contact Anne Cordon via email at [email protected] or via phone at 470-246-4218.

Instructions 

Answers are required for all questions. Please select “Not Applicable” in dropdowns (where applicable) or enter "N/A" for any open text fields for which you have no input. 

In order to progress through this assessment, please use the following navigation buttons:

• Click the Next button to continue to the next page.

• Click the Previous button to return to the prior page.

• Click the Submit button on the last page to complete and submit your assessment.

• Click the Exit the Assessment Early button at the top of the page to exit the assessment. Note: This will not save the answers on the current page, but will save other answers. To save the current page and exit first click Next or Previous.

The link for you to respond to this assessment is unique for your email address; that is how we know who has responded. We understand you may need to consult with others in your organization to answer all the questions, but it is preferable that a single person enter the information and submit the finished response. This link will preserve saved responses if the assessment is filled out in more than one session or location. Please remember to press Next or Previous to save data from the current page.

Examination of IIS Services and Value in a Changing Landscape

Section 1 – Process and Functionality

 

The following definitions may help in answering the questions in this section.

Service: is any process, procedure or activity that supports an IIS or is a function of an IIS application or program. Services may include automated functions, IT infrastructure delivered as a service e.g. server hosting, and manual processes. The primary focus of the IIS-ESV project is automated functions and IT infrastructure. The secondary focus, manual processes, may include back office functions such as help desk and IT support but may also be administrative support in the form of technical consulting on IT, legal or regulatory consulting and similar service engagements. 

Shared service: is a service that can be used by more than one organization such as sharing among IIS or sharing between IIS and state agencies. This may include both “technical services” and “support services.” The objective of shared services is to gain cost reductions and improved quality through the elimination of duplicative efforts and adoption of standardization. This will be the primary focus of the IIS-ESV team in developing ideas for inclusion in a "catalog" of services that can be shared across IIS.

1. Below is a list of common technical functions performed by an IIS. For each function, please indicate how well your IIS performs this function 1) optimally (i.e. your program is fully satisfied with the ability of the IIS to complete this function); 2) adequately (i.e. your IIS performs this function but enhancements would need to be made for optimization), 3) marginally (i.e. there are many key gaps in functionality), or 4) does not perform this function

	Capability assessment
a) Process Newborn Record	Dropdown menu
b) Process Vital Record Amendments	Dropdown menu
c) Maintain Facility/Organization Registration	Dropdown menu
d) Record Vaccine Administration Event	Dropdown menu
e) Identify Vaccine Funding Source (public or private)	Dropdown menu
f) Identify Vaccine Dose-Level VFC Eligibility	Dropdown menu
g) Query, Add, and/or Update Patient Record	Dropdown menu
h) Manage Inventory By the IIS	Dropdown menu
i) Manage Public Purchase Vaccine Ordering	Dropdown menu
j) Manage Forecasting Rules (actual IIS functionality)	Dropdown menu
k) Generate User/Provider Reports	Dropdown menu
l) Generate IIS management reports	Dropdown menu
m) Conduct Patient Reminder/Recall	Dropdown menu
n) EHR/IIS Data Exchange	Dropdown menu
o) Manage Onboarding for EHR/IIS Data Exchange	Dropdown menu
p) Perform Ongoing Data Quality Monitoring for EHR/IIS Data Exchange	Dropdown menu
q) Perform Patient Record De‐Duplication	Dropdown menu
r) Perform Vaccine Dose De‐Duplication	Dropdown menu

2. Describe how your IIS produces these reports and the level of effort, cost and value

	What is the relative level of effort/cost to produce these reports (e.g. staff time, financial costs)?	How valuable are these reports to your IIS in fulfilling program requirements?	How valuable are these reports to stakeholders?
a) Immunization Coverage	Dropdown menu	Dropdown menu	Dropdown menu
b) Vaccine Inventory	Dropdown menu	Dropdown menu	Dropdown menu
c) VFC Accountability	Dropdown menu	Dropdown menu	Dropdown menu
d) Reports for external stakeholders	Dropdown menu	Dropdown menu	Dropdown menu
e) IIS internal operational (audits)	Dropdown menu	Dropdown menu	Dropdown menu
f) Individual immunization reports to clients	Dropdown menu	Dropdown menu	Dropdown menu
g) School related reports	Dropdown menu	Dropdown menu	Dropdown menu
h) Interoperability reports (results of data transfer)	Dropdown menu	Dropdown menu	Dropdown menu
i) Other reports

Please describe any reports in item i) Other reports.

3. If a service/process does not fully meet your IIS requirements, please select all reasons that apply:

	Not Applicable/ Fully meets IIS needs	Need improved functionality for users	Have new mandates or policies to follow	Need increased efficiency	Environmental factors external to your IIS operations	Other reason(s) not listed
a) Process Newborn Record						
b) Process Vital Record Amendments						
c) Maintain Facility/Organization Registration						
d) Record Vaccine Administration Event						
e) Identify Vaccine Funding Source						
f) Identify Vaccine Dose-Level VFC Eligibility						
g) Query, Add, and/or Update Patient Record						
h) Manage Inventory By the IIS						
i) Manage Public Purchase Vaccine Ordering						
j) Manage Forecasting Rules (actual IIS functionality)						
k) Generate User/Provider Reports						
l) Generate IIS management reports						
m) Conduct Patient Reminder/Recall						
n) EHR/IIS Data Exchange						
o) Manage Onboarding for EHR/IIS Data Exchange						
p) Perform Ongoing Data Quality Monitoring for EHR/IIS Data Exchange						
q) Perform Patient Record De‐Duplication						
r) Perform Vaccine Dose De‐Duplication						

Please elaborate on any barriers to optimization e.g. limited financial resources, limited skills, competing priorities, legal / policy barriers, etc.

4. For the services in Question 1, please rate the technical feasibility for sharing the service with other IIS, for example by “modularizing” the service. Rate the services on a scale from 1-5, with 1 the least feasible and 5 being the most feasible

Service	Sharing Feasibility
a) Process Newborn Record	Dropdown menu
b) Process Vital Record Amendments	Dropdown menu
c) Maintain Facility/Organization Registration	Dropdown menu
d) Record Vaccine Administration Event	Dropdown menu
e) Identify Vaccine Funding Source	Dropdown menu
f) Identify Vaccine Dose-Level VFC Eligibility	Dropdown menu
g) Query, Add, and/or Update Patient Record	Dropdown menu
h) Manage Inventory By the IIS	Dropdown menu
i) Manage Public Purchase Vaccine Ordering	Dropdown menu
j) Manage Forecasting Rules (actual IIS functionality)	Dropdown menu
k) Generate User/Provider Reports	Dropdown menu
l) Generate IIS management reports	Dropdown menu
m) Conduct Patient Reminder/Recall	Dropdown menu
n) EHR/IIS Data Exchange	Dropdown menu
o) Manage Onboarding for EHR/IIS Data Exchange	Dropdown menu
p) Perform Ongoing Data Quality Monitoring for EHR/IIS Data Exchange	Dropdown menu
q) Perform Patient Record De‐Duplication	Dropdown menu
r) Perform Vaccine Dose De‐Duplication	Dropdown menu

Please provide any additional comments

5. Please review the list of IIS support services and select the response that best describes how your IIS manages these services:

	Entirely within the IIS	Central IT within Health Department	Central IT outside of Health Department	Outsourced to 3rd party	Other/ Explain	Not applicable	Don't know
a) Server hosting	O	O	O	O	O	O	O
b) Software licensing	O	O	O	O	O	O	O
c) Software development	O	O	O	O	O	O	O
d) Software maintenance	O	O	O	O	O	O	O
e) Software testing, bug fixes, etc.	O	O	O	O	O	O	O
f) Interoperability-related services	O	O	O	O	O	O	O
g) Data Quality Review (manual)	O	O	O	O	O	O	O
h) Ad hoc report generation / data analysis (to elaborate further on different types of reports, please use the comments box below)	O	O	O	O	O	O	O
i) Vaccine forecasting / clinical decision support (interpretation of ACIP recommendations for coding)	O	O	O	O	O	O	O
j) Project management and planning	O	O	O	O	O	O	O
k) Business Requirements	O	O	O	O	O	O	O
l) Support of Immunization Program functions, such as VFC, AFIX	O	O	O	O	O	O	O
m) Grant-writing	O	O	O	O	O	O	O
n) Marketing (includes outreach, education, awareness, etc.)	O	O	O	O	O	O	O
o) Provider onboarding, orientation, and initial training	O	O	O	O	O	O	O
p) Ongoing user training	O	O	O	O	O	O	O
q) Help desk	O	O	O	O	O	O	O

If any of the responses to question 4 were “Other/Explain,” please elaborate here

6. For the support services listed in Question 4, please rate the technical feasibility for sharing the service with other IIS, for example by “modularizing” the service. Rate the services on a scale from 1-5, with 1 the least feasible and 5 being the most feasible

Service	Sharing Feasibility
a) Server hosting	Dropdown menu
b) Software licensing	Dropdown menu
c) Software development	Dropdown menu
d) Software maintenance	Dropdown menu
e) Software testing, bug fixes, etc.	Dropdown menu
f) Interoperability-related services	Dropdown menu
g) Data Quality Review (manual)	Dropdown menu
h) Ad hoc report generation / data analysis (to elaborate further on different types of reports, please use the comments box below)	Dropdown menu
i) Vaccine forecasting / clinical decision support (interpretation of ACIP recommendations for coding)	Dropdown menu
j) Project management and planning	Dropdown menu
k) Business Requirements	Dropdown menu
l) Support of Immunization Program functions, such as VFC, AFIX	Dropdown menu
m) Grant-writing	Dropdown menu
n) Marketing (includes outreach, education, awareness, etc.)	Dropdown menu
o) Provider onboarding, orientation, and initial training	Dropdown menu
p) Ongoing user training	Dropdown menu
q) Help desk	Dropdown menu

Please provide any additional comments

7. Please list and describe any major IIS service or support service not listed in Questions 1 and 5 which your IIS is:

Currently performing

Planning to perform

Would like to perform but there are barriers, please describe

8. Could any of the additional services in Question 6 be reused or shared with other IIS? If so, please state which service(s):

• Yes

• No

• Not applicable

• Don't know

If Yes, please state which service(s):

9. Which of the following best describes your IIS's IT Solution?

• Homegrown system / developed in-house

• Open source system / free licensing/ public domain

• 3rd party / vendor-owned system / licensing at cost

• Not applicable

• Don't know

10. Who is primarily responsible for operating and maintaining your IT Solution?

• Primarily operated and maintained internally by IIS Staff

• Primarily operated and maintained by State IT Staff

• Primarily operated and maintained by 3rd Party / Vendor Staff (Outsourced)

• Not Applicable

• Don't Know

Examination of IIS Services and Value in a Changing Landscape

Section 2 - Financials

 

The questions in this section will ask about the current state cost for your IIS and estimates of what your IIS would need to reach optimal performance (i.e. capacity, functionality, staffing, etc.). Other questions will delve into details about current staffing and an estimate of time spent on various activities. The final section will ask about sources of current funding and experience with obtaining funding outside of 317 or VFC.

11. Please list your current costs in each of the following categories. We do not expect these answers to be exact, but rather to give a general sense of where costs are incurred. Please enter your responses as whole numbers with no decimals.

Cost Category	Current State Total Cost
Personnel (include non-contractual IIS staff that reside within the IIS, as well as immunization program, public health department or IT personnel within the jurisdiction)
Contractual expenses (include software licenses, contractual personnel, as well as ongoing support/maintenance from vendors)
Purchasing of materials, equipment, and supplies
Travel
Fringe
Indirect
Other
TOTAL (Note: this should be the sum of all the rows above)

12. Referring to the previous question, please estimate your total spending needs in each of the following categories for you to achieve your ideal IIS future state, if funding were not an issue. We do not expect these answers to be exact, but rather to give a general sense of where there is the largest need for additional spending to achieve IIS goals. Please enter your responses as whole numbers with no decimals.

Cost Category	Ideal Future State Total Cost
Personnel (include non-contractual IIS staff that reside within the IIS, as well as immunization program, public health department or IT personnel within the jurisdiction)
Contractual expenses (include software licenses, contractual personnel, as well as ongoing support/maintenance from vendors)
Purchasing of materials, equipment, and supplies
Travel
Fringe
Indirect
Other
TOTAL (Note: this should be the sum of all the rows above)

13. For each of the following categories, please describe the major items (top 1 to 5) that you would spend the additional funding on to achieve your ideal IIS future state.

Cost Category	Major items to spend additional funding on (top 1-5)
Personnel (include non-contractual IIS staff that reside within the IIS, as well as immunization program, public health department or IT personnel within the jurisdiction)
Contractual expenses (include software licenses, contractual personnel, as well as ongoing support/maintenance from vendors)
Purchasing of materials, equipment, and supplies
Travel
Fringe
Indirect
Other
TOTAL (Note: this should be the sum of all the rows above)

14. How many FTEs (Full Time Equivalents) support the IIS? Please include dedicated full time employees, as well as the partial time of immunization program employees and State IT personnel supporting the IIS. For example, 5 full time IIS employees and 1 who dedicates half time to IIS would be 5.5 FTEs)

FTE Category	# of FTEs
FTEs from IIS Staff
FTEs from Immunization Program Personnel
FTEs from State IT Personnel
FTEs from Other State Programs/Departments, not including contractors

15. Please allocate your total current personnel cost reported earlier in Question 11 across the following categories. Please enter the percentage of internal IIS staff time, as whole numbers with no decimals (e.g. 25% would be entered as 25). Your responses must all sum to 100. What percentage goes toward...?

Activity	% of Staff Time
Software development and maintenance
Other IT and data-related services, including data exchange, data quality, and database administration
Reporting, analytics, vaccine forecasting, and clinical decision support
Onboarding activities, including outreach, marketing, training, and help desk
Programmatic activities, including project management, community involvement, and Support of Immunization Program functions such as VFC and AFIX
Other (Business requirements development, grant writing, etc.

16. Please allocate your total current contractual cost reported earlier in Question 11 across the following categories. Please enter the percentages as whole numbers with no decimals (e.g. 25% would be entered as 25). Your responses must all sum to 100. What percentage of your contractual spend (if any) goes to...

Cost Category	% Contractual Cost
A vendor for licensing of a software product?
A vendor for software development?
A vendor for business analysis?
A vendor for database maintenance?
A vendor for project management?
A vendor for rent / lease of hardware or equipment?
The State for support / maintenance / staffing?
The State for usage of equipment or space?
Some other category?

17. Have you tried in the past or do you currently engage in any activities to save costs by working with other entities, including but not limited to other IIS, public health programs, state-run agencies, immunization coalitions, etc.? This would include any sharing of personnel, functions, IT system components, or materials.

• Yes

• No

• Not applicable

• Don't know

If Yes, please describe those cost sharing efforts.

18. Please provide any comments or clarifications regarding the cost-related questions 11-17.

19. Quantify the sources of funding your IIS received in your most recently completed fiscal year. This must be entered as whole dollar amounts (no decimals or commas).

Source of Funding	Amount of Funding
Total Funding from all sources
317 (incl. VFC and DA)
State
Local
Private
In-kind
CMS / MMIS
CMS / HITECH
HRSA
Other federal programs
Emergency preparedness
Non-Profit
Other

Please provide any additional comments or clarifications

20. Please describe your experience with funding sources other than VFC or 317 funding. This may include fee-for-service, private funding, funding from your jurisdiction or other public sources, etc. Please note that Medicaid funding is specifically discussed in the following question, so does not need to be mentioned here.

	Yes	No	Don’t Know	N/A	Please Describe
Have you attempted to secure any such funding?	O	O	O	O
Are you currently receiving any such funding?	O	O	O	O
Have you had any successes in the past securing such funding?	O	O	O	O
Have you had any challenges securing such funding?	O	O	O	O

21. Please describe your awareness of the following funding options which may be available through CMS, ARRA, and your state Medicaid Program.

	Are you aware of this funding option?	Have you applied for this funding option?	Are you currently receiving this funding?
CMS Health Information Technology for Economic and Clinical Health (HITECH) funds	Dropdown menu	Dropdown menu	Dropdown menu
CMS Medicaid Management Information Systems Program (MMIS) funds	Dropdown menu	Dropdown menu	Dropdown menu

Please elaborate on any other kind of funding your IIS receives through CMS

22. In regards to the previous question on CMS Funding, please elaborate further on the following:

If you have not applied for this funding, please explain why?
If you have applied for this funding, what have been the challenges, if any, actually receiving the funds?

Please provide any additional comments or clarifications

23. Please provide any comments or clarifications regarding the funding-related questions 19-22.

Examination of IIS Services and Value in a Changing Landscape

Section 3 – Assessment Feedback

 

Please provide feedback on the assessment instrument and process so that we can improve future data gathering.

24. Do you feel that the assessment responses will be helpful in understanding how to further the success of IIS?

• Yes

• No

If no, please identify any important content areas we may have missed

25. What is your perspective on the time commitment needed to complete the assessment?

• It took too long to complete and was too burdensome

• It took a reasonable and appropriate amount of time to complete

• It was too brief and did not ask enough questions

Please add any additional comments

26. Did you have any technical difficulties in completing the assessment?

• Yes

• No

If yes, please explain

27. Is there anything else you would like the CDC to know, either about your IIS or about the assessment instrument?

Examination of IIS Services and Value in a Changing Landscape

Conclusion

 

Thank you for your participation in this assessment. The responses will ultimately help CDC to better understand the IIS current state and further the success of IIS.

If you have any questions regarding this assessment or would like to provide any additional information, you can contact Anne Cordon via email at [email protected] or via phone at 470-246-4218.

Public reporting burden of this collection of information is estimated to average 45 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number.  Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to CDC/ATSDR Information Collection Review Office, 1600 Clifton Road NE, MS D-74,  Atlanta, Georgia 30333; ATTN:  PRA (0920-0879).