Challenges in Implementing CDC Immunization Funding in Health Departments
OSTLTS Generic Information Collection Request
OMB No. 0920-0879
Supporting Statement – Section A
Submitted: April 8, 2014
Program Official/Project Officer
Jack Nemecek, PhD, MBA
Public Health Advisor
Centers for Disease Control and Prevention (CDC)
National Center for Immunization and Respiratory Diseases
Immunization Services Division
Program Operations Branch
1600 Clifton Rd. N.E.
Mailstop: A-19
Atlanta, GA 30333
Telephone: 404-639-8219
Fax: 404-639-8615
Section A – Justification
Circumstances Making the Collection of Information Necessary
Background
This data collection is being conducted using the Generic Information Collection mechanism of the OSTLTS Survey Center (OSC) – OMB No. 0920-0879. The respondent universe for this data collection aligns with that of the OSC. This data collection is authorized by Section 301 of the Public Health Service Act (42 U.S.C. 241).
Data will be collected from immunization program managers at the funded 64 state, local, and territorial health departments acting in their official capacities (50 states, 8 territories, and 6 cities including the District of Columbia). Each health department has one immunization program manager related to this work, and a listing of the health department awardees involved in this assessment can be found in Attachment A – List of Health Department Awardees.
Vaccine-preventable diseases have a costly impact on the United States, resulting in lost work productivity, increased hospitalizations, and premature deaths.1 It is estimated that more than 50,000 adults die each year in the U.S. from vaccine-preventable diseases, such as influenza, pneumonia, and Hepatitis B.2 Vaccines for both adults and children are an important public health intervention to curb these negative health outcomes. In addition to reducing morbidity and mortality, routinely recommended pediatric vaccines have been estimated to save $9.9 billion in direct costs and $43.3 billion in societal costs over the lifetime of a single-year birth cohort.3
Although immunization rates generally have been increasing over the last ten years, immunization coverage is not equal across the United States. Lower county-level vaccination rates for pediatric vaccines (i.e. rates for children 19-35 months) have been found to be significantly associated with higher poverty per capita, a higher percentage of black children among children aged <5 years, higher levels of housing stress (i.e., ≥30% income for rent or mortgage and certain inadequate housing characteristics), and a higher number of pediatric intensive care beds per capita.4
As
noted by the National Vaccine Advisory Committee, the United States
vaccine infrastructure is a large, multifaceted system comprised of
many components spanning the entire life-cycle from basic vaccine
research, development, testing, licensure, and widespread use.5,6
State, local, and territorial health departments play a critical
role in this system, particularly in the acquisition and delivery of
vaccines to the public. In their role, health departments work
with public and private physicians to assure effective immunization
practices, including proper storage and delivery of vaccines. Through
the use of immunization information systems administered by health
departments, savings are achieved by avoiding duplicative
vaccinations, improving inventory management, and identifying gaps in
immunization coverage among persons and groups.
To support health departments in this work, CDC provided funding in Fiscal Year 2012 and 2013 to 64 state, local, and territorial health departments via three major federal funding streams. These funding streams included: 1) the Vaccines for Children (VFC) program, 2) Section 317 funds (from the Public Health Service Act), and 3) Special federal funding such as from the Prevention and Public Health Fund (PPHF) or American Recovery and Reinvestment Act (ARRA). These three funding streams supported health departments in their operations, infrastructure, and vaccine acquisition to improve immunization coverage in their specified geography. In Fiscal Year 2012 alone, CDC allocated over $317 million in vaccine-related funding through the VFC and Section 317 funding streams.
Upon review of awardee progress and annual reports on met and unmet performance goals and expenditures of grant funding, the Immunization Services Division at CDC’s National Center for Immunization and Respiratory Diseases (NCIRD) has identified that many health department awardees encounter challenges in utilizing their grant awards and implementing their proposed activities in a timely manner.
The purpose of this assessment is part of the Division’s continuous quality improvement effort to address health departments’ challenges and successes in implementing federal immunization grant funding as well as to identify opportunities for improvement to facilitate a more streamlined funding implementation process in the future.
Privacy Impact Assessment
Overview of the Data Collection System – The data collection system consists of a Phase 1 web-based instrument (see Attachment B: Web Version of Web-based Instrument and Attachment C: MS Word Version of Web-based Instrument) and Phase 2 follow-up telephone interviews (see Attachment D: Interview Guide).
The Phase 1 web-based instrument is designed to gather information from the immunization program managers of the 64 health departments receiving federal immunization funding about their utilization of the funding, challenges of implementing grant activities, successes, and suggested improvements to the process for future funding cycles. The data collection instrument will be administered as a web-based data collection tool. The data collection tool was pilot tested with six (6) public health professionals who have worked in state or large local health departments or worked closely with health departments and have experience with federal immunization funding. Feedback from this group was used to refine questions as needed, ensure accurate programming and skip patterns, and establish the estimated time required to complete the questions.
The Phase 2 telephone interviews are designed to build off of the responses from the web-based instrument and gather more nuanced information and examples from a sample of immunization program managers who complete the web-based instrument. After the analysis of the web-based instrument responses, the telephone interviews will be conducted with a sample of 24 immunization program managers (4 managers randomly selected among each of 6 stratified segments; see Statement B) to complement the close-ended responses from the web-based instrument and to provide a more nuanced, comprehensive portrait with illustrative examples of the challenges, facilitators, and suggestions for improvement around federal immunization funding. This phase 2 of the data collection process will be administered as a semi-structured telephone interview guide. The data collection tool was pilot tested with two (2) public health professionals who completed the initial pilot testing of the web-based instrument, since the interview guide probes into the web-based instrument responses in more depth. Feedback from this group was used to refine questions as needed and establish the estimated time required to complete the interview.
Items of Information to be Collected –
Web-based Instrument: There are a maximum of 36 questions in the web-based instrument; yet, due to the automatic skip patterns, it is expected that not all respondents will be presented with all questions. Twenty-eight questions are multiple choice or scale-based, and eight are open ended. For many of the multiple-choice questions, respondents have the opportunity to add another response option under the “Other, please specify” answer choice. Six of the open-ended questions provide respondents with the opportunity to give a narrative response after a question for further elaboration. This instrument will be distributed and data will be collected using the web-based data collection tool, Qualtrics®. The web-based instrument will collect information on the following:
Respondent and health department characteristics (required) – (4 questions) whether they are the immunization program manager (eligibility requirement for the instrument), their health department of employment, length of service at the health department, staffing size of the health department’s immunization program (multiple response and open-ended format)
Health department experiences with using federal immunization funds – three separate sections, one section per funding initiative (Section 317, VFC, and Special Federal Funding). Each section includes 9 questions, with the first one asking if the health department received this specific funding in the last two fiscal years. If no, the respondent skips the entire section of questions on that funding stream. Questions in each section include: challenges and facilitators to spending the specific federal funding, overall ease or difficulty in spending the funding, percent of funding spent, areas for which it was easiest and most challenging to spend money, and suggestions that would be most help health department awardees spend that funding in the future (multiple response, rating, and open-ended format)
Health department experiences with using other funding – (2 questions) percent of funds from state or local sources, overall ease or difficulty in spending state or local funding (multiple response and rating)
Other issues – (3 questions) whether health departments experienced any other issues (not yet named) to spending federal, state, or local immunization funding, if yes among which funding streams, overall suggestions on resources, information, or assistance that can help health departments in the future spend their immunization funding (multiple response, rating, and open-ended format)
Interview Guide: There are three main questions in the semi-structured telephone interview guide with probes under each question. If all probes are asked, then the guide totals nine questions. All questions are open-ended and ask the respondent to provide specific examples that directly build on his/her responses from the web-based instrument to provide a more nuanced and comprehensive portrait of the health departments’ experiences.
Prior to the interviews, responses from the web-based instrument will be analyzed. Based on responses in the pilot-testing of the web-based instrument and exploratory conversations with stakeholders, it is expected that health department challenges and facilitators will be clustered into specific overall categories and may be similar across all funding streams. Therefore, the interview guide will collect information on the following:
Specific examples of challenges to spending federal funding that were identified on the web-based instrument as being experienced across all funding streams (open-ended question); an additional probe will be asked if a unique set of challenges emerged under a specific funding stream
Specific examples of facilitators to spending federal funding that were identified on the web-based instrument as being experienced across all funding streams (open-ended question); an additional probe will be asked if a unique set of facilitators emerged under a specific funding stream
Elaboration on implementation of specific suggestions for improvement for future funding processes between health departments and CDC (open-ended question with a probe on what CDC would need to consider in implementation); an additional probe will be asked if a unique set of suggestions emerged under a specific funding stream
Identification of Website(s) and Website Content Directed at Children Under 13 Years of Age –No website content will be directed at children.
Purpose and Use of the Information Collection
As part of a continuous quality improvement effort, the purpose of this information collection is to allow CDC’s Immunization Services Division to understand the experiences of health departments in using their federal immunization funding and how to improve this utilization in the future. The web-based instrument and interviews will identify: 1) the challenges health departments have encountered in utilizing this funding; 2) the successes and facilitators of their success in utilizing their grant funding; and 3) suggestions and opportunities for improvement at the health department level and for CDC to consider during subsequent funding cycles.
Progress reports provided to CDC’s Immunization Services Division indicate that awardees have had significant challenges to utilizing their immunization-related funding. However, it is not clear why that is. Preventing vaccine-related diseases is an important public health problem that the literature has indicated needs continued support. Since health departments play a critical role in the vaccine delivery system, it is important to understand why they cannot utilize their current funding and if there are opportunities to improve the funding expenditure process in the future so that the immunization-related public health infrastructure can be improved.
The report resulting from this assessment will be for internal CDC use and aims to describe each health department’s specific experience with their vaccine-related funding. It will also discuss patterns and themes common across all awardees or by awardee type to identify shared experiences, successes, challenges, internal or external factors associated with these challenges, factors that have facilitated success, and lessons learned and strategic directions for the Immunization Services Division to consider moving forward to support awardees in the future. Information from this assessment will inform the processes around future immunization funding and technical assistance provided to health departments.
Privacy Impact Assessment
No sensitive information is being collected through this data collection. All respondents will remain anonymous in the report and will not be identified to CDC. A question in the data collection tool asks the respondent for his/her health department, but the responses will not be linked to the specific individual in the analysis.
Data from the web-based instrument and telephone interviews will be stored on Windows 2008 servers that are kept in a secure physical location, are behind a firewall, require authentication to access, are patched and updated regularly and are protected by Antivirus and Antimalware software. A firewall is in place and prevents unauthorized access to the internal network. Devices used to access the data are always patched and kept updated and protected with Antivirus software.
Use of Improved Information Technology and Burden Reduction
Web-based instrument: Data will be collected via a web-based instrument using Qualtrics® allowing respondents to complete and submit their responses electronically. This method was chosen to reduce the overall burden on respondents by enabling them to easily access the data collection tool and complete it at a convenient time and location. The instrument was designed to collect the minimum information necessary for the purposes of this project (i.e., limited to a maximum of 36 assessment questions). Additionally, automatic skip patterns will be used for the web-based instrument, so if a respondent’s health department did not receive a specific immunization award, the instrument will automatically skip to the next section and will not present questions for the awards that the respondent did not receive. The web-based data collection tool also uses easy-to-read response scales or text boxes that are embedded in the online data collection tool.
Interview guide: The interviews will be conducted with a sample of respondents who completed the web-based instrument to complement the limitations of close-ended questions on web-based tool and provide a more nuanced and comprehensive portrait of the health department’s experiences. To minimize burden, interviews will be scheduled at the respondents’ convenience within a two-week period. While questions are open-ended, they directly build off of the respondents’ answer on the web-based tool and ask for specific examples and illustrations of challenges, facilitators, and suggestions. The conversation is designed to be brief and focused. Qualitative data from the interviews will be imported in NVivo® qualitative data analysis software. The collected qualitative data will be coded and analyzed thematically, where data analysts will identify key themes that emerged across groups of interviews by segment or other characteristics. Frequency and intensity of discussions on a specific topic will be key indicators used for extracting main themes.
Efforts to Identify Duplication and Use of Similar Information
While awardee progress reports for this funding have identified which performance-related goals and objectives they have met or not met, they do not discuss in detail their challenges related to utilizing the funds, their successes and facilitators of success for utilizing funding, and suggestions for CDC to improve the process for future immunization-related funding efforts.
No other efforts in the Immunization Services Division—or the larger field—have captured this information since it is specific to the 64 awardees that have been funded through these initiatives. This information is important to inform the guidance and technical assistance CDC provides to health departments in relation to their immunization funding. This information will be for internal CDC use only, shared in a report developed by the National Network for Public Health Institutes (NNPHI) for CDC’s Immunization Services Division, and will serve as a critical document to inform CDC’s continuous quality improvement process.
Impact on Small Businesses or Other Small Entities
No small businesses will be involved in this data collection.
Consequences of Collecting the Information Less Frequently
The purpose of this request is to ensure collection of data that is not otherwise available in current, time sensitive, or relevant formats to specific priorities of HHS and CDC. Specifically, without this data, there would be:
No timely feedback regarding the challenges to utilizing federal immunization funding among health department awardees,
Available funding at the state and local level not being used for important immunization activities and interventions, due to potentially preventable or surmountable challenges
Less effective public health infrastructure and capacities to address future vaccine-preventable outbreaks.
This request is for a one time data collection. There are no legal obstacles to reduce the burden.
Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
There are no special circumstances with this information collection package. This request fully complies with the regulation 5 CFR 1320.5 and will be voluntary.
Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency
This data collection is being conducted using the Generic Information Collection mechanism of the OSTLTS Survey Center (OSC) – OMB No. 0920-0879. A 60-day Federal Register Notice was published in the Federal Register on October 31, 2013, Vol. 78, No. 211; pp. 653 25-26. No comments were received.
CDC partners with professional STLT organizations, such as the Association of State and Territorial Health Officials (ASTHO), the National Association of County and City Health Officials (NACCHO), and the National Association of Local Boards of Health (NALBOH) along with the National Center for Health Statistics (NCHS) to ensure that the collection requests under individual ICs are not in conflict with collections they have or will have in the field within the same timeframe.
Explanation of Any Payment or Gift to Respondents
CDC will not provide payments or gifts to respondents.
Assurance of Confidentiality Provided to Respondents
The Privacy Act does not apply to this data collection. Employees of state and local public health agencies will be speaking from their official roles and will not be asked, nor will they provide individually identifiable information. This data collection is not research involving human subjects.
Justification for Sensitive Questions
No information will be collected that are of personal or sensitive nature.
Estimates of Annualized Burden Hours and Costs
Web-based instrument: The estimate for burden hours is based on a pilot test of the web-based instrument by six (6) public health professionals with backgrounds and experiences similar to the target respondents. In the pilot test, the average time to complete the web-based instrument including time for reviewing instructions, gathering needed information and completing the questions was approximately 15-20 minutes. The upper limit of this range (20 minutes) was used for the purposes of our estimated burden hours.
Interview guide: The estimate for burden hours is based on a pilot test of the interview guide by two (2) public health professionals with similar backgrounds and experience to the target respondents. In the pilot test, the average time to complete the interview including time for introductions, asking core questions, and asking probes was approximately 14-16 minutes. The upper limit of this range (16 minutes) was used for the purposes of our estimated burden hours.
Estimates for the average hourly wage for respondents are based on the Department of Labor (DOL) National Compensation Survey estimate for management occupations – medical and health services managers in state government (http://www.bls.gov/ncs/ocs/sp/nctb1349.pdf). Based on DOL data, an average hourly wage of $57.11 is estimated for all respondents.
The web-based instrument will be sent to the immunization program manager at each of the 64 health departments (n=64). The interviews will be conducted with 24 of the immunization program managers who completed the web-based instrument.
Table A-12 shows estimated burden and cost information for the total 88 respondents.
Table A-12: Estimated Annualized Burden Hours and Costs to Respondents – Immunization Awardee Assessment
Type of Respondent |
No. of Respondents |
No. of Responses per Respondent |
Average Burden per Response (in hours) |
Total Burden Hours |
Hourly Wage Rate |
Total Respondent Costs |
Web-based instrument: Immunization program managers from awarded health department |
64 |
1 |
20/60 |
21 |
$57.11 |
$1,199.31 |
Interviews: Immunization program managers from awarded health department (following completion of web-based instrument) |
24 |
1 |
16/60 |
6 |
$57.11 |
$342.66 |
TOTALS |
88 |
1 |
|
27 |
|
$1,541.97 |
Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers
There will be no direct costs to the respondents other than their time to participate in the data collection.
Annualized Cost to the Government
There are no equipment or overhead costs. The only cost to the federal government would be the salary of CDC contractors and sub-contractors supporting the data collection activities and associated tasks.
The data collection tool will be prepared by contractors and sub-contractors to CDC. A senior level sub-contractor will oversee data collection and manage all aspects of the process including data collection for both the web-based instrument and telephone interviews. The estimated cost to the federal government for the entire data collection period is $4,950. Table A-14 describes how this cost estimate was calculated.
Table A-14: Estimated Annualized Cost to the Federal Government
Staff or Contractor |
Average Hours per Collection |
Average Hourly Rate |
Average Cost |
Web-based Instrument |
|
|
|
Director of Research and Assessment (Sub-contractor to CDC) Manage data collection for web-based instrument, oversee process for quality control |
20 |
$60 |
$1,200 |
Senior Research Associate (Sub-Contractor to CDC) Web-based instrument programming, data collection, response tracking, quality control |
20 |
$35 |
$700 |
Assessment Coordinator (Contractor to CDC) Consultation on data collection for web-based instrument and quality control |
10 |
$35 |
$350 |
Telephone interviews |
|
|
|
Director of Research and Assessment (Sub-contractor to CDC) Manage data collection for telephone interviews, oversee process for quality control |
10 |
$60 |
$600 |
Senior Research Associate (Sub-Contractor to CDC) Interview administration and response tracking, quality control |
50 |
$35 |
$1,750 |
Assessment Coordinator (Contractor to CDC) Consultation on data collection interviews and quality control |
10 |
$35 |
$350 |
Estimated Total Cost of Information Collection |
$4,950 |
||
Explanation for Program Changes or Adjustments
This is a new data collection.
Plans for Tabulation and Publication and Project Time Schedule
Project Time Schedule
Upon completion of the data collection for the web-based instrument, data analysis will be conducted within two weeks. After examining responses to the web-based instrument, a sample of 24 respondents will be asked to participate in the next round of data collection, a telephone interview, to further delve in-depth into the specific challenges and facilitators related to utilizing federal immunization funding.
Task |
Timeline |
|
COMPLETE |
|
COMPLETE |
|
COMPLETE |
|
COMPLETE |
|
COMPLETE |
|
COMPLETE |
|
COMPLETE |
|
TBD |
|
(data collection open 2 weeks) |
|
4 weeks |
|
(data collection open 2 weeks) |
|
3 weeks |
|
5 weeks |
|
2 weeks |
Analysis Plan
The data collection from the web-based instrument and telephone interviews aim to answer the following questions:
What are the challenges that health departments have encountered in utilizing their federal immunization funding and have these differed across funding streams?
What are the factors that have facilitated health departments utilization of their federal immunization funding and have these differed across funding streams?
What suggestions and opportunities are there in the future via technical assistance, resources, or changes in processes to improve federal immunization funding utilization in the subsequent funding cycles?
The web-based instrument will provide data across all awarded health departments to understand the breadth of issues encountered by health departments, while the interviews will delve in-depth to gather specific examples of the experiences on the ground and how recommended changes could be implemented in the future.
For the web-based instrument, data will be exported from the Qualtrics® web-based instrument into SPSS, version 20. Data analysis will include descriptive statistics and where appropriate, bivariate statistics to compare health departments by certain characteristics and funding stream. Microsoft Excel will be used to create charts and graphics depicting the responses received. Open-ended questions on the instrument will be converted to text responses and analyzed thematically.
For the interviews, qualitative data from the interviews will be imported into NVivo® qualitative data analysis software. The collected qualitative data will be coded and analyzed thematically, where data analysts will identify key themes that emerged across groups of interviews by segment or other characteristics. Frequency and intensity of discussions on a specific topic will be key indicators used for extracting main themes.
Data from the web-based instrument and interviews will be analyzed and integrated to provide a description of the experience of each health department as well as the themes and patterns that emerged across health departments (or groups of health departments with similar characteristics) to identify challenges, facilitators of success, and opportunities for improvement.
The results of this data collection will be used for quality improvement purposes, to help guide CDC’s NCIRD enhancements, and to improve the communication, technical assistance, and other resources provided to state, local, and territorial health departments receiving federal immunization funding in the future. The goal is to understand the experiences and challenges at the health department level so that future immunization funding can be utilized in a timely manner to improve the public health infrastructure around immunizations.
Reason(s) Display of OMB Expiration Date is Inappropriate
We are requesting no exemption.
Exceptions to Certification for Paperwork Reduction Act Submissions
There are no exceptions to the certification. These activities comply with the requirements in 5 CFR 1320.9.
LIST OF ATTACHMENTS – Section A
Attachment A - List of Health Department Awardees
Attachment B - Web Version of Web-based Instrument
Attachment C - MS Word Version of Web-based Instrument
Attachment D - Interview Guide
Reference List
1 Centers for Disease Control and Prevention. Ten Great Public Health Achievements --- United States, 2001—2010. Morbidity and Mortality Weekly, 2011;60(19);619-623.
2 Research!America. Investment in Research Saves Lives and Money. Facts about Vaccines, Series #27, https://www.researchamerica.org/uploads/factsheet27vaccines.pdf, Accessed online on February 3, 2014.
3 National Vaccine Advisory Committee. White Paper on the United State Vaccine Safety System. September 2011.
5 National Research Council. Priorities for the National Vaccine Plan. Washington DC: The National Academies Press, 2010.
6 National Vaccine Advisory Committee. White Paper on the United State Vaccine Safety System. September 2011.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | gel2 |
| File Modified | 0000-00-00 |
| File Created | 2021-01-28 |