Assessment of State, Tribal, Local and Territorial Grantees’ Technical Assistance Needs and Support from CDC Project Officers
OSTLTS Generic Information Collection Request
OMB No. 0920-0879
Supporting Statement – Section A
Submitted:
Program Official/Project Officer
Karen Mumford, PhD
Health Scientist
Center for Disease Control and Prevention, Office for State, Tribal, Local and Territorial Support, Division of Public Health Performance Improvement
1825 Century Center, Atlanta, GA 30345
Phone: 404.498.0365 Fax: 404.498-0340
Email: [email protected]
Section A – Justification
Circumstances Making the Collection of Information Necessary
Background
This data collection is being conducted using the Generic Information Collection mechanism of the OSTLTS Survey Center (OSC) – OMB No. 0920-0879. The respondent universe for this data collection aligns with that of the OSC. Data will be collected from state, tribal, local, and territorial level (STLT) program mangers/directors acting in their official capacities.
Over the past 4 years, a multitude activities and recommendations (see Attachment A) have led to the development and work of the Technical Assistance and Service Improvement Initiative (TASII) and ultimately, the development of the TASII External Stakeholder Survey. Beginning in 2008, the Association of State and Territorial Health Officials prepared and delivered a letter (see Attachment B) to the Director of CDC, Dr. Frieden. In this letter, this national organization requested “Greater stability, reliability and engagement of the CDC Project Officers”. As a result of this letter, the Partner Services Workgroup was established. In 2009, this workgroup convened a diverse group of public health officials to identify core themes and recommendations aimed at improving the provision of technical assistance provided by project officers. Specifically, consensus was reached for the following 5 recommendations (see Attachment C):
Identify and define standardized training for project officers,
Develop a recruitment strategy including consistent criteria for the level of experience, training or skills for project officer positions,
Develop and implement universal expectations and performance indicators or standards of performance for project officers,
Establish an evaluation framework including feedback from external partners assessing the services provided by project officers, and
Provision of senior leadership guidance and oversight to ensure consistency, continuity, and accountability.
Likewise, the State, Tribal, Local and Territorial (STLT) workgroup of the Advisory Committee to the Director underscored the need for external evaluation and assessment of the technical assistance provided by CDC to external stakeholders. In a 2011 meeting, STLT partners emphasized that project officers played a critical role in establishing and strengthening partnerships between external stakeholders and CDC, but lack of consistency, continuity, and agency-wide clarity undermined utility of project officers in stewarding cooperative agreements. The workgroup provided recommendations for areas of improvement including identifying universal expectations of project officers, establishing performance indicators, standardizing training, and evaluating and incorporating feedback from external partners to improve the technical assistance process. A full report of the recommendations was disseminated widely among CDC leadership and program staff (see Attachment D).
In addition, the Office for State, Territorial, Local, and Tribal Support (OSTLTS) completed the 2010 Transformation Project (see Attachment E). This activity not only concurred with previous workgroup recommendations to complete an evaluation of external stakeholders’ perceptions and satisfaction with the technical assistance provided by project officers, but specified the following topics for inclusion in the assessment: (1) the impact of project officer turn-over on program success, (2) alignment of project officers priorities with SLTLT priorities for program implementation, and (3) clarity and consistency of communications.
In response to the culmination of recommendations, at the directive of the Director of CDC, the Technical Assistance and Service Improvement Initiative (TASII) emerged aimed at understanding the needs and expectations of external stakeholders and improving the accuracy, consistency, and timeliness of the technical assistance provided by project officers. To meet this goal, TASII is comprised of several work streams:
Project Officer Portal – a website for project officers to easily access CDC resources that are useful for their daily functions.
Learning Opportunities for Project Officers - learning events tailored to project officers, covering topics such as public health law, accreditation, and budget and grants management.
Project Officer Registry – a database of project officers, program consultants and COTRs across CDC divisions and branches.
All Hands Meetings for Project Officers – conduct CDC-wide events for project officers and their managers to highlight CIO strategies developed for project officer training, performance management, and quality improvement.
Project Officer Competencies – identification of core project officer competencies, an effort led by the TASII workgroup and CDC University.
TASII Assessment – three separate data collection tools for project officers, CDC leadership and supervisors, and external stakeholders, to assess the type and quality of technical assistance and program support provided by CDC.
One method being utilized in the TASII assessment is the implementation of surveys aimed at collecting information regarding indices of the quality of the technical assistance and customer support provided by CDC project officers to CDC-funded grantees. The comprehensive approach of the TASII Assessment elicits perceptions of CDC’s technical assistance from both internal and external stakeholders. Information collected from internal stakeholders includes:
TASII Survey for Project Officers– providing insight regarding service delivery models, organizational supports and resources provided, and professional development opportunities.
TASII Survey for Supervisors– collecting information from the managers/supervisors of project officers regarding standardized processes and protocols, expectations and standards of practice, and workforce recruitment, retention and development.
The third component of the assessment is the External Stakeholder Survey. The External Stakeholder data collection tool will be administered to STLT grantees who are recipients of Cooperative Agreements with a focus on programmatic and prevention activities in FY 2011. The tool is tasked with (1) identifying STLT needs and expectations for capacity building and technical assistance, with a particular focus on the characteristics of project officers and organizational performance, and (2) assessing STLT satisfaction with current capacity building and technical assistance provided by CDC project officers. The specific content and structure for this survey was identified and defined through a series of focus group interviews conducted with 35 project officers and 26 supervisors across 7 CIOs during 2011 (see Attachment F). Specifically, these focus groups identified a broad scope of concepts and constructs important for inclusion in the survey including items measuring aspects of technical assistance provided through all phases of a funded program, service delivery models, methods for the provision of technical assistance, expectations and support provided for continuous improvement, skills and characteristics necessary to provide quality technical assistance and capacity building support to grantees. Findings from the data collection will inform ongoing cross-agency (as part of the Technical Assistance & Service Improvement Initiative) and program-specific efforts to ensure consistent, high-quality project officer performance to meet programmatic requirements as well as STLT expectations and technical assistance needs.
Privacy Impact Assessment
Overview of the Data Collection System – The data collection system consists of a web-based survey instrument (see Attachment G & H) designed to elicit information from program managers/directors regarding their perceptions of the technical assistance and support provided by CDC project officers. The web-based tool will be distributed using Qualtrics software by emailing respondents a link to the survey instrument. This email will contain instructions for completing the survey online. Desk reviews were conducted with eight representatives from CDC’s partner organizations including Association of State and Territorial Health Officials (ASTHO), National Association of County and City Health Officials (NACCHO), and Pacific Island Health Officers Association (PIHOA) as well as Tribal representation coordinated through CDC’s Tribal Liaison. The purpose of the desk review was to (1) ensure that the survey measures aspects of TA provided by CDC’s project officers that are relevant and important to external stakeholders, and (2) ensure that all items included in the survey contribute to an increased understanding of STLT expectations and informs priorities for future TASII activities. Feedback from this group was used to refine questions as needed and establish the estimated time required to complete the survey scheduled for implementation in July 2012.
Items of Information to be collected – The survey consists of 92 questions of various types including dichotomous, multiple response, interval, filter, and open-ended questions. The survey is organized into two parts:
Respondent Information
Level of interaction with CDC project officer
Number of years in current position
Technical Assistance Delivery and Organizational Support
Respondent characteristics –current job title and level of interaction with project officers
Funding Opportunity/Program Announcements – satisfaction with the clarity and consistency of the scope and expectations of program announcements
Technical Assistance –questions in this section cover the following technical assistance areas:
Description of the type of technical assistance delivery system
Selection of CDC staff that serves as the primary source of TA for each programmatic function
Perceptions of the frequency of site visits and conference calls
Perceptions of the accuracy, consistency and timeliness of the TA provided by CDC project officers
Perceptions of the impact of turn-over rate or reassignment of project officers
Satisfaction with the availability of tools and resources that supplement project officer TA
Progress reporting – satisfaction with the clarity of the progress reporting requirements, the level of burden in terms data collection and reporting, and the feedback received from project officers on progress reports.
Project Officer Characteristics & Competencies
Satisfaction with project officers’ interpersonal characteristics and traits
Perceptions of which knowledge, skills and abilities are required for project officers to provide high-quality technical assistance
Satisfaction with project officers’ knowledge, skills and abilities
Additional comments and recommendations for how CDC can improve the delivery of technical assistance
Identification of Website(s) and Website Content Directed at Children Under 13 Years of Age – The data collection system involves using a web-based survey. Respondents will be sent a link directing them to the online survey only (i.e., not a website). No website content will be directed at children.
Purpose and Use of the Information Collection
The information will be collected through a web-based questionnaire, and analyzed by an evaluation specialist in the Applied Systems Research and Evaluation Branch. This is a one-time data collection. Details of the purposes of this data collection are provided below:
Purpose 1: Identify Technical Assistance Needs and Expectations of STLT Awardees – This data collection effort represents the first time that STLT grantees will be asked for their perceptions about how CDC can improve their technical assistance delivery for successful implementation of Cooperative Agreements. STLTs will be asked to identify which knowledge, skills and abilities (KSAs) are most important for project officers to demonstrate in areas of budget and grants management, program planning and evaluation, and organization consultation. This information will be shared with the workgroup led by CDC University that is currently developing and validating project officer competencies.
Purpose 2: Assess STLT satisfaction with current Technical Assistance – In addition to the identification of technical assistance needs, this data collection tool will also assess STLT satisfaction with current capacity building and technical assistance provided by CDC project officers. STLTs will be asked about their satisfaction with the clarity of funding opportunity announcements, the feedback received from progress reports, the frequency of site visits and conference calls, and the overall quality of technical assistance. STLTs will also report their satisfaction with the KSAs and interpersonal characteristics demonstrated by project officers.
Information derived from the data collection tool will be used to inform the following TASII work streams:
Learning opportunities for project officers.
Competency and training development for project officers.
Development of high-quality resources and tools to supplement TA/support provided by project officers.
Improvements to CDC’s technical assistance delivery models.
Center/division efforts to improve TA.
Serve as a baseline for evaluating TASII’s efforts to improve TA for STLT awardees.
Privacy Impact Assessment
No sensitive information is being collected. No individually identifiable information is being collected. The proposed data collection will have little or no effect on respondent privacy. Respondents are participating in their official capacity as program managers/directors in state (or District) departments of health.
Considerations Given to Information Technology
Data will be collected via a web-based tool allowing respondents to complete and submit their responses electronically. The web-based tool will be developed using Qualtrics, an advanced platform for the creation of online surveys that allows for complex branching and skip logic and provides respondents with the flexibility to complete the tool in more than one session. Web surveys reduce respondent burden by enabling easy access and completion at a convenient time and location. The online survey will consist of either easy-to read response selections or embedded text boxes, and skip patterns will be programmed into the survey to direct respondents to appropriate questions. This software will also keep track of grantees information (branch, division and CIO) so that reminders can automatically be emailed on day 10 and day 15 of the survey. This method was chosen to reduce the overall burden on respondents. The survey was designed to collect the minimum information necessary for the purposes of this project.
Duplication of Information
TASII is a new agency-wide initiative to enhance CDC’s technical and capacity building support for STLT partners. This survey represents the first attempt to assess the expectations and satisfaction of STLT awardees receiving a range of technical assistance across CDC’s centers, divisions, and branches. There is no information available that can substitute this data collection.
Reducing the Burden on Small Entities
No small businesses will be involved in this data collection.
Consequences of Not Conducting Collection
This data collection is a direct response to concerns raised by CDC’s national partners regarding variability in the accuracy, consistency, and timeliness of the technical assistance provided by project officers across CDC programs. The survey allows us to evaluate the quality of TA along these dimensions for a range of activities and functions of CDC project officers.
The consequences of not collecting this information would be:
Failure to systematically obtain information about the quality of CDC’s technical assistance in advancing the implementation of Cooperative Agreements from STLT partners.
Failure to identify and develop training in core knowledge and skills that are essential for project officers to provide high-quality technical assistance.
Limited guidance on how to enhance the delivery of technical assistance at the agency-level and how to tackle program-specific gaps in service provided by project officers.
This request is for a one time data collection. There are no legal obstacles to reduce the burden.
Special Circumstances
There are no special circumstances with this information collection package. This request fully complies with the regulation 5 CFR 1320.5 and will be voluntary.
Consultation with Persons Outside the Agency
This data collection is being conducted using the Generic Information Collection mechanism of the OSTLTS Survey Center (OSC) – OMB No. 0920-0879. This data collection is being conducted using the Generic Information Collection mechanism of the OSTLTS Survey Center (OSC) – OMB No. 0920-0879. A 60-day Federal Register Notice was published in the Federal Register on October 22, 2010, Vol. 75, No. 204; pp. 65353-54. Two comments were received from the Association of State and Territorial Health Officials (ASTHO), and the National Association of County and City Health Officials (NACCHO).
CDC partners with professional STLT organizations, such as the Association of State and Territorial Health Officials (ASTHO), the National Association of County and City Health Officials (NACCHO), and the National Association of Local Boards of Health (NALBOH) along with the National Center for Health Statistics (NCHS) to ensure that the collection requests under individual ICs are not in conflict with collections they have or will have in the field within the same timeframe.
Payment or Gift
CDC will not provide payments or gifts to respondents.
Confidentiality
The Privacy Act does not apply to this data collection. Employees of state and local public health agencies will be speaking from their official roles and will not be asked, nor will they provide individually identifiable information.
This data collection is not research involving human subjects.
Sensitive Nature
No information will be collected that are of personal or sensitive nature.
Burden of Information Collection
The estimate for burden hours is based on a pilot test of the survey instrument by eight individuals affiliated with national partners including ASTHO, NACCHO, Tribal liaison, and PIHOA. In the pilot test, the average time to complete the survey including time for reviewing instructions, gathering needed information and completing the survey, was 35 minutes. Based on these results, the estimated time range for actual respondents to complete the survey is 25-45 (range) minutes. Reviewers who represent our target population felt the length of the data collection tool was reasonable given that the tool is required to collect information in several areas related to the stakeholders’ interactions with CDC including clarity of Funding Opportunity Announcements, utility and burden of progress reporting, overall impressions of the technical assistance on multiple quality dimensions, and access to resources and information (covered in Part A of data collection tool), as well as an and assessment of the technical assistance needs and satisfaction with ongoing technical assistance provided by project officers (covered in Part B of data collection tool).
Estimates for the average hourly wage for respondents are based on the Department of Labor (DOL) National Compensation Survey estimate for management occupations – medical and health services managers in state government (http://www.bls.gov/ncs/ocs/sp/nctb1349.pdf). Based on DOL data, an average hourly wage of $47.49 is estimated for all 2,541 respondents. Table A-12 shows estimated burden and cost information.
Table A-12: Estimated Annualized Burden Hours and Costs to Respondents – STLT Technical Needs Assessment
Type of Respondent |
No. of Respondents |
No. of Responses per Respondent |
Average Burden per Response (in hours) |
Total Burden Hours |
Hourly Wage Rate |
Total Respondent Costs |
Primary Investigator / Primary Point of Contact |
2,541 |
1 |
35/60 |
1,482 |
$ 47.49 |
$ 90,504.07 |
TOTALS |
2,541 |
1 |
|
1,482 |
|
Costs to Respondents
There will be no direct costs to the respondents other than their time to participate in each survey.
Cost to Federal Government
There are no equipment or overhead costs. Contractors are not being used to support this data collection. The only cost to the federal government would be the salary of CDC staff supporting the data collection activities and associated tasks.
Table A-14: Estimated Annualized Cost to the Federal Government
Staff (FTE) |
Average Hours per Collection |
Average Hourly Rate |
Average Cost |
Health Scientist (GS-14) Instrument development, pilot testing, OMB package preparation, data collection, data coding and entry, quality control, data analysis, and report preparation |
300 |
$51.64 |
$ 15,492.00 |
Deloitte Contractor Web-based survey programming, data collection |
150 |
$137.09 |
$20, 563.50 |
ORISE Fellow Assisting with instrument development, OMB package preparation, data collection, data coding and entry, quality control, data analysis, and report preparation |
175 |
$23.55 |
$ 4,121.25 |
Estimated Total Cost of Information Collection |
$40,176.75 |
||
Reason for Changes
This is a new data collection.
Tabulation of Results, Schedule, and Analysis Plan
The results of this assessment will be shared internally with CDC leadership and staff across the agency as well as externally with CDC’s national partners and grantees. Findings will be posted on CDC’s intranet in the following locations: (1) Project Officer Portal, (2) OSTLTS web page, and (3) TASII web page. In addition, aggregate findings will be posted to CDC’s internet including, but not limited to, CDC’s Gateway for STLTs. The results will be used internally to set priorities and inform activities for TASII and externally to communicate results with STLT partners.
Both quantitative and qualitative analyses will be performed. Quantitative analyses will involve using descriptive statistics to determine frequency distributions and corresponding variances for responses to each survey question. Responses will be cross-tabulated to compare responses between divisions, high-funded and low-funded programs, nationally funded versus discretionary programs, level of experience with project officers, and technical assistance service delivery models. Qualitative thematic analyses will be performed on open-ended questions to compile recommendations for improving CDC’s technical assistance and increasing utility of project officers in each programmatic area. Evaluation questions that are to be answered by the survey tool are provided (See Attachment I).
Project Time Schedule
Design survey questionnaire ……………………………………………… (COMPLETE)
Develop survey protocol, instructions, and analysis plan……… (COMPLETE)
Pilot test survey questionnaire…………………………………………… (COMPLETE)
Prepare OMB package………………………………………………………… (COMPLETE)
Submit OMB package…………………………………………………………… (COMPLETE)
OMB approval……………………………………………………………………… (TBD)
Conduct survey…………………………………………………………………… (Survey open 4 weeks)
Collect, code, enter, quality control, and analyze data…………… (2 weeks)
Prepare report……………………………………………………………………… (2 weeks)
Disseminate results/reports…………………………………………………… (Sept 2012)
Display of OMB Approval Date
We are requesting no exemption.
Exceptions to Certification for Paperwork Reduction Act Submissions
There are no exceptions to the certification. These activities comply with the requirements in 5 CFR 1320.9.
LIST OF ATTACHMENTS – Section A
Note: Attachments are included as separate files as instructed.
Sources of Recommendations to Implement an External Stakeholder Survey
2008 ASTHO Letter to CDC Director
Partner Services Workgroup Summary
STLTS Workgroup of the Advisory Committee to the Director (ACD) of CDC
OSTLTS Transformation Project Report
TASII Focus Group Report
TASII External Stakeholder Survey Instrument - Word Version
TASII External Stakeholder Survey Instrument – Web Screen Shots
Key Evaluation Questions and Survey Instrument Items
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | gel2 |
| File Modified | 0000-00-00 |
| File Created | 2021-01-31 |