Assessments of the Quality and Utility of National Public Health Improvement Initiative to Support States, Tribes, Locals and Territories
OSTLTS Generic Information Collection Request
OMB No. 0920-0879
Supporting Statement – Section B
Submitted: March 4, 2013
Program Official/Project Officer
Anita McLees
Health Scientist
Applied Systems Research and Evaluation Branch
Division of Public Health Performance Improvement
Office of State, Tribal, Local and Territorial Support
Centers for Disease Control
1825 Century Center MS E-70
Tel: 404-498-0316
Email: [email protected]
Sonal R. Doshi
Health Scientist
Applied Systems Research and Evaluation Branch
Division of Public Health Performance Improvement
Office of State, Tribal, Local and Territorial Support
Centers for Disease Control
1825 Century Center MS E-70
Tel: 404-498-0322
Email: [email protected]
Section B – Data Collection Procedures
Respondent Universe and Sampling Methods
Focus groups- The respondent universe consists of all seventy-three states, tribes, locals and territories (STLTs) awarded funding in Year Three of the NPHII. The universe includes forty-nine state health departments representing forty-eight states and the District of Columbia, nine local health departments, eight organizations representing the U.S. Territories and Pacific Islands, and seven organizations representing American Indian/Alaska Native Tribes/Organizations (Attachment B –Listing of FY 2012 NPHII Awardees). All awardees from territories and tribes will participate in focus groups. All awardees from local health departments who received only Component I funding will participate in focus groups. A sample of state health departments combined with the 3 locals that received Component II funding (N=52) will be selected based on1) funding level and 2) the maturity of their quality improvement/performance management system, completion of accreditation pre-requisites, and tenure of the Performance Improvement Management (PIM).
TA satisfaction- The respondent universe for this data collection instrument includes forty-nine state health departments representing forty-eight states and the District of Columbia, nine local health departments, eight organizations representing the U.S. Territories and Pacific Islands, and seven organizations representing American Indian/Alaska Native Tribes/Organizations. South East Alaska Regional Health Consortium decided not to apply for FY 2012 NPHII funding, and will not be included in the respondent population for the Year 3 data collection (Attachment B –Listing of FY 2012 NPHII Awardees). Due to the desire to understand satisfaction with TA, recognizing that there is significant variability in the request process and the delivery of TA, all recipients of technical assistance whose requests are closed post April 1, 2013 through March 31, 2014 will be asked to complete the survey.
It is estimated that there will be an average of 10 TA requests closed per month for this data collection instrument; resulting in a total of 120 TA request closures over the year. It is expected that a number of the 73 potential respondents will be asked to complete this data collection instrument more than one time. Many of these 73 respondents will be expected to engage in a variety of TA activities with CDC and their partners over the year and those different TA activities will require individual/distinct data collection activities using this data collection tool.
Table B-1: Potential Respondent Universe
Entity |
Potential Respondent |
N |
NPHII Focus group STLTS awarded funds from CDC’s National Public Health Improvement Initiative |
Performance Improvement Managers or Principal Investigators in STLT organizations |
56 |
NPHII TA Satisfaction STLTS awarded funds from CDC’s National Public Health Improvement Initiative |
Performance Improvement Managers or Principal Investigators in STLT organizations |
120 |
Total Universe of Potential Respondents |
176 |
|
Procedures for Collection of Information
Focus Groups- Potential participants will be invited to participate in the focus groups (Attachment J- Invitation to NPHII Focus Group (grantee); Attachment K- Invitation to NPHII Focus Group (offsite)), and those who agree to participate in the focus group will be sent a confirmation email detailing date, time and the location of the focus group (Attachment L- NPHII Focus Group Confirmation Email (grantee); (Attachment M- NPHII Focus Group Confirmation Email (offsite)). Out of the entire population universe (N=73), 56 STLTs will be sampled. Obtaining information from this sample of the respondent universe will allow NPHII evaluators and ultimately NPHII program staffs to better understand how PIMs representing key sub-groups of awardees perceive the utility and value of elements of the NPHII program.
Data will be collected via focus groups that will last no more than two hours. The focus groups will be facilitated by two members of the NNPHI evaluation team, one serving as the lead facilitator, the other as a support person. The focus group methodology was chosen over a web-based survey because it allows for more in-depth questions about the usefulness of various elements of the NPHII program. Additionally, the focus group environment will allow for more discussion among participants to help identify potential barriers and successes to achieving NPHII goals. The findings from the NPHII focus groups will be used to generate recommendations for NPHII program improvements. The focus group data collection tool was designed to collect the minimum information necessary by a facilitator for the purposes of this project (i.e., limited to 10 questions). To reduce travel burden, two focus groups for tribal and territorial NPHII grantees will be conducted during the NPHII grantee meeting, and the remaining focus groups with a sample of local and state grantees will occur in regional settings. Once the information has been collected, it will be analyzed using qualitative data software such as NVivo. The software will facilitate analysis by organizing, classifying, and sorting data into frequencies. Qualitative thematic analyses will be performed on open-ended questions to compile recommendations for improving the NPHII program and technical assistance.
TA satisfaction- The NPHII TA Satisfaction data collection will be conducted using an online data collection tool. STLTs will be instructed to have their Performance Improvement Manager respond to the assessment if that individual has been hired. If a PIM had not been hired or appointed at the time of data collection, the STLT principal investigator will complete the assessment. The data collection tool, which was developed by CDC, contains questions addressing the following areas: description of the TAand satisfaction with the TA provided.
A personalized email notification, which will include a link to the online assessment tool, will be sent to all potential respondents following the closure of their TA request (Attachment N –TA Tool Notification). The data collection tool will be open for 14 business days to allow ample time for respondents to complete the tool. A reminder email (Attachment O – TA Satisfaction Reminder) will be sent out on Day 7 for non-respondents.
Data will be collected and stored in survey software maintained by CDC and protected under data privacy policies. CDC will conduct analysis of the data using SPSS. Data will be stored at CDC on a secured shared drive with access limited to NPHII evaluation staff and select NPHII leadership staff at CDC.
Methods to Maximize Response Rates & Deal with Nonresponse
Focus group- Advance email notification will be sent to all potential participants. Confirmation emails will be used to reduce no-show rates at the focus groups (Attachments L and M). Lastly, to reduce travel burden, focus groups will occur at convenient locations including the NPHII grantee meeting and regional settings.
TA satisfaction- Email notification and reminder email (Attachment O). The purpose of this data collection tool is to capture the NPHII awardees’ perspectives on the TA received; higher response rates will yield more reliable information, however, no scientific inferences will be made.
Test of Procedures or Methods to be Undertaken
Focus group- To ensure that items and responses are understandable by respondents and provided the information needed, eight public health professionals, including 5 CDC NPHII project officers and 3 other CDC public health professionals, participated in a pilot test of the focus group guide and process. Following the focus group pilot test, participants were asked to reflect on the facilitation and experience of the focus group, and were also asked to provide specific feedback on the clarity of the questions and suggestions for improving item and response wording. The length of the focus group protocol was reduced and questions wording was clarified based on pilot testing results and expert review. In the pilot test for the focus group, the average time to complete the focus group data collection including time for reviewing instructions, gathering needed information and completing the focus group, was approximately 120 minutes. The introduction and introductory question took 25 minutes to complete, questions about technical assistance took 30 minutes, questions about the PIMs role took 40 minutes, and closing took 25 minutes. For the purposes of estimating burden hours, the upper limit of this range (i.e. 120 minutes) is used. For the purposes of estimating burden hours, 120 minutes is used.
TA satisfaction- To ensure that items and responses are understandable by respondents; six public health professionals were asked to review the data collection tool and provide specific feedback on how to improve item and response wording. The length of the data collection tool was also reduced based on pilot testing results and expert review. The estimate for burden hours is based on the pilot test of the data collection tool by the six public health professionals. The average time to complete the data collection tool including time for reviewing instructions, gathering needed information and completing the data collection tool, was approximately 12 minutes. Based on these results, the estimated time range for actual respondents to complete the data collection tool is 10-15 minutes. For the purposes of estimating burden hours, the upper limit of this range (i.e., 15 minutes) is used.
Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
Focus Groups
Anita McLees
Health Scientist
Applied Systems Research and Evaluation Branch
Division of Public Health Performance Improvement
Office of State, Tribal, Local and Territorial Support
Centers for Disease Control
1825 Century Center MS E-70
Tel: 404-498-0316
Email: [email protected]
Nikki Lawhorn
Research Manager
National Network of Public Health Institutes
1515 Poydras St, Suite 1490
New Orleans, LA
Tel: 251-928-8534
Email: [email protected]
TA Satisfaction
Sonal Doshi
Health Scientist
Applied Systems Research and Evaluation Branch
Division of Public Health Performance Improvement
Office of State, Tribal, Local and Territorial Support
Centers for Disease Control
1825 Century Center MS E-70
Tel: 404-498-0322
Email: [email protected]
Saira Nawaz
CDC Evaluation Fellow
Applied Systems Research and Evaluation Branch
Division of Public Health Performance Improvement
Office of State, Tribal, Local and Territorial Support
Centers for Disease Control
1825 Century Center MS E-70
Phone: 404-498-0338
Email: [email protected]
LIST OF ATTACHMENTS – Section B
Note: Attachments are included as separate files as instructed.
J. Invitation to NPHII Focus Group (grantee)
K. Invitation to NPHII Focus Group (offsite)
L. NPHII Focus Group Confirmation Email (grantee)
M. NPHII Focus Group Confirmation Email (offsite)
N. TA satisfaction data collection tool Instructions and Link Email
O. TA Satisfaction Reminder Email
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | gel2 |
| File Modified | 0000-00-00 |
| File Created | 2021-01-31 |