CDC Prevention Status Reports: User Satisfaction and Impact
OSTLTS Generic Information Collection Request
OMB No. 0920-0879
Supporting Statement – Section B
Submitted: 12/10/14
Program Official/Project Officer
Garry Lowry, MPH
Health Scientist
Centers for Disease Control and Prevention, Office for State, Tribal, Local and Territorial Support, Division of Public Health Performance Improvement
4770 Buford Hwy, NE, MS E-70, Atlanta, GA 30024
Phone: 404-357-8769
Fax: 404-315-2369
Email: [email protected]
Section B – Data Collection Procedures
Respondent Universe and Sampling Methods
A mixed methods approach (online assessment and key informant interviews) will be used to examine users’ experiences with the PSRs, including successes, challenges, and suggestions for improvement approximately a year after the PSRs were released in January 2014.
Web-based Assessment
The online instrument will be used to gather quantitative information from state public health staff that were sent and have accessed the reports, to provide a portrait of users’ experiences with the PSRs. The individuals mentioned below are the universe of respondents for the web-based assessments. No sampling procedures are required as everyone in the respondent universe will be asked to participate in the assessment. The final respondent population consists of 612 state health department officials and staff totaling 612 respondents.
The respondent universe consists of the state health official (n=51) and the senior deputy health official (n=51) for all 50 states and the District of Columbia. The universe also includes one respondent (i.e., program director/manager or other key staff identified by CDC program) for each of the 10 PSR topics for each state and the District of Columbia (n=510). Each of these categories of public health staff are included as target respondents because each is an important user group for the CDC Prevention Status Reports (PSRs) and have been emailed and received the PSRs.
The PSRs were disseminated via email messages that included a link to the CDC website where the reports were posted. Individual email addresses for the respondent universe were obtained from the original email distribution lists for the PSRs maintained by CDC programs.
Key Informant Interviews
Participants of the telephone interview will be a sample of respondents who completed the web assessment. The goal of the interviews is to gather more in-depth information from users regarding their experiences using the PSRs. The interviews will provide greater context to understand perceptions and usefulness of the PSRs, including perceived successes, challenges, and suggestions for improvement.
A total of 20 interviews will be conducted, with 10 interviewees randomly selected in each of two segments based on their responses to the web assessment. Web assessment responses will be used to identify all respondents who indicate they have used the PSRs (i.e., selected at least one way they have used the PSRs in response to question 7). Interviewees will be stratified by the following two segments:
State health department executive leader or manager who reported using the PSRs
State health department division, unit, or program leader or manager who reported using the PSRs
Data collection will occur in two phases. Phase 1 will include a web-based assessment sent to 612 respondents, and Phase 2 will build off of the earlier phase to explore responses to the web-based assessment more in-depth by conducting telephone interviews with 20 participants.
Web-based Assessment
In Phase 1, data will be collected through a one-time web-based assessment and respondents will be recruited through a notification email to the 612 STLT eligible respondents (see Attachment H—Web Assessment Notification Email). The notification email will explain:
The purpose of the assessment, and why their participation is important
Method to safeguard their responses
That participation is voluntary
The expected time to complete the assessment
Contact information for the assessment team
The possibility for recruitment for a follow-up interview, depending on their indicated use of the PSRs
The email will also state instructions for participating and a link to the online assessment and consent form. The Qualtrics online remote data collection tool will be used to develop the assessment instrument and gather the data. This will reduce the burden of subscribers by allowing them to take the assessment online at their own convenience and by allowing them to skip irrelevant questions. The assessment was designed to collect the minimum information necessary for the purposes of this project.
Respondents will be asked for their response to the instrument within a 2-week period (10 business days) to allow ample time for respondents to complete it. Respondents may complete the assessment in multiple sessions, if necessary. Reminders will be sent on the last week to non-respondents to urge them to complete the assessment (see Attachment I— Web Assessment Reminder Email).
Key Informant Interviews
The 20 interview participants will be a subset selected from the online assessment respondents who indicated use of the PSRs, as described in the section above.
Upon completion of the data collection for the web-based instrument, preliminary data analysis will be conducted within two weeks. After examining responses to the web-based instrument, a sample of 20 respondents will be asked to participate in the next round of data collection, a telephone interview. Interviewees will be contacted via email thanking them for completing the web-based instrument and asking them participate in the brief telephone interview (see Attachment H – Interview Introductory Email).
Respondents will be asked to participate in an interview within a 2-week period (10 business days (see Attachment J—Interview Notification Email). If no response is received within 5 business days, a reminder email will be sent out to non-responders (see Attachment K—Interview Follow-up Email).
Data from the web-based instrument will be downloaded, cleaned, and analyzed in SPSS Version 20. Frequencies and bivariate analyses will be conducted for close-ended questions when looking at responses across all respondents or by respondent characteristics. Open-ended questions on the instrument will be converted to text responses.
For the interviews, qualitative data from the interviews will be imported in NVivo® qualitative data analysis software. The collected qualitative data will be coded and analyzed thematically, where data analysts will identify key themes that emerged across groups of interviews by segment or other characteristics. Frequency and intensity of discussions on a specific topic will be key indicators used for extracting main themes.
Methods to Maximize Response Rates Deal with Nonresponse
Although participation in the assessment is voluntary, the project lead will make every effort to maximize the rate of response. The assessment tool was designed with particular focus on streamlining questions to allow for skipping questions based on responses to previous questions, thereby minimizing response burden. A reminder email will be sent to those who have not completed the assessment during the last week of the assessment period (see Attachment L—Web Assessment Reminder Email). To ensure participation in the 20 follow-up interviews, participants will only be selected if they indicated they used the PSRs, which might indicate their vested interest in the improvement of the product.
Test of Procedures or Methods to be Undertaken
The web-based version of the assessment was pilot tested by 3 CDC public health professionals. Feedback from this group was used to refine questions as needed, ensure accurate programming and skip patterns and establish the estimated time required to complete the assessment. In the pilot test, the average time to complete the assessment, including time for reviewing instructions, gathering needed information and answering the questions, was approximately 11 minutes. Based on these results, the estimated time range for actual respondents to complete the assessment is 10 to 15 minutes. For the purposes of estimating burden hours, the upper limit of this range (i.e., 15 minutes) is used.
The interview guide was pilot tested by 3 CDC public health professionals. Feedback from this group was used to refine questions as needed and establish the estimated time required to complete the interviews. The estimated time range for actual respondents to complete the interview is 15 to 20 minutes. For the purposes of estimating burden hours, the upper limit of this range (i.e., 20 minutes) is used.
Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
CDC staff will provide oversight for development of the assessment instrument and analysis plan, data collection, documentation of results, and development of the OMB application package. Contractors are being used to support development of the assessment tool, data collection, and data analysis.
Garry Lowry, MPH
Health Scientist
Applied Systems Research and Evaluation Branch
Division of Public Health Performance Improvement
Office for State, Tribal, Local and Territorial Support
4770 Buford Hwy, NE, MS E-70, Atlanta, GA 30024
Phone: 404-357-8769
Fax: 404-315-2369
Email: [email protected]
Angeline Boey, MPH
Health Communication Specialist
Applied Systems Research and Evaluation Branch
Division of Public Health Performance Improvement
Office for State, Tribal, Local and Territorial Support
4770 Buford Hwy, NE, MS E-70, Atlanta, GA 30024
Phone: 404-498-6469
Email: [email protected]
Brittany Bickford, MPH
Research & Evaluation Analyst
National Network of Public Health Institutes
1515 Poydras St., Suite 1490
New Orleans, LA 70112
Phone: 504-872-0755
Email: [email protected]
Valerie L. Polletta, MSc
Senior Research Associate
Health Resources in Action, Inc.
Research and Evaluation
95 Berkeley Street
Boston, MA 02116
Phone: 617-279-2240
E-mail: [email protected]
LIST OF ATTACHMENTS – Section B
Note: Attachments are included as separate files as instructed.
Web Assessment Notification Email
Web Assessment Reminder Email
Interview Notification Email
Interview Follow-Up Email
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | gel2 |
| File Modified | 0000-00-00 |
| File Created | 2021-01-28 |