Supporting Statement B

B. Collections of Information Employing Statistical Methods

The program evaluation of the CPHP Program is summative and retrospective in design. Further, the evaluation will focus on the aggregate outcomes of the CPHP Program and not on individual Centers. The overall sample of respondents includes key staff from all 27 CPHPs, representatives of organizations that may have utilized emergency preparedness training or technical assistance resources developed, delivered, or sponsored by the CPHP Program (i.e. customers), and a selection of key partners.

As described in Part A of the Supporting Statement, CDC will employ 3 related data collection instruments to gather information to complete the evaluation of the CPHP Program. These are: (1) CPHP Interview Instrument (Attachment C), (2) CPHP Customer/Partner Survey Instrument (Attachment D), and (3) CPHP Customer/Partner Follow-Up Interview Instrument (Attachment E). The respondent universe and sample for each instrument is described next.

1. Respondent Universe and Sampling Methods

The potential respondent universe for the CPHP Interview Instrument includes key staff from each of the 27 CPHPs across the country. It is our intent to sample all 27 CPHPs by interviewing up to 3 key staff together, such as the Principal Investigator, Center Director/Coordinator and/or the Center Evaluator of each Center (n = 81). The plan to survey the universe of CPHPs is justified by the small number of Centers, the low time burden of response, and the importance of complete CPHP representation because of the varied characteristics of the Centers (e.g. years of operation, areas of programmatic emphasis, and geographical location). It is anticipated that a participation rate of above 80% of CPHPs will be achieved.

The potential respondent universe for the second and third data collection instruments, the CPHP Customer/Partner Survey Instrument and Follow-Up Interview respectively, is a selection of 171 key partners and customers. Customers and partners may be representatives of organizations that have utilized emergency preparedness training or technical assistance resources developed, delivered, or sponsored by the CPHP Program, such as state and local health departments and a variety of other governmental and non-governmental organizations involved in public health emergency planning and response. Identical versions of the survey will be distributed to two groups from the following potential sample frame:

1. Customers and partners identified by the CPHPs. Each CPHP will identify up to 3 non state and major territory customers and/or partners they have worked with during the Cooperative Agreement (1 respondent from each CPHP-identified organization for a total of 81 respondents). All those identified will be contacted as respondents. Because the representatives of these organizations will be expected to have some familiarity with the CPHP Program, an 80% or higher survey response rate is anticipated. With the exception of state health department respondents, results from this level of analysis may not be generalizable to the universe of potential customers or partners of the CPHP Program. However, this information will still be very useful for informing the development of the new Cooperative Agreement and will also add insights on the accomplishments of the CPHP Program to date.

2. Representatives of organizations who may have utilized emergency preparedness training or technical assistance resources developed, delivered, or sponsored by the CPHP Program This sample will include Preparedness Coordinators within state and major territorial health departments, as identified by the CDC Public Health Emergency Preparedness (PHEP) Cooperative Agreement program (n = 60). The plan to survey state and major territorial health departments is justified by the small number of states and major territories, the low time burden of response, and the importance of complete representation of this group. Additionally, the PHEP Cooperative Agreement between CDC and states directs the states to work in conjunction with the CPHP Program to build workforce preparedness and response capabilities. The inclusion of each state and major territory in this evaluation seems appropriate and it is reasonable to anticipate a response rate around 90% in this group.

Thirty respondents in this group will include those from partner organizations such as the Association of State and Territorial Health Officials or the National Association of City and County Health Officials, American Indian and Alaska Native Tribal entities, and other non-profit groups that may have had contact with the CPHP Program. In order to gather accurate evaluation data, respondents that are, at a minimum, familiar with the program should be part of this sample frame. The universe of organizations in this group is very large and unknown, and within this universe, the data collection effort intends to focus on organizations with some knowledge of and experience working with the CPHP program. However, a complete set of these organizations is also unknown. The 27 CPHPs have worked with or provided training to thousands of such organizations over the period of the Cooperative Agreement. Therefore, it is not possible to develop a complete sample frame from which a random sample of 30 potential respondents that have used the services of the CPHP program can be drawn. It is important, nevertheless, for this class of organizations to be represented in the evaluation. Because the representatives of these organizations will be expected to have some familiarity with the CPHP program, we anticipate a reasonably high survey response rate of around 70%. We recognize that, with the exception of state health department Preparedness Coordinator respondents, results from this level of analysis may not be generalizable to the universe of potential customers or partners of the CPHP program. However, this information will still be very useful for informing the development of the new Cooperative Agreement and will also add insights on the accomplishments of the CPHP program to date.

The sample frame for the CPHP Customer/Partner Follow-Up Interview is a volunteer sample from the pool of respondents to the CPHP Customer/Partner Survey Instrument. Survey respondents will be provided the opportunity to reply under separate cover (so that responses are not linked to identifying information) with an indication of their willingness to participate in a follow-up semi-structured interview. The purpose of this data collection is to gather follow-up, additional qualitative data that explains the quantitative findings from the survey. Considering this intent, a representative sample of 20 respondents will be generated from the pool of volunteer respondents including those whose responses were both positive and negative regarding the CPHP Program. Of those who volunteer for a follow-up interview, it is anticipated that 100% will participate.

2. Procedures for the Collection of Information

As previously described, the procedures for information collection include semi-structured telephone interviews and web-based or paper surveys. All data collection will be completed by employed staff of the contractor, JSI.

Potential respondents for the CPHP Interview Instrument include the universe of CPHPs, thus sampling or estimation procedures are not required. The method of information collection is by semi-structured telephone interview. As described in the next section, respondents will have advance notice and will have scheduled appointments for the information collection. Semi-structured interviews will be conducted by JSI staff with training and experience in interview techniques and qualitative evaluation. Note takers will similarly have appropriate background and training. Transcription will be completed by a JSI staff member certified in transcribing.

Potential respondents for CPHP Customer/Partner Survey Instrument include the universe of Preparedness Coordinators for all state and major territorial health departments, as well as a large and diverse set of other governmental and non-governmental organizations involved in public health emergency preparedness planning and response. While sampling is not necessary for the first group (state, local, and territorial health departments), the challenges of developing a sample frame for the second group and the process for developing the sample are described in the preceding section. The survey will be primarily administered on-line, but a paper copy will also be available to potential respondents as many smaller, rural local health departments may prefer not to use web-based surveys; respondents will be offered the method most convenient for them. This flexibility should increase response rates and minimize perceived burden. The web administration option also should contribute to maximizing the timeliness, efficiency, and response rate of data collection.

Potential respondents for the CPHP Customer/Partner Follow-Up Interview Instrument will include those who volunteer for a follow-up interview in the CPHP Customer/Partner Survey. This is intended to add descriptive qualitative information to the quantitative survey findings. The method of information collection is by semi-structured telephone interview. Respondents will have advance notice and scheduled appointments for the information collection. Semi-structured interviews will be conducted by JSI staff with training and experience in interview techniques and qualitative evaluation. Note takers will similarly have appropriate background and training and transcription will be completed by a JSI staff member certified in transcribing.

3. Methods to Maximize Response Rates and Deal with Nonresponse

Following OMB approval, interview and survey materials displaying the OMB approval number and expiration date will be sent electronically to respondents. With respect to the CPHP Interview Instrument, key CPHP staff to be interviewed will be contacted in advance to establish a time for an interview. They will also receive an advance copy of the interview instrument in order to allow thoughtful responses and efficient use of the interview time. Lead CPHP staff have been informed of the program evaluation effort in national CPHP Program meetings and conference calls. Thus, CPHP staff are anticipating the information collection process and, given that we are collecting information from active grantees, it is reasonable to expect that 100% of the CPHPs will participate.

With respect to the CPHP Customer/Partner Survey Instrument, the introduction to the survey will contain the purpose of the information collection and directions for accessing the web-based survey. This introduction will emphasize the importance of input from CPHP organizational partners and customers. Each correspondence with the respondents will be personalized to improve response and ensure that the correct respondent is the recipient of the survey. If a respondent chooses the option to receive a paper survey, a packet will be mailed to the respondent with information identical to that received by respondents opting for the web-based survey as well as instructions for returning the completed survey to JSI.

As previously described, the target respondents for the survey instrument will be selected from a sample of organizational representatives thought to have a high likelihood of knowing of or having had contact with the CPHP Program. Approximately 1 week prior to disseminating the survey each sample respondent will be contacted by email or telephone to confirm his or her contact information and to provide advance notice of the survey. The name, job title, and contact information of the respondent will be recorded in a sample tracking database (maintained separately from survey responses) which will be used for all subsequent follow-up correspondence.

A 3-step process will be used to maximize the response rate to the survey:

1. Telephone call establishing initial contact information (Attachment F)

2. Email follow-up (Attachment G)

3. Telephone follow-up (Attachment H)

The availability of a web-based format is also expected to increase the response rate because it will ease administration of the survey.

With respect to the CPHP Customer/Partner Follow-Up Interview Instrument, the potential respondents are a subset of the survey sample. As part of the survey administration process, potential respondents may volunteer to be contacted for a brief follow-up interview to collect additional qualitative information on their experiences, perceptions and opinions.

4. Test of Procedures or Methods to be Undertaken

In order to obtain an estimate of the burden associated with the information collection instruments, an Evaluation Work Group was convened with representation from four CPHPs and the Association of Schools of Public Health. This work group assessed the efficacy of elements selected for inclusion in the instruments and provided feedback on the length, content and structure of the instruments, as well as the overall study design.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The plans for statistical analyses for this study were developed under a contract with JSI. The JSI team also participated with CDC staff in the design of the data collection and will conduct the data collection and analysis. Contact information for the JSI team is provided below:

Lea Ayers LaFave, PhD, RN

Senior Project Director

Community Health Institute/JSI

501 South Street

Bow, New Hampshire, 03304

603.573.3335

Email: [email protected]

Jonathan Stewart, MA, MHA

Director

Community Health Institute/JSI

501 South Street, Bow, NH 03304

tel: 603.573.3300 fax: 3301

Email: [email protected]

Rachel Kohn, MSW, MPH

Senior Consultant

Community Health Institute/JSI

501 South Street

Bow, NH 03304

Phone: 603-573-3321

Fax: 603-573-3301

Email: [email protected]