Supporting Statement B - 5.11.09

Improving the Quality and Delivery of CDC’s Heart Disease and Stroke Prevention Programs

Supporting Statement Part B

Contact: Lauren Gase, Health Scientist

Division for Heart Disease and Stroke Prevention

Centers for Disease Control and Prevention

Atlanta, GA

Telephone: (770) 488-8007

Fax: (770) 488-8151

Email: [email protected]

May 11, 2009

TABLE OF CONTENTS

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Respondent Universe and Sampling Methods

a. State Departments of Health

b. Private Sector Partners

2. Procedures for the Collection of Information

a. Surveys

b. Interviews and Focus Groups

3. Methods to Maximize Response Rates and Deal with Nonresponse

4. Tests of Procedures or Methods to be Undertaken

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or

Analyzing Data

LIST OF ATTACHMENTS

Attachment 1 Section 301 of the Public Health Service Act

Attachment 2A Federal Register Notice

Attachment 2B Summary of Public Comments and CDC Response

Attachment 3 Reference Set of Relevance, Quality, and Impact Questions

Attachment 4 DHDSP Evaluation Plan Overview and Logic Model

Attachment 5 Example Solicitation Letter

Attachment 6 Example Web-based Survey

Attachment 7 Example Interview Guide

Attachment 8 Example Focus Group Guide

Attachment 9 Generic Clearance Users Guide

Attachment 10 Data Collection Request Form

Attachment 11 Annual Report to OMB Template

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

B-1. Respondent Universe and Sampling Methods

Each DHDSP survey, interview, or focus group will target a specific set of partners. The respondent universe for each information collection will be different, depending on the type of DHDSP activity or service being evaluated. See Table B1.1 for a list of example partners. This is not an exclusive list; however, all respondents will be drawn from these categories of partners: DHDSP’s partners in State and local departments of health, the private sector, including nonprofit organizations, public health organizations, and professional organizations, and academic institutions.

Table B1.1 Respondent Universe
Partner	Examples (not exclusive list)
State (and District of Columbia) and Local Departments of Health	Minnesota Department of Health Texas Department of State Health Services California Department of Health Services DeKalb County Board of Health
Private Sector Partners: Nonprofit Organizations	American Heart Association National Stroke Association Kaiser Family Foundation Robert Wood Johnson Foundation National Business Group on Health
Private Sector Partners: Public Health Organizations	Association and State and Territorial Health Officials National Association of County and City Health Officials
Private Sector Partners: Professional Organizations	National Association of Chronic Disease Directors Association of Black Cardiologists American Medical Association American Public Health Association Society for Public Health Education
Academic Institutions	Emory Prevention Research Center University of Washington Health Promotion Research Center UCLA Center for Adolescent Health Promotion

The DHDSP estimates that it will conduct up to 17 web-based surveys, 120 interviews, and 12 focus groups each year over the three year life of the project. The actual distribution of activities may vary from year to year; estimates allow us to describe partners and estimate burden. See Table B1.2 for the breakdown of data collection mechanism by respondent type. The DHDSP cannot ask its partners to provide feedback on all of its products and activities. The DHDSP will set priorities for evaluation, based on its Division Evaluation Plan (See Attachment 4). The generic OMB clearance will provide the DHDSP the flexibility that it needs to evaluate its work while ensuring that its partners are not experiencing undue burden.

Table B1.2 Type of Data Collection Mechanism by Type of Respondent
Type of Respondent	Data Collection Mechanism	Number per Year	Number of People per Mechanism	Total Respondents per year
State and Local Health Department Chronic Disease Employees	Web-based survey	5	50	250
	In-person/telephone interview	2	15	30
	In-person/telephone focus group	4	8	32
Private Sector Partners: Nonprofit Organizations	Web-based survey	3	20	60
	In-person/telephone interview	3	10	30
	In-person/telephone focus group	2	8	16
Private Sector Partners: Public Health Organizations	Web-based survey	3	20	60
	In-person/telephone interview	3	10	30
	In-person/telephone focus group	2	8	16
Private Sector Partners: Professional Organizations	Web-based survey	3	20	60
	In-person/telephone interview	3	10	30
	In-person/telephone focus group	2	8	16
Academic Institutions	Web-based survey	3	20	60
	In-person/telephone interview	3	10	30
	In-person/telephone focus group	2	8	16

State and Local Departments of Health
Since State Department of Health employees are such a key partner, when they are identified as the respondent audience for a survey, the survey will be distributed to the full population. Surveying a representative from each of the 50 States and the District of Columbia will allow the DHDSP to capture the diverse opinions and needs of State Health Department staff and enable all States and the District of Columbia opportunity to provide feedback including those not funded by the DHDSP. Because the DHDSP’s guidance and products are often audience-specific (geared toward the State or local program manger, epidemiologist, or program evaluator), no one type of respondent will be asked to participate in more than two surveys, interviews, or focus groups annually.

When conducting surveys with local health department employees, a sample of up to 50 local health departments’ staff will be selected. In addition, when interviews or focus groups are conducted with State or Local partners, random selection, purposive sub-sampling, or snowball sampling will be used to select respondents. It is not feasible to conduct interviews or focus groups with representative from all 50 states or all local health departments; therefore, on average, 15 individuals will be selected for interviews and 8 individuals will be selected for each focus group. Selecting 15 individuals for interviews will give a representative cross section of responses. On average, selecting 8 individuals for focus group participation will allow everyone enough time to speak and should allow the topic to be covered completely.

When using random selection, the full list of State or Local health departments will be assembled. Each State or Local health department will be assigned a number and a computer program will be used to generate a random number to select the necessary number of States/Local health departments. The identified respondent (i.e. HDSP program manager, HDSP epidemiologist, WISEWOMAN program manger) from the State/Local health departments that are selected will then be contacted. Alternately, purposive sub-sampling will be used to select respondents when it is necessary to select only State/Local health departments who have accessed the service being evaluated (e.g. State/Local health departments who participated in the training, State/Local health departments who requested a copy of the CDC Stroke Atlas). In addition, purposive sampling may be used to ensure representativeness (i.e. to ensure a mix of funded and unfunded States, to ensure geographic variety of State/Local health departments). Finally, snowball sampling may be used when the full respondent universe is unknown (i.e. State/Local health departments may be asked to identify others who used the evaluation guide).

Private Sector Partners and Academic Institutions
A sample of partners may be surveyed instead of the full population when DHDSP conducts a large survey on a major DHDSP initiative that cuts across multiple partner groups. The specified sample for each of the private sectors partners and academic institutions, as shown in Table A12.1, is 20 individuals for surveys, 10 individuals for interviews, and 8 individuals for focus groups. Selecting 10 individuals for interviews will give a representative cross section of responses. On average, selecting 8 individuals for focus group participation will allow everyone enough time to speak and should allow the topic to be covered completely.

These individuals will be selected using random selection, purposive sub-sampling, or snowball sampling for the reasons described above.

A description of the respondent universe and the selected sample for each data collection will be provided to OMB in each annual report.

B-2. Procedures for the Collection of Information

Any individual within the DHDSP who wants to use this generic clearance must submit a “Data Collection Request Form” (See Attachment 10). Each proposed data collection will be reviewed by the DHDSP contact to ensure that it is consistent with the DHDSP evaluation plans and goals, as outlined in this package. In addition, each proposed data collection will be reviewed by the National Center for Chronic Disease Prevention and Health Promotion (NCCDPHP) PRA contact for preliminary determination of conformation to OMB regulatory requirements.

Instructions will be provided to all potential users of the DHDSP Generic Clearance on the procedures for use of the Generic Clearance and the process for submitting the request to the DHDSP and NCCDPHP PRA contacts (See Attachment 9). No data collection will be allowed to proceed until approval has been received from both the DHDSP and the NCCDPHP PRA contacts.

An instrument will be designed for each information collection based on the examples presented in Attachments 6-8. Questions will be exclusively drawn verbatim from the referenced set of preapproved questions (See Attachment 3). Instructions on how to use the reference set of question are included within the question bank itself (See “Instructions to Users” section of Attachment 3). As specified in A-1, if a data collection instrument must include questions that are not contained in the question bank, a change request approval will be submitted to OMB.

Three methodologies will be used to collect information: surveys, interviews, and focus groups. These will be conducted via the web, in-person, or over the telephone. Whenever feasible, data collections will be conducted over the web to minimize the burden on respondents.

Surveys
A web survey program, such as SurveyMonkey, will be used to design and conduct the surveys. As noted in section A-10, this program cannot be used to identify users.

In general, participation for surveys will be solicited through email. An example of this language is provided in Attachment 5. The contact information used to solicit participation in surveys will be kept separate from survey responses and no effort will be made to link surveys with the contact lists. The link to the Web-based survey will be included in the e-mail sent to potential respondents. The e-mail text will ask interested respondents to complete the survey within two weeks. All potential respondents will also receive another email two weeks after the initial email thanking those who have completed the survey and reminding others that they may do so in the following 2 weeks and that the survey will not be available after that time. Thus, the web-based surveys will be available on the Web for a total of 4 weeks.

Before taking the survey respondents will be informed about the purpose, provided an estimate of how long the survey will take, and notified that their participation is purely voluntary. Furthermore, respondents will be assured that they will not be penalized in any way if they choose not to participate or to skip any of the questions in the survey. Consent will be obtained from all participants before they begin. The consent form for web-based surveys will be the presented after information about purpose of the survey and use of the data is provided. An example of this language is presented in an example web-based survey (See Attachment 6).

Interviews and Focus Groups
In-person and telephone interviews or focus groups will only be used when web surveys are impractical, more burdensome, or in-depth responses are required. In-person interviews will be conducted in Atlanta, GA, during program site visits, or at the location of events attended by both CDC staff and partners. The interviews and focus groups will be audio taped, with the permission of participants, or extensive notes will be taken. In order to reduce expenses and the burden on respondents, in-person surveys or focus groups involving geographically distributed respondents will be conducted by phone.

CDC employees, fellows, full-time contractors, or contract vendors will collect the data. Any individuals conducting interviews or focus groups will be trained on the protocol before implementation.

Participation for interviews and focus groups will be solicited through email and phone calls. Before participating in a focus group or interview, respondents will by informed about the purpose, provided an estimate of how long the interview or focus group will take, and notified that their participation is purely voluntary. An example of this language is provided in Attachment 5. Furthermore, respondents will be assured that they will not be penalized in any way if they choose not to participate. Consent will be obtained from all participants before they begin. Consent for interviews and focus groups will be obtained through an oral process after information about purpose of the interview or focus group and use of the data is provided. An example of this language is presented in an example interview guide (see Attachment 7) and an example focus group guide (see Attachment 8).

B-3. Methods to Maximize Response Rates and Deal with Non-response

Multiple strategies during the instrument conception, design and administration will be used to maximize response rates. Before designing a data collection instrument, consultations with partners and DHDSP leadership will ensure the instrument addresses a topic that contributes to improvements in DHDSP activities. The DHDSP will set priorities for evaluation based on its Division Evaluation Plan (See Attachment 4). In addition, the DHDSP will solicit input from a few volunteers from the identified partner group and/or individuals within the DHDSP who work closely with the identified partner group as to which products are most frequently used. By pre-screening topics and selecting services which partners have accessed and used, the DHDSP will increase partners’ motivation to participate and provide their feedback.

The survey design process will maximize responses by ensuring the survey is easy to complete. The clarity of the survey will be enhanced by using primarily pre-screened questions from the question bank. Moreover, for each instrument, brief pilot tests with up to five members of the target audience and/or individuals within the DHDSP who work closely with the identified audience, will be conducted to ensure that it is clear and does not exceed time limits.

The method of distribution will maximize response rates by delivering the survey to respondents in the least burdensome manner and using systematic communication. The majority of surveys will be conducted through the internet so respondents can complete the survey at their convenience and at their own computer. Phone and in-person interviews or focus groups will be used when the survey includes predominantly open-ended questions. These strategies reduce respondent burden by enabling them to provide their answers orally instead of writing them down. Finally, response rates will be boosted by using a systematic communication process to notify respondents of the survey, distribute the survey, and provide reminders to respondents to complete the survey. Two email messages to potential respondents will be sent; both the initial and the reminder email messages will include a direct link to the survey Web site, so those who desire to respond to the survey only have to click on the link.

The DHDSP estimates that the response rate for these data collections will be 80% or higher. A similar inquiry undertaken by the DHDSP (OMB# 0920-0764, exp. 2/28/09), to evaluate if and how State Health Departments used the Successful Business Strategies to Prevent Heart Disease and Stroke Toolkit, achieved a response rate of 80% (40 out of 50 States Health Department respondents completed the survey). Data collected through this clearance will utilize similar data collection mechanisms and procedures.

The DHDSP will track non-response rates. If non-response rates are high, DHDSP will assess the reasons for non-response rates and modify the appoach to address these issues.

B-4. Tests of Procedures or Methods to be Undertaken

All questions will be selected from the question bank that has been reviewed by two groups of survey experts for usability. For each instrument, brief pilot tests with up to five members of the target audience, will be conducted to ensure that it is clear and does not exceed time maximums. Changes to the instrument will be made based on pilot test results.

B-5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Each information collection will be led by an employee in the Applied Research and Evaluation Branch of the DHDSP. This person will be responsible for overseeing all phases of the project including designing the data collection instruments, collecting the data, and analyzing the data. For each information collection, the team of individuals performing this work is likely to vary. However, as described in A-14, all information collections will be supervised by a DHDSP coordinator. The DHDSP coordinator will be a federal employee who, in close consultation with the NCCDPHP PRA contact, will review all the data collection instruments to monitor the number of partners being surveyed, confirm that instruments comply with time limits, ensure that a group of partners is not being overburdened by multiple surveys, and verify that the instruments comply with the guidelines outlined in this OMB request.

Analyses will be conducted by a member of the evaluation team trained in research methods or a contractor with appropriate training. In addition to the statistical training of DHDSP evaluation staff, a number of whom are PhD scientists, statistical support can be requested from DHDSP statisticians and epidemiologists if needed.

The team of individuals working on each information collection, including instrument development, data collection, and data analysis will consist of members of the DHDSP’s Evaluation and Program Effectiveness Team, as listed in Table B5.1.

Table B5.1 Staff Responsible for Instrument Design, Data Collection and Analyses
Name	Agency	Telephone Number	Email
Diane Dunet	CDC/DHDSP	770 488-8007	[email protected]
Lauren Gase	CDC/DHDSP	770 488-8007	[email protected]
Eileen Chapelle	CDC/DHDSP	770 488-8144	[email protected]
Rashon Lane	CDC/DHDSP	770 488-8036	[email protected]
Aisha Tucker-Brown	CDC/DHDSP	770 488-8179	[email protected]
Rachel Barron-Simpson	CDC/DHDSP	770 488-4825	[email protected]
Susan Ladd	CDC/DHDSP	770 488-5448	[email protected]

The majority of data will be analyzed using basic descriptive analyses. Because the major purpose of this data collection is program improvement, DHDSP does not anticipate needing to use complex statistical techniques.

9