OMB SS_20070417 - SG 4-23-2007 - FINAL1 A corrected A12EH,jfk - FINAL

Revised 4/26/07

OMB SUPPORTING STATEMENT

SUMMARY and SECTION A – JUSTIFICATION

EVALUATION OF THE CENTERS FOR DISEASE CONTROL AND PREVENTION’S CONSUMER RESPONSE SERVICES CENTER

CDC-INFO

Dottie Knight

Program Manager

1600 Clifton Road

M/S E-06

Atlanta, GA 30333

(404) 498-3208

[email protected]

National Center for Health Marketing

Centers for Disease Control and Prevention

Table of Contents

Table of Contents

Summary................................................................................................................................ 1

Program/Study Objectives......................................................................................... 1

Similar Information…............................................................................................... 2

Legislative authority.................................................................................................. 2

Study Design.............................................................................................................. 2

Customer Satisfaction.................................................................................... 3

1. Interactive Voice Response Survey (Satisfaction Survey for Callers)…… 3

2. Live Phone Follow-up Survey (Follow Up Survey) 3

3. Web Survey for E-Mail Inquirers (Satisfaction Survey for E-Mail Inquiries)………………………....................................................... 3

4. Postcard Survey for Single Publication Orders (Postcard Survey for Individual Publications) 4

5. Postcard Survey for Bulk Mailing 4

6. Web Survey for Internet Publication Orders (Web Survey for

Internet Publications) 4

7. Web Survey for E-Mailed Publication Orders................................... 4

Quality Improvement..................................................................................... 5

8. Customer Representative Survey (Key Informant Survey)………………………......................................................... 5

Outreach Assessment..................................................................................... 5

9. Special Outreach Surveys (General Public) (Special Event/Outreach Survey—General Public) 5

10. Special Outreach Surveys (Professionals) (Special Event/Outreach Survey—Professionals)......................................... 5

Emergency Response..................................................................................... 5

11. Emergency Response Surveys (General Public; Level 1-4) 6

12. Emergency Response Surveys (Professionals; Level 1-4) 7

Use of Information..................................................................................................... 8

A. Justification....................................................................................................................... 9

1. Circumstances Necessitating the Information Collection.................................... 9

2. Purpose and Use of Information Collection 10

3. Use of Improved Information Technology and Burden Reduction 11

4. Efforts to Identify Duplication and Use of Similar Information 11

5. Impact of Small Businesses of Other Small Entities 11

6. Consequences of Collecting the Information Less Frequently 11

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5 12

3. Comments in Response to the Federal Register Notice and Efforts to Consult

Outside the Agency 12

3. Explanation of Any Payment or Gift to Respondents 12

4. Assurance of Confidentiality Provided to Respondents 12

- Overview of privacy protections by survey type and data collection method

13

- Overview of privacy obligations by personnel involved 14

- Privacy protections pertaining to data storage and retrieval 15

- Privacy protections pertaining to the dissemination of evaluation results 15

- Other information relevant to privacy protections for respondents 15

- Privacy and confidentiality of general operations 16

3. Justification for Sensitive Questions 19

4. Estimate of Annualized Burden Hours and Costs 21

5. Estimates of Other Total Annual Cost Burden to Respondents or Recordkeepers 29

6. Annualized Cost to the Government 29

7. Explanation for Program Changes or Adjustments 36

8. Plans for Tabulation and Publication and Project Time Schedule 36

9. Reason(s) Display of OMB Expiration Date is Inappropriate 36

10. Exceptions to Certification for Paperwork Reduction Act Submissions 36

List of Attachments................................................................................................... 37

SUMMARY

The Centers for Disease Control and Prevention proposes to conduct continual national evaluation of the agency’s Consumer Response Services Center—CDC-INFO. CDC-INFO is a proactive, unified, and integrated approach to the delivery of public health information. CDC-INFO is designed to contribute to improving the health and safety of the public. Customers are defined as any individual or group seeking health or public health information from CDC. This includes the public, media, medical and healthcare professionals, public health professionals, partner groups, businesses, researchers, and others. Customer interactions occur through multiple channels, e.g., telephone calls, e-mails, and postal mail and may occur in Spanish or English.

Program/Study Objectives

(1) Proactively evaluate customer interactions and service effectiveness by employing assessment measures and data collection mechanisms to support performance management, gathering insights and understandings for improving service levels, and implementing effective measures to meet customer satisfaction goals;

(2) Develop an ongoing understanding of customer requirements and satisfaction trends to achieve “best practice” quality standards and to provide qualitative assessments, quantitative data, and cost factors to drive improvement and reinforce operational objectives; and

(3) Measure CDC-INFO contractor service performance to assist in determining whether performance incentives have been achieved.

(4) Contribute to overall learning in order to advance public sector consumer response service centers.

In February 2005, CDC began a four-year effort to combine its 40+ hotlines, interactive voice response services system, and clearinghouses into one integrated service--CDC-INFO. There are three integrated components contractors that comprise CDC-INFO. One contract (Pearson Government Solutions) is the customer call, e-mail and mail center. A separate contract will be established with a fulfillment contractor to distribute printed materials requested by inquirers. The third contract is with Evaluation, Management and Training Associates, Inc. (EMT), an independent program evaluation contractor (PEC) to support knowledge development and continual evaluation of the conduct and customer satisfaction of CDC-INFO. The customers see only a single focus for all these activities through the integrated systems which support CDC-INFO. CDC-INFO has an exceptionally wide scope because of the range of health topics handled by CDC. In its first year the contact center alone will have handled 700,000 inquiries, including response to Hurricane Katrina, seasonal influenza, HIV Testing Day and Avian and Pandemic Flu. When transitions are complete, CDC-INFO will handle an estimated 3,000,000 inquiries annually. CDC-INFO itself will be operational for at least the next six years. The planned transition schedule is depicted in Attachment B.

The CDC seeks OMB approval for data collection over the next three years. An OMB extension will be sought at the end of the first three year period, for an additional three years. Data collection would begin upon OMB approval and conclude in October of 2012, provided the OMB extension is approved.

Similar information

Only one of the 40+ CDC hotlines, interactive voice response services systems, and clearinghouses (National AIDS and Sexually Transmitted Disease) was evaluated on a national scale (OMB approval 0920-0295, expiration date 12/31/2006). Funded in 1998, the National AIDS and STD Hotline Survey of Callers involved the collection of data used for the following purposes: to assess the impact of AIDS and STD public information programs; to manage the AIDS and STD hotline effectively; to determine the extent of their utilization by the general population; and to evaluate CDC’s AIDS and STD hotline services.

Literature searches within CDC’s programs and with 30 other Federal agencies were conducted to identify studies with similar goals and expected outcomes. No similar study of consumer response services which integrate information and referral services for more than 40 health-related topics has been conducted. Even though the present evaluation is unique from others that have been conducted in the field, information from these literature searches has helped the proposed evaluation design and enhanced the expected utility of study results. In addition, a great deal of published information is available about private sector call centers. An exhaustive review of this body of research informed the evaluation of CDC-INFO presently being proposed.

The proposed evaluation of CDC-INFO is compliant with the procedures and principles outlined in the U.S.A. General Services Administration’s Citizen Service Levels Interagency Committee (CSLIC) report. The CSLIC report describes performance measures, practices and approaches for government-wide citizen contact activities.

Legislative authority

This study is authorized under the Public Health Service Act (42 USC 241) Section 301 (Attachment C).

Measurement of performance indicators such as consumer satisfaction and timeliness of response are now mandated at all levels of the Federal government, and for Federal Contact Centers in particular (CSLIC Report: Proposed Performance Measures, Practices and Approaches for Government-wide Citizen Contact Activities; Attachment D).

Study Design

There are twelve (12) potential evaluation points, divided into four (4) major categories. All survey tools provide the participant an opportunity to decline. All evaluation processes are voluntary and all available in English and Spanish.

Customer Satisfaction

1. Interactive Voice Response Survey (Satisfaction Survey for Callers):

Each year, 100% of the CDC-INFO callers will be offered the opportunity to participate in an initial interactive voice response (IVR – touchtone answer) completely automated survey at the end of their call with the customer service representative. The total number of expected callers is 500,000. They will be routed electronically, without human intervention (see English questions in Attachment E, Spanish questions in Attachment F). Based on previous evaluation efforts conducted by EMT and private sector literature related response rates for consumer response services centers, EMT estimates that approximately 5% will choose to participate. The resulting sample size would number 25,000 callers who opt to participate in the IVR satisfaction survey.

2. Live Phone Follow-up Survey (Follow Up Survey):

If callers accept and complete the initial automated survey, they will then at the end of the survey (#1 above) and be offered an opportunity to be called with an additional follow-up survey which they accept by inputting their phone number using their touchtone pad (no names will be obtained). Callers who opt to participate in the live phone follow-up survey will be given an easy code word (i.e., CDC) to say when contacted for the live phone follow up survey. When those who consent are contacted for the follow up survey, they will be required to provide the code word before any questions are asked. Callers who opt to participate in the live follow up phone survey will be surveyed using a structured interview protocol by a live representative with the PEC not sooner than 15 days later and if not initially reached, efforts to contact then will continue for not more than 6 weeks. After either completion of the follow-up survey or unsuccessful contact, the phone number will be discarded. Based on EMT’s work conducting follow up surveys of callers to a Los Angeles County consumer response services center, the projected acceptance rate is of 25%. The projected sample size of 25% of 25,000 IVR survey respondents would yield a sample size of 6,250 callers for the live follow up telephone survey (see English protocol in Attachment G, Spanish protocol in Attachment H). It is projected that we will actually reach 50% of those who accept, for a total sample size of 3,125 follow-up interviews.

3. Web Survey for E-Mail Inquirers (Satisfaction Survey for E-Mail Inquiries):

All e-mail inquiries (not including publications) will be offered an opportunity to complete a web-based feedback survey by selecting the link shown at the bottom of the response e-mailed to them. The total volume of expected e-mail inquiries is 11,000. Based on EMT’s prior evaluation work with consumer response services centers, it is estimated that approximately 3% will respond. The sample size of 3% out of 11,000 e-mail inquirers is 330 (see Attachment I for the English version, Attachment J for the Spanish version).

4. Postcard Survey for Single Publication Orders (Postcard Survey for Individual Publications):

In all mailed materials (not including bulk orders), a brief paper and pencil survey (prepaid postcard addressed to EMT) will be included by the fulfillment contractor. Customers can elect to complete the questions and drop the pre-paid postcard in the mail. EMT estimates that out of the projected 210,000 single publication orders filled, only 1% of those receiving publications will elect to participate. The projected sample size when considering 1% of 210,000 single publication orders is 2,100 (see Attachment K for the English version, Attachment L for the Spanish version).

5. Postcard Survey for Bulk Mailing (Postcard Survey for Bulk Mailing):

In all bulk mailed materials (when more than one of the same publication is ordered), a brief paper and pencil survey (prepaid postcard addressed to the PEC) assessing satisfaction with the fulfillment process will be included by the fulfillment contractor. Customers can elect to complete the questions and drop the prepaid postcard in the mail. EMT estimates that out of a projected 95,000 bulk mailings, only 1% of those receiving bulk mailing will elect to participate. The sample size given a projected 95,000 bulk mailings and 1% response rate is 950 (see Attachment M for the English version, Attachment N for the Spanish version).

6. Web Survey for Internet Publication Orders (Web Survey for Internet Publications):

Following placement of an Internet publication order, a confirmation e-mail is sent to the requestor. Included in the confirmation e-mail will be a link to a web survey assessing the ordering process. Customers can elect to complete the web-based feedback survey by selecting the link shown at the bottom of the confirmation e-mail. EMT estimates that out of a projected 95,000 Internet publication orders, only 1% of those ordering publications over the Internet will elect to participate. The sample size given a projected 95,000 Internet orders and 1% response rate is 950 (see Attachment O for the English version, Attachment P for the Spanish version).

7. Web Survey for E-Mailed Publication Orders (Web Survey for E-Mail Publications):

Following placement of an order for a single electronic publication, an e-mail containing the publication in PDF format and/or the URL where the publication can be downloaded from the Internet is sent to the requestor. Included in the e-mail will be a link to a web survey Customers can elect to complete the web-based feedback survey by selecting the link shown at the bottom of the confirmation e-mail. EMT estimates that out of a projected 100,000 e-mailed publication orders, only 1% of will elect to participate. The sample size given a projected 100,000 e-mailed publication orders and 1% response rate is 1,000 (see Attachment Q for the English version, Attachment R for the Spanish version).

Quality Improvement

8. Customer Representative Survey (Key Informant Survey)

CDC-INFO customer service representatives (key informants; approximately 100) will be surveyed following training using a web questionnaire. It is anticipated that training will occur on an annual basis. These surveys will allow further insight into CDC-INFO quality (see Attachment S—all Customer Service Representatives at CDC-INFO speak English).

Outreach Assessment

9. Special Outreach Surveys (General Public) (Special Event/Outreach Survey—General Public)

Whenever CDC conducts a national information campaign or media event and inquirers are directed to contact CDC-INFO for further information, special outreach surveys may be offered by the live customer service representative at the end of the call to assess general public response. Separate special outreach surveys have been designed for professionals versus the general public (callers who are not contacting CDC-INFO on behalf of an agency, organization, or as part of their job). Out of an estimated total of 32,000 inquiries from the general public, the estimated response rate is approximately 80% or 25,600 respondents (see Attachment T for the English version, and Attachment U for the Spanish version).

10. Special Outreach Surveys (Professionals) (Special Event/Outreach Survey—Professionals)

As noted above, whenever CDC conducts a national information campaign or media event and inquirers are directed to contact CDC-INFO for further information, special outreach surveys may be offered by the live customer service representative at the end of the call to assess inquirer response. Out of an estimated total of 13,000 inquiries from professionals, the estimated response rate among professional callers is 80% or 10,400 respondents (see Attachment V for the English version, and Attachment W for the Spanish version).

Emergency Response

CDC requires flexibility in the emergency response sample size because the size and scope of emergencies cannot be predicted in advance. A localized emergency, for example, would result in fewer calls to CDC-INFO compared to a national emergency such as an outbreak of pandemic avian influenza. Therefore a range of sample sizes is proposed based on the size of the locale impacted by the emergency. CDC does not expect to utilize the maximum sample size for emergency response surveys annually. As the title of the survey implies, it will be activated only in the event of a public health emergency. Outlined below are four levels of emergency response, based on data collected by the CDC.

Level 1

• Impact: Metro-area event  (Could also be county-wide/multi-county or state low-level surge)

• Population: 828,780

• Est. Calls Per Day: 8,288- 41,439 (1%-5% of pop)

• Example: SC Train derailment (chlorine), 2006; SARS, Toronto, 2003

Level 2

• Impact: State-wide event

• Population: 5,757,949

• Est. Calls Per Day: 57,579- 287,897 (1%-5% of pop)

• Example: Hantavirus, Colorado, 1993; Bay Area earthquake, 1994

Level 3

• Impact: Regional or multi-state event

• Population: 35,185,263

• Est. Calls Per Day: 351,853-1,759,263 (1%-5% of pop)

• Example: Hurricane Katrina, 2005;  Anthrax, 2001; Avian flu in US birds

Level 4

• Impact: National event

• Population: 298,251,839

• Est. Calls Per Day: 2,982,518-14,912,592 (1%-5% of pop)

• Example: 9/11terrorist events, 2001; Pandemic Flu, human-to-human transmission

11. Emergency Response Surveys (General Public) (Emergency Response Survey—Levels 1-4—General Public)

In the event of a public health emergency or outbreak where CDC has directed callers to CDC-INFO for more information and/or to report their exposure or their health status, live customer service representatives may administer the Emergency Response Survey for Individuals. At the conclusion of the call, customer service representatives will ask callers (non-professionals) for voluntary demographic information (included in Attachment X). The survey will ask the kinds of questions that would have to be answered in the course of helping the caller through the emergency Attachment X for the English version, Attachment Y for the Spanish version).

• For a Level 1 emergency, the estimated number of respondents is 31,151. The expected response rate is approximately 75% of 41,439 callers.

• For a Level 2 emergency, the estimated number of respondents is 57,579. The expected response rate is 80%--however, CDC-INFO data collection capacity dictates that only 20% of the respondents can be surveyed out of 287,897 callers. In the event of a Level 2-4 emergency a certain proportion of the incoming calls will be handled by overflow systems which do not include CDC-INFO data collection protocols. The sample sizes projected below take CDC-INFO capacity into consideration.

• For a Level 3 emergency, the estimated number of respondents is 351,863. The expected response rate is 80%--however, CDC-INFO data collection capacity dictates that only approximately 20% of the respondents can be surveyed out of 1,759,263 callers.

• For a Level 4 emergency, the estimated number of respondents is 645,630. The expected response rate is 80%--however, CDC-INFO data collection capacity dictates that only 5% of the respondents can be surveyed out of 12,912,592 callers.

CDC currently has umbrella approval for Emergency Epidemic Investigations (OMB No. 0920-0008, expires 3/31/2010) which authorizes EIS officers to collect data in response to public health emergencies. Data collected through CDC-INFO will serve as a complement to the field epidemiological data collection by EIS.

12. Emergency Response Surveys (Professionals) (Emergency Response Survey—Levels 1-4—General Public)

In the event of a public health emergency or outbreak where CDC has directed professional callers to CDC-INFO for more information and/or to report on the public health emergency or outbreak, live customer service representatives may administer the Emergency Response Survey for Professionals. The survey will ask the kinds of questions that would have to be answered in the course of helping the caller through the emergency (see Attachment Z for the English version, Attachment AA for the Spanish version).

• Emergency Response Surveys (Professionals): For a Level 1 emergency, the estimated number of respondents is 7,459. The expected response rate is 90% of 8,288 callers.

• For a Level 2 emergency, the estimated number of respondents is 51,821. The expected response rate is 90% out of 57,579 callers.

• For a Level 3 emergency, the estimated number of respondents is 316,678. The expected response rate is 90% out of 351,853 callers.

• For a Level 4 emergency, the estimated number of respondents is 596,504. The expected response rate is 90%--however, CDC-INFO limitations on data collection capacity mandate that only 20% of 2,982,518 callers can be surveyed. . In the event of a Level 4 emergency a certain proportion of the incoming calls will be handled by overflow systems which do not include CDC-INFO data collection protocols. The sample size projected takes CDC-INFO capacity into consideration.

The evaluation of all the collected data will be conducted by EMT. This data collection is scientifically appropriate, employs measures to safeguard the privacy and confidentiality of participants, and supports program and research needs of CDC and other Federal agencies. Sample size, respondent burden, and intrusiveness have been minimized consistent with study objectives (see Section B for complete survey collection methods and sampling methods). Pilot testing with not more than nine (9) respondents assisted in controlling burden and ensuring the user-relevance of questions.

Use of Information

The evaluation results will have significant implications for Federal consumer response service programs and research activities conducted by multiple Federal, state, and local government agencies. Results will be used to refine and develop operational procedures, promote replication and dissemination of model consumer response services operational procedures, and provide technical assistance to ensure that such programs can contribute appropriately to ensure consistent, timely, reliable information is disseminated to a variety of consumers (public, health professionals, researchers, etc.) and address variations in inquiry volumes related to public health emergencies, news events, and dynamic, shifting public health priorities. In addition the results can contribute to the learning of other health-related consumer response services programs, such as the national 211 and 311 initiatives. 211 and 311 represent alternatives to 911, with 311 for county/borough/parish non-emergency services, and 211 for social, educational and public health services. Each state is presently developing plans for the implementation of 211 and 311. Information generated by the CDC-INFO evaluation will certainly be helpful in furthering these initiatives. While a great deal of information is available about private sector call centers (Attachment BB), little evaluation information is available about public sector consumer response services centers. The evaluation of CDC-INFO will provide this much-needed information.

A. JUSTIFICATION

1. Circumstances Necessitating the Information Collection

The Centers for Disease Control and Prevention’s (CDC) mission is to promote health, safety, and quality of life by preventing and controlling disease, injury, and disability. As the Nation’s lead prevention agency, CDC works with States, local public health agencies, and partners throughout the Nation and the world to accomplish this mission.

Increasingly, the dissemination of health and safety information plays a critical role in the Nation’s efforts to prevent and control diseases, injury and disabilities. While recent events (e.g., anthrax attacks) have heightened public awareness and their need to know, a growing focus is on CDC to educate the public, medical personnel, clinicians, public health workers, and the media on disease specific details, risk, and prevention information. Direct customer interaction has become an important and visible part of the agency’s effort to carry out its mission. Therefore, CDC is faced with the major challenges associated with information delivery that has become more voluminous and complex.

While these challenges are not new, they are afforded a higher level of attention and driven by an evolving mission. Historically, CDC employed several methodologies for disseminating information that responded to a narrowly defined constituency, to a specific need or legislative mandate, or in accordance with budget and technology available. Unfortunately, these approaches grew to be disparate, fragmented, and resource-intensive. Additionally, fragmented and independent collections of information resulted in inconsistency in levels of service and a disjointed interface to the public. (Historical Evolution of Consumer Response Centers, Attachment BB)

From an agency/enterprise perspective, this approach to information dissemination resulted in significant operational inefficiencies. Each organizational unit replicated processes and redundant activities decreasing the availability of funding resources. Given the rising public expectation of services and diversity of needs, costs associated with meeting demand are increasing exponentially. Clearly, it is more difficult to measure accountability in a fragmented environment. The agency is unable to view/evaluate the overall dissemination delivery process, to subsequently formulate predictive models of needs and to potentially introduce industry “best practices.” With the constrained budgetary and resource environment, the agency is seeking a streamlined model that will reduce unit costs while enriching service.

Policy makers need to know how to reach more consumers and health professionals with cost-efficient consumer response services. With CDC-INFO, CDC has funded consolidation of 800 topics into one contact center, designed to be the CDC’s “one face to the public.” The goal of CDC-INFO is to optimize customer interaction by providing a proactive, unified, and integrated approach to the delivery of public health information and thereby improve the health and safety of the public. The goal of this evaluation is to assess customer satisfaction and needs, and contribute to improved service.

This study is authorized under the Public Health Service Act (42 USC 241) Section 301 (Attachment C). Measurement of performance indicators such as consumer satisfaction and timeliness of response are now mandated at all levels of the Federal government (U.S.A. General Services Administration’s CSLIC Report, Attachment D).

2. Purpose and Use of Information Collection

The proposed study and attendant data collection activities will help support CDC in achieving its information dissemination objective “to ensure consistent, timely, reliable health information is disseminated to meet the needs of a evolving diversity of consumers (public, health professionals, researchers, etc.) and to address variations in inquiry volumes related to public health emergencies, news events, and dynamic, shifting public health priorities” (CDC-INFO Request for Funding Announcement). Providing a full complement of information support for CDC-INFO, the new “one face to the public” will require a comprehensive, multi-phased system of quality assurance, performance monitoring and impact evaluation. This evaluation will provide CDC with information about the effectiveness of CDC-INFO.

The information generated through this evaluation will be used primarily by CDC in its efforts to assess consumer satisfaction and service effectiveness and to continually improve services. CDC Programs will use evaluation information to refine efforts to reach elect populations. In addition, CDC will examine health messages for effectiveness and customer satisfaction. Information gained from this evaluation and lessons learned will be shared with other Federal organizations operating consumer response services centers.

In addition, the information will be made available to State and local program planners and the public through publications. States and localities launching 211 and 311 efforts will find the evaluation findings informative as they design state-of-the-art consumer response services centers. 211 and 311 represent alternatives to 911, with 311 for county/borough/parish non-emergency services, and 211 for social, educational and public health services. Each state is presently developing plans for the implementation of 211 and 311. Information generated by the CDC-INFO evaluation will certainly be helpful in furthering these initiatives. While a great deal of information is available about private sector call centers, little evaluation information is available about public sector consumer response services centers. The evaluation of CDC-INFO will provide this much-needed information. Policy makers in State and local governments will have evidence on “best practice” with regard to information dissemination by consumer response services centers and strategies for maintaining high customer satisfaction with disparate populations. The evidence will be useful in setting policy priorities (e.g., topics and services to be covered through statewide and local 211 services).

In sum, the findings and conclusions from the study will be a crucial resource for the CDC policy priorities, performance measurement, and in designing and promoting an optimally effective consumer response services center. Although the study is designed to address primarily the needs of the CDC, the evaluation results will be useful to other Federal agencies that are involved in information dissemination through consumer response services centers.

3. Use of Improved Information Technology and Burden Reduction

To the maximum extent possible, the evaluation has taken advantage of improved technology to reduce burden to consumers. The majority of the surveys use either interactive voice response (touch tone) technology or web surveys. Only the follow-up survey, special outreach and emergency response involves live operators to administer the surveys (List of Technologies Used—Attachment CC).

4. Efforts to Identify Duplication and Use of Similar Information

Among the over 40 consumer response services programs operated in the past by CDC, only one has been evaluated (AIDS and STDs, OMB No. 0920-0295, expiration date 12/31/06). To date, no published report has been produced documenting the results of the AIDS and STDs evaluation.

Staff at more than 30 Federal agencies were interviewed in an effort to identify duplication. None of these Federal organizations has conducted a national evaluation of a consumer response services center that is similar to the one being proposed for this study. The majority of those contacted offer specialized information specific to one health issue. These centers serve as referral resources for CDC-INFO (e.g., individuals who call CDC-INFO requesting specialized treatment information for advanced AIDS patients are referred to the National Institutes of Health’s AIDS INFO service).

A comprehensive literature search was conducted. To date there has been no published evaluation of consumer response services programs comparable scope to this study - a study spanning more than 40 health-related topics, or using the same measures to document satisfaction and impact. Those Federal agencies who conducted evaluation studies of their consumer response services centers were examined closely to take advantage of applicable methods and to identify any methodological problems that might detract from the validity, generalizability, or application of results.

5. Impact of Small Businesses or Other Small Entities

There is no burden on small businesses or small entities. Participation will be voluntary.

6. Consequences of Collecting the Information Less Frequently

Participation will be voluntary and one-time responses are expected from potential responders. There are no legal obstacles to reduce the burden.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances with this information collection package.

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

A. A 60-day Federal Register Notice was published in the Federal Register on March 22, 2007, Vol. No. 72, pages 13495-13496 (Attachment A). One comment was received and it is attached to this ICR.

B. No consultation was conducted outside the agency. Extensive consultation was obtained from CDC Program Officers.

9. Explanation of Any Payment or Gift to Respondents

There will be no remuneration to respondents.

10. Assurance of Confidentiality Provided to Respondents

The CDC Privacy Officer reviewed this submission and determined that the Privacy Act does not apply to evaluation data collected for CDC-INFO. Evaluation data to be collected under this clearance are separate from the general operations of the Consumer Response Services Center. Evaluation surveys do not involve the collection of personal information from respondents, evaluation data will not be identifiable, and evaluation data will not be stored in a system of records as defined in the Privacy Act.

CDC’s Consumer Response Service Center was established to respond to the public’s requests for public health information.  Data relevant to the requestor’s identity (e.g., name, telephone or fax number, e-mail or postal address) are often required for CDC-INFO’s routine operational purposes.  By “relevant to the request’s identity,” we mean that the data may suggest identity, without being absolute identifiers of an individual person (e.g., SSN).

In some cases, the evaluation data to be collected under this clearance can be distinguished from the general operations-related information that is needed to fulfill CDC-INFO’s core mission.  Distinctions between evaluation data (which is the major subject of this OMB approval request) and operations-related information (which is generally outside the scope of this clearance request) are described in this section, where relevant to Privacy protections or Privacy Act applicability.

As described in the Summary section of this submission, the overall evaluation of CDC-INFO involves 12 evaluation points (or survey types) in four categories (customer satisfaction, quality improvement, outreach assessment, and emergency response). Because the evaluation data will be collected using a variety of methods (telephone, postal, electronic), the procedures for safeguarding respondent privacy vary according to the data collection method and the personnel involved (CDC staff and/or contractors). A summary of measures for safeguarding the privacy of respondents follows. The protections for respondent privacy are therefore relative to the data collection method, the sensitivity of the information being collected, the specificity of the data with respect to respondent identity, and the personnel or organizations (e.g., CDC and contractors) involved in the data collection.  The following discussion of privacy protections for survey respondents is organized by data collection purpose and method.

Overview of privacy protections by survey type and data collection method

1. Interactive Voice Response (IVR) customer satisfaction survey (Satisfaction Survey for Callers) (Attachments E and F). Callers who agree to participate are routed electronically to the IVR survey and there is no further contact with a Customer Service Representative. No personal information about the respondent, such as name or telephone number, is collected or stored with the responses to the customer satisfaction survey.

2. Live follow-up customer satisfaction survey (Follow Up Survey) (Attachments G and H). A caller who completes the IVR survey and opts to participate in the extended, live follow-up survey leaves his or her telephone number via an automated voice response system. The caller’s number is not captured automatically and the call-back number is not linked to the caller’s audio record in any way. After the caller keys-in the call-back number, the caller is given a code word to be used for future authentication purposes. The code word is used at the time of follow-up contact for the satisfaction survey to ensure that no identifying information about the study is provided until the person answering the telephone (i.e., the intended respondent) provides the code word. This step prevents an individual who did not contact CDC-INFO from possibly finding out that their partner, child, roommate, etc. contacted CDC-INFO. Following the collection of live follow-up telephone survey data, the phone number record is destroyed. The survey respondent does not disclose his or her name at any time in the process of participating in the live follow-up survey. The PEC (Program Evaluation Center) will NOT record the survey calls. Quality control will instead be accomplished by the PEC’s Quality Assurance Manager sitting in with the PEC surveyor on 10% of the surveys (randomly selected) and listening to the PEC surveyor. The PEC Quality Assurance Manager will NOT be on the phone call but rather in the room listening to the PEC Surveyor administer the survey. The purpose is to gauge adherence to the survey. It is therefore not necessary for the PEC Quality Assurance Manager to hear the respondent’s answers. In summary, the PEC will NOT tape ANY of the live follow-up customer satisfaction surveys.

3. Postcard-format customer satisfaction survey (Postcard Survey for Bulk Mailing and Postcard Survey for Individual Publications) (Attachments K, L, M, and N). The fulfillment contractor will insert a postcard survey in each package of publications/printed materials mailed to a customer. Return postage for the postcard survey is prepaid, and the postcard survey is pre-addressed with the return address for the PEC (Program Evaluation Center). The satisfaction survey does not request any identifying information from the respondent, and as a result, completed postcard surveys received by the PEC are completely anonymous.

4. Email and web-based contacts (Web Survey for E-Mail Publication Orders and the Web Survey for Internet Publications). Requestors who contact CDC-INFO via email or the Internet are provided with a link to the appropriate web-based customer satisfaction survey (see Attachments I, J, O, P, Q, and R). The web-based surveys are hosted by EMT, which will not ask for or capture any personal information from the respondent, or the respondent’s IP address. These procedures also apply to special outreach surveys (Special Event/Outreach Survey for the General Public and the Special Event/Outreach Survey for Professionals) (Attachments T, U, V, and W) and to the web-based Customer Service Representative Survey (Key Informant Survey) (Attachment S). Note that although all customer service representatives are invited to complete the CSR survey, there will be no way to know which CSRs participated because the responses will be anonymous.

5. For the special event (Special Event/Outreach Survey for the General Public and the Special Event/Outreach Survey for Professionals) and emergency response surveys ( Emergency Response Survey for the General Public, Level 1-4 Emergency and the Emergency Response Survey for Professionals, Level 1-4 Emergency), the caller’s name will not be sent to the PEC (Attachments Y and Z). These surveys may be recorded as a part of the call monitoring process conducted for operational purposes. CDC-INFO conducts call monitoring and records a randomly selected sample of calls. Callers are informed of the fact that their call may be recorded for quality control purposes (see the script in Attachment DD). Potential respondents will be reminded verbally by the Customer Service Representative prior to beginning the special event and emergency response surveys that their call may be monitored for quality control purposes. Callers can then decide to opt out of the survey if they wish.

Overview of privacy obligations by personnel involved

Pearson Government Solutions is the fulfillment contractor for customer calls and e-mails. In all cases, PGS is the customer’s initial point of contact with CDC-INFO. These operations are separate from the collection of evaluation data for CDC-INFO, except for PGS’s role in referring customers to the anonymous customer satisfaction surveys. PGS will not provide any potentially identifying information about callers/survey respondents to CDC, EMT/Program Evaluation Center, or other contractors affiliated with CDC-INFO.

The fulfillment contractor for publication requests will be responsible for including the appropriate customer satisfaction postcard survey in each package of publications mailed to a requestor. The fulfillment contractor will not provide any potentially identifying information about publication requests to the Program Evaluation Center.

Evaluation, Management and Training Associates, Inc. EMT is an independent program evaluation contractor (PEC) to support knowledge development and continual evaluation of the conduct and customer satisfaction of CDC-INFO. All PEC staff will take a pledge, administered by the PEC Program Manager, agreeing that all information provided by respondents will be treated in a confidential manner. Access to the evaluation data is restricted to PEC personnel. The President of the PEC signed a non-disclosure agreement with the CDC-INFO contractor guaranteeing that all data will be protected from unauthorized viewing, use and/or release. The agreement also specifies that electronic and hard copies of data are to be kept secure and that access to data be limited to only those staff directly working on the CDC-INFO evaluation. All evaluation data retained by EMT is anonymous. Any data collected in the process of contacting or receiving data from survey respondents is destroyed following entry of the survey data at the PEC.

Privacy protections pertaining to data storage and retrieval

Stored responses to evaluation surveys are non-identifiable. In the evaluation database, each response is identified by a unique, randomly-generated code number. The evaluation data cannot be linked back to the original customer inquiries to CDC-INFO.

Throughout the approved data collection period, all completed (anonymous) questionnaires will be stored in locked files at the Program Evaluation Center's offices and will be accessible only to staff directly involved in the project. Surveys will be retained by the Program Evaluation Center for a period of three years and then destroyed, unless otherwise directed by the CDC Federal Project Officer.

Privacy protections pertaining to the dissemination of evaluation results

The results of data collection will be reported in aggregate format only. Only reports with adequate cell sizes (e.g. 10 or more) will be released in order to prevent any individual responses from being reported.

Other information relevant to privacy protections for respondents

The proposed data collection will be conducted for the purpose of improving CDC-INFO program operations and customer service. Data will not be generalizable and will not be used for research purposes. Although IRB approval is not required for this type of data collection, interactions with respondents will incorporate standard ethical principles and procedures.

All responses to evaluation surveys are voluntary. Respondents are not required to participate in the evaluation surveys in order to obtain customer services from CDC-INFO.

All respondents signify their consent to participating in evaluation surveys. In some cases, consent is verbal, for example, when a respondent gives verbal consent related to the Emergency Response or Special Event/Outreach surveys, or verbally provides a code word for the live customer satisfaction follow-up survey (Follow Up Survey). In other cases, the respondent demonstrates his or her consent by taking a proactive action. Examples include manually clicking on a link to connect to a web-based survey (Satisfaction Survey for E-Mail Inquiries; the Web Survey for E-Mail Publication Orders; and the Web Survey for Internet Publications), key-punching in a numerical code that switches the respondent to the IVR survey (Satisfaction Survey for Callers), key-punching in their telephone number to enable call-back for the live follow-up survey (Follow Up Survey), and return-mailing the postcard survey (Postcard Survey for Bulk Mailing and the Postcard Survey for Individual Publications). Procedures for obtaining consent are included with each data collection instrument in the appendices.

All data collection instruments include the OMB control number and the burden statement, which inform respondents of the time commitment for participating and the authority for the data collection. The Burden Statement has been translated into Spanish for the Spanish instruments. The translation was back-translated and the back-translation compared to the original English version in order to ensure accuracy.

Privacy and confidentiality of general operations

Customer service representatives may be exposed to personal identifiers during routine CDC-INFO operations; however, these activities are separate and distinct from the proposed collection of evaluation data for CDC-INFO. Personal identifiers are only collected for purposes of providing mail (electronic or postal) responses, or in the event of a public health emergency (name and phone number). Personal information about individuals collected by CDC-INFO is limited to that which is legally authorized and necessary and is maintained in a manner which precludes unwarranted intrusions upon individual privacy. These are considered transitory and must be maintained for a period of 90 days, per NARA. CDC-INFO records are compliant with requirements of the National Archives and Records Administration (NARA).

Complete telephone numbers are not captured or recorded for routine calls to CDC-INFO. On a routine basis, the customer service representative only has access to the caller’s area code and exchange (first three digits of the number). The two exceptions are:

1. An emergency situation in which a caller confesses to a crime or makes a threat against the public. In this situation, the customer service representative would take direct action to capture the caller’s entire telephone number, which would then be provided to the appropriate law enforcement agency. This action would precede an invitation to participate in a customer satisfaction survey; thus, there would be no connection with the proposed data collection related to the evaluation of CDC-INFO.

2. Contacts with CDC-INFO that are monitored for quality control purposes. Callers and e-mail inquirers are provided an opening script (see Attachment DD) which alerts them that some calls/e-mails may be monitored for quality assurance. Calls which are monitored are recorded. A recorded message stating that the call may be monitored for quality control purposes is provided prior to the caller talking to a live person.

Callers and e-mail inquirers are provided an opening script (see Attachment DD) which alerts them that some calls/e-mails may be monitored for quality assurance. Calls which are monitored are recorded. A recorded message stating that the call may be monitored for quality control purposes is provided prior to the caller talking to a live person. Caller ID will only note the first 6 digits of the number (area code and first three digits of the phone number—called the “exchange”) unless the caller confesses to a crime or makes a threat against the public. In these instances the entire phone number is captured through direct action taken by the Customer Service Representative. This information is provided to the appropriate law enforcement agency. CDC-INFO is compliant with the Privacy Act and National Archives and Records Administration (NARA). Personal information about individuals collected by CDC-INFO is limited to that which is legally authorized and necessary and is maintained in a manner which precludes unwarranted intrusions upon individual privacy.

Personal identifiers are only collected for purposes of providing mail (electronic or postal) responses, or in the event of a public health emergency (name and phone number). These are considered transitory and must be maintained for a period of 90-days, per NARA.

Regarding the confidentiality of data from survey respondents, no personal information is collected. All survey responses are identified using a randomly-generated identification number. For callers who opt to participate in the live follow up survey (Follow Up Survey), the phone number is asked for but caller name is not. The phone number is collected via automated voice response and not by a live person. The phone number is not linked back to the caller’s audio record in any way. Following the collection of live follow-up telephone survey data, the phone number record is destroyed.

For all data collected, verbal consent will be obtained. Procedures for obtained consent are included with each data collection instrument in the appendices.

Several actions will be taken to ensure confidentiality of study participants.

• All data collected will be maintained in a safe and private manner. The Program Evaluation Center, Pearson Government Solutions and MCRB will conform to all requirements of the Privacy Act of 1974 under the System of Records, Alcohol, Drug Abuse and Mental Health Epidemiologic Data, HHS/SAMHSA/OA, #09-30-0036.

• When recording calls, Pearson Government Solutions will not send identifying information (i.e. name of respondent or address) to the PEC. For the emergency response survey (Emergency Response Survey—Level 1-4 for the General Public and the Emergency Response Survey—Level 1-4 for Professionals), the caller’s name will not be sent to the PEC. For the live follow-up survey (Follow Up Survey), only the phone number of those who provide consent and who key-punch in their telephone number will be provided to the PEC (the telephone number will not be captured by Pearson Government Solutions—potential respondents who consent to participate will be asked to key-punch in their telephone number).

• For the live follow-up survey (Follow Up Survey), no identifying information about the study will be provided until the person answering the telephone provides a code word. Nothing will be said about CDC or CDC-INFO or what CDC-INFO does until the potential respondent provides the correct code word. This step will prevent individuals who did not contact CDC-INFO from finding out that their partner, child, roommate, etc. contacted CDC-INFO.

• When extracting e-mails, Pearson Government Solutions will not send identifying information (i.e. name of respondent or address) to the Program Evaluation Center. When sending the link to participate in web-based surveys (Satisfaction Survey for E-Mail Inquiries; the Web Survey for E-Mail Publication Orders; and the Web Survey for Internet Publications), Pearson Government Solutions will not include any cookies which would capture personal information about the Internet inquirer. When participating in web-based surveys (Satisfaction Survey for E-Mail Inquiries; the Web Survey for E-Mail Publication Orders; and the Web Survey for Internet Publications), EMT (the Program Evaluation Center) will not capture or ask for any personal information from the respondent. IP addresses will NOT be captured.

• When mailing out postcards (Postcard Survey for Bulk Mailing and the Postcard Survey for Individual Publications), the fulfillment contractor will not include identifying information (i.e. name of respondent or address) to the Program Evaluation Center. The Program Evaluation Center will not ask for any identifying information on the postcard (Postcard Survey for Bulk Mailing and the Postcard Survey for Individual Publications).

• For the Customer Service Representative Survey (Key Informant Survey), all customer service representatives will be invited to participate in the survey. Because it is an anonymous web survey hosted by the Program Evaluation Contractor, there will be no way to tell who participates and who does not. The PEC will NOT ask for e-mail address or any identifying information. Nor is participation mandatory. No personally identifiable information will be collected or captured. Access to the actual data entered into the website will be restricted to the PEC.

• Access to the data will be limited to Program Evaluation Center staff directly involved in the evaluation. At the end of the contract a public use data diskette or CD-ROM will be made available, along with detailed documentation. These public use data files will contain no individual identifiers. Reports prepared by the Program Evaluation Center as contract deliverables will present data in aggregated form only.

• All Program Evaluation Center staff will take a pledge, administered by the Program Manager for the Program Evaluation Center, agreeing that all information provided by respondents will be accorded complete confidentiality.

Throughout data collection, all completed questionnaires will be stored in locked files at the Program Evaluation Center's offices and will be accessible only to staff directly involved in the project. Surveys will be retained by the Program Evaluation Center for a period of three years and then destroyed, unless otherwise directed by the CDC Federal Project Officer.

The data collection anticipated through this program evaluation does not fall under the Federal regulations regarding protection of human subjects in research. The data collection is for the purposes of program evaluation, designed to improve customer service. The proposed program evaluation will not generate generalizable findings. The proposed program evaluation is not research. In addition, regulations at the Centers for Disease Control and Prevention only require Institutional Review Board review when data will be collected from pregnant women, prisoners and children/youth under 18 years of age. Data will not be collected from any of these groups under the proposed program evaluation of CDC-INFO.

11. Justification for Sensitive Questions

The live follow-up (Follow Up Survey) and special outreach surveys (Special Event/Outreach Survey for the General Public) ask about health-related knowledge, attitudes and behaviors. While the responses are open-ended, a respondent may choose to report on sensitive topics such as sexual risk behaviors. By including open-ended questions in the instruments, we introduce the possibility of sensitive responses. These questions are necessary to obtain data regarding the potential impact of CDC-INFO.

CDC does not anticipate that the questions used in the special outreach (Special Event/Outreach Survey for the General Public) and follow-up surveys (Follow Up Survey) will cause discomfort to callers. Experience on the CDC’s evaluation of the HIV/STDs hotline confirms that this has not been a problem. Callers telephone the hotlines voluntarily, and routinely discuss issues of a far more sensitive nature. Callers are advised that they may refuse to answer any and all questions. No sensitive questions are asked on the brief interactive voice response satisfaction survey up (Satisfaction Survey for Callers), the postcard up (Postcard Survey for Bulk Mailing and the Postcard Survey for Individual Publications) or any of the web-based surveys up (Satisfaction Survey for E-Mail Inquiries; the Web Survey for E-Mail Publication Orders; and the Web Survey for Internet Publications).

The survey of customer service representatives up (Key Informant Survey) could be viewed as sensitive by some respondents, in that questions relate to job performance. Although potentially sensitive, these questions are justified because the evaluation process for CDC-INFO would be incomplete without the perspective of customer service representatives, who occupy a pivotal role. In addition, potential sensitivities are offset by the procedures that ensure that responses are not identifiable. In addition, participation is completely voluntary.

Consent is required from all respondents:

• Manual—by clicking on a link for Internet surveys ( Satisfaction Survey for E-Mail Inquires; the Web Survey for E-Mail Publication Orders; and the Web Survey for Internet Publications), key-punching in a number to indicate consent for the brief interactive voice response survey up (Satisfaction Survey for Callers), key-punching in a phone number via interactive voice response for the follow-up survey up (Follow Up Survey), and mailing the postcard survey (Postcard Survey for Bulk Mailing and the Postcard Survey for Individual Publications)

• Verbal—by verbally stating that they are willing to participate when asked by the Customer Service Representative conducting Emergency Response up (Emergency Response Survey—Level 1-4 Emergency—General Public and the Emergency Response Survey—Level 1-4 Emergency—Professionals) or Special Event surveys up (Special Event/Outreach Survey for the General Public and the Special Event/Outreach Survey for Professionals), and by giving the code word and verbally stating willingness to participate in the live follow-up telephone survey up (Follow Up Survey).

The OMB control number is displayed on all instruments (included in the appendix), as well as the Burden Statement. The Burden Statement has been translated into Spanish for the Spanish instruments. The translation was back-translated and the back-translation compared to the original English version in order to ensure accuracy.

12. Estimate of Annualized Burden Hours and Costs

Table A12. Estimated Annualized Burden Hours

Data collection instrument	Number of Respondents	Responses /Respondent	Average burden/ Response (in hrs)	Average An­nual Burden Hours
Satisfaction survey (callers)........................................................	25,000	1	3/60	1,250
Satisfaction survey (e-mail inquiries).........................................	330	1	3/60	17
Follow up survey........................................................................	3,125	1	7/60	365
Key informant survey.................................................................	100	1	7/60	12
Postcard survey for bulk mailing................................................	950	1	1/60	16
Postcard survey for individual publications……………….……	2,100	1	1/60	35
Web survey for e-mail publication orders………………………	1,000	1	1/60	17
Web survey for internet publications………………………….…	950	1	1/60	16
Special event/Outreach survey – General Public..........................	25,600	1	5/60	2,133
Special event/Outreach survey – Professionals………………….	10,400	1	5/60	867
Emergency response survey – Level 1 emergency – General Public............................................................................................	31,151	1	5/60	2,596
Emergency response survey – Level 1 emergency -- Professionals…..	7,459	1	5/60	622
Emergency response survey – Level 2 emergency – General Public……….	57,579	1	5/60	4798
Emergency response survey – Level 2 emergency—Professionals….	51,821	1	5/60	4318
Emergency response survey – Level 3 emergency – General Public……………………………………………………….…	351,863	1	5/60	29,322
Emergency response survey – Level 3 emergency – Professional….	316,678	1	5/60	26,390
Emergency response survey – Level 4 emergency – General Public……………………………………………………….…	645,630	1	5/60	53,803
Emergency response survey – Level 4 emergency – Professional……	596,504	1	5/60	49,709
Total Burden Hours.......................................................................	……………	……………	………….	176,283

Below is a table outlining the average annual cost to respondents for participating in data collection activities.

Table A12-2a. Estimated Annualized Burden Costs (Excluding Emergency Response)

Estimated Annualized Burden Hours	Number of Respondents	Responses/ Respondent	Hours/ Respondent	Hourly Wage Rate	Annual Respondent Costs1
Interactive Voice Response Survey up (Satisfaction Survey for Callers)	25,000	1	3/60	$5.15	$6,437.50
Live Follow-Up Telephone Survey (Follow Up Survey)	3,125	1	7/60	$5.15	$1,877.60
Web Survey for E-Mail Inquirers (Satisfaction Survey for E-Mail Inquiries)	330	1	3/60	$5.15	$84.98
Postcard Survey for Single Publication Orders (Postcard Survey for Individual Publications)	2,100	1	1/60	$5.15	$180.25
Postcard Survey for Bulk Mailing (Postcard Survey for Bulk Mailing)	950	1	1/60	$5.15	$81.54
Web Survey for Internet Publications (Web Survey for Internet Publications)	950	1	1/60	$5.15	$81.54
Web Survey for E-Mail Publication Orders (Web Survey for E-Mail Publication Orders)	1,000	1	1/60	$5.15	$85.83
Customer Service Representative survey (Key Informant Survey)	100	1	7/60	$10.57	$123.32
Special Outreach Surveys (General Public) (Special Event/Outreach Survey for the General Public)	25,600	1	5/60	$5.15	$10,986.67
Special Outreach Surveys (Professionals) (Special Event/Outreach Survey for Professionals)	10,400	1	5/60	$100.00	$86,666.67
Total Respondent Cost (Excluding Emergency Response)					$106,605.90

The minimum wage rate is included in this table to reflect what the average callers’ time would be worth when contacting the hotline. Since it is uncertain of the caller’s occupation, the minimum wage rate was used. The hourly rate used when calculating cost for the Customer Service Representative Survey is the prevailing hourly wage for health information workers. For health professionals, an hourly rate of $100 per hour was used in the calculations.

Below is a summary of the information contained in the two tables shown above and on the prior page:

• Interactive Voice Response Survey (Satisfaction Survey for Callers): The estimated number of persons to be surveyed per year is 25,000. The survey length is 3 minutes with a yearly burden of 1,250 hours. Given the minimum wage of $5.15 per hour, the annual cost to respondents to participate in the Interactive Voice Response Survey is $6,437.50.

• Live Follow-Up Telephone Survey (Follow Up Survey): The estimated number of persons participating in this survey each year is 3,125. The survey length is 7 minutes, with a yearly burden of 365 hours. Given the minimum wage of $5.15 per hour, the annual cost to respondents to participate in the Live Follow-Up Telephone Survey is $1,877.60.

• Web Survey for E-Mail Inquirers (Satisfaction Survey for E-Mail Inquiries): The estimated number of respondents to be surveyed each year is 330. The survey length is 3 minutes, for a total annual burden of 17 hours. Given the minimum wage of $5.15 per hour, the annual cost to respondents to participate in the Web Survey for E-Mail Inquirers is $84.98.

• Postcard Survey for Single Publication Orders (Postcard Survey for Individual Publications): The estimated number of respondents is 2,100 annually. The postcard survey length for completion is 1 minute with a yearly burden of 35 hours. Given the minimum wage of $5.15 per hour, the annual cost to respondents to participate in the Postcard Survey for Single Publication Orders is $180.25.

• Postcard Survey for Bulk Mailing (Postcard Survey for Bulk Mailing): The estimated number of respondents is 950. The postcard survey length for completion is 1 minute with a yearly burden of 16. Given the minimum wage of $5.15 per hour, the annual cost to respondents to participate in the Postcard Survey for Bulk Publication Orders is $81.54.

• Web Survey for Internet Publication Orders (Web Survey for Internet Publication Orders): The estimated number of respondents is 950 annually. The survey is 1 minute in length with a yearly burden of 16 hours. Given the minimum wage of $5.15 per hour, the annual cost to respondents to participate in the Web Survey for Internet Publication Orders is $81.54.

• Web Survey for E-Mail Publications (Web Survey for E-Mail Publications): The estimated number of respondents is 1,000 annually. The survey is 1 minute in length for a total annual burden of 17 hours. Given the minimum wage of $5.15 per hour, the annual cost to respondents to participate in the Web Survey for E-Mail Publications is $85.83.

• Customer Service Representative Survey (Key Informant Survey): The estimated number of respondents is 100 annually. The survey length for completion is 7 minutes. Customer Service Representatives will be asked to complete the survey on an annual. The total annual burden is 12 hours. Given the prevailing wage of $10.57 per hour for Customer Service Representatives in the health field, the annual cost to respondents to participate in the Customer Service Representative Survey is $123.32.

• Special Outreach Surveys (General Public) (Special Event/Outreach Survey for the General Public): The estimated number of respondents is 25,600. The survey length for completion is 5 minutes for a total annual burden of 2,133 hours. Given the minimum wage of $5.15 per hour, the annual cost to respondents to participate in the Special Outreach Surveys (General Public) is $10,986.67.

• Special Outreach Surveys (Professionals) (Special Event/Outreach Survey for Professionals): The estimated number of professional respondents is 10,400. The survey length for completion is 5 minutes for a total annual burden of 867 hours. Given the prevailing wage for health professionals (averaged across professions and geography) of $100.00 per hour, the annual cost to respondents to participate in the Special Outreach Surveys (Professionals) is $86,666.67.

Table A12-2b. Estimated Annualized Burden Costs (Emergency Response ONLY)

Estimated Annualized Burden Hours	Number of Respondents	Responses/ Respondent	Hours/ Respondent	Hourly Wage Rate	Annual Respondent Costs2
Emergency Response Surveys—Level 1 Emergency—General Public	31,151	1	5/60	$5.15	$13,368.97
Emergency Response Surveys—Level 2 Emergency—General Public	57,579	1	5/60	$5.15	$24,710.99
Emergency Response Surveys—Level 3 Emergency—General Public	351,863	1	5/60	$5.15	$151,007.87
Emergency Response Surveys—Level 4 Emergency—General Public	645,630	1	5/60	$5.15	$277,082.88
Emergency Response Surveys—Level 1 Emergency--Professional	7,459	1	5/60	$100.00	$62,158.33
Emergency Response Surveys—Level 2 Emergency--Professional	51,821	1	5/60	$100.00	$431,841.67
Emergency Response Surveys—Level 3 Emergency--Professional	316,678	1	5/60	$100.00	$2,638,983.33
Emergency Response Surveys—Level 4 Emergency--Professional	596,504	1	5/60	$100.00	$4,970,866.67
Total Respondent Cost (Emergency Response ONLY)					$8,570,020.70

• Emergency Response Surveys (General Public): For a Level 1 emergency, the estimated number of respondents is 31,151. The survey length for completion is 5 minutes for a total annual burden of 2,596 hours. Given the minimum wage of $5.15 per hour, the annual cost to respondents to participate in the Emergency Response Surveys (General Public) is $13,368.97.

• For a Level 2 emergency, the estimated number of respondents is 57,579. The survey length for completion is 5 minutes for a total annual burden of 4,798 hours. Given the minimum wage of $5.15 per hour, the annual cost to respondents to participate in the Emergency Response Surveys (General Public) is $24,710.99.

• For a Level 3 emergency, the estimated number of respondents is 351,863. The survey length for completion is 5 minutes for a total annual burden of 29,322 hours. Given the minimum wage of $5.15 per hour, the annual cost to respondents to participate in the Emergency Response Surveys (General Public) is $151,007.87.

• For a Level 4 emergency, the estimated number of respondents is 645,630. The survey length for completion is 5 minutes for a total annual burden of 53,803 hours. Given the minimum wage of $5.15 per hour, the annual cost to respondents to participate in the Emergency Response Surveys (General Public) is $277,082.88.

• Emergency Response Surveys (Professionals): For a Level 1 emergency, the estimated number of respondents is 7,459. The survey length for completion is 5 minutes for a total annual burden of 622 hours. Given the prevailing wage of $100.00 per hour, the annual cost to respondents to participate in the Emergency Response Surveys (Professionals) is $62,158.33.

• For a Level 2 emergency, the estimated number of respondents is 51,821. The survey length for completion is 5 minutes for a total annual burden of 4,318 hours. Given the prevailing wage of $100.00 per hour, the annual cost to respondents to participate in the Emergency Response Surveys (Professionals) is $431,841.67.

• For a Level 3 emergency, the estimated number of respondents is 316,678. The survey length for completion is 5 minutes for a total annual burden of 26,390 hours. Given the prevailing wage of $100.00 per hour, the annual cost to respondents to participate in the Emergency Response Surveys (Professionals) is $2,638,983.33.

• For a Level 4 emergency, the estimated number of respondents is 596,504. The survey length for completion is 5 minutes for a total annual burden of 49,709 hours. Given the prevailing wage of $100.00 per hour, the annual cost to respondents to participate in the Emergency Response Surveys (Professionals) is $4,970,866.67.

The total estimated annualized costs to the respondents, including emergencies and non-emergencies, is $8,676,626.60.

The table below shows the range of the total annual cost to respondents to participate in survey data collection for CDC-INFO, depending upon the size of the emergency.

Table A12-2c. Estimated Annualized Burden Costs

Estimated Annualized Burden Hours	Number of Respondents	Responses/ Respondent	Hours/ Respondent	Hourly Wage Rate	Annual Respondent Costs
Total Respondent Costs (NOT including Emergency Response)					$106,605.90
Emergency Response Surveys—Level 1 Emergency—General Public	31,151	1	5/60	$5.15	$13,368.97
Emergency Response Surveys—Level 1 Emergency--Professional	7,459	1	5/60	$100.00	$62,158.33
Total Respondent Costs Including All Non-Emergency Data Collection & Emergency Response Data Collection in the event of a Level 1 Emergency					$182,133.20
Total Respondent Costs (NOT including Emergency Response)					$106,605.90
Emergency Response Surveys—Level 2 Emergency—General Public	57,579	1	5/60	$5.15	$24,710.99
Emergency Response Surveys—Level 2 Emergency--Professionals	51,821	1	5/60	$100.00	$431,841.67
Total Respondent Costs Including All Non-Emergency Data Collection & Emergency Response Data Collection in the event of a Level 2 Emergency					$563,158.56
Total Respondent Costs (NOT including Emergency Response)					$106,605.90
Emergency Response Surveys—Level 3 Emergency—General Public	351,863	1	5/60	$5.15	$151,007.87
Emergency Response Surveys—Level 3 Emergency--Professionals	316,678	1	5/60	$100.00	$2,638,983.33
Total Respondent Costs Including All Non-Emergency Data Collection & Emergency Response Data Collection in the event of a Level 3 Emergency					$2,896,597.10

Total Respondent Costs (NOT including Emergency Response)					$106,605.90
Emergency Response Surveys—Level 4 Emergency—General Public	645,630	1	5/60	$5.15	$277,082.88
Emergency Response Surveys—Level 4 Emergency--Professionals	596,504	1	5/60	$100.00	$4,970,866.67
Total Respondent Costs Including All Non-Emergency Data Collection & Emergency Response Data Collection in the event of a Level 4 Emergency					$5,354,555.44

The total annual respondent costs range from $182,993.68 in the event of a Level 1 emergency to $5,354,555.44 in the event of a Level 4 emergency.

13. Estimates of Other Total Annual Cost Burden to Respondents or Recordkeepers

There are no additional costs to the respondents.

14. Annualized Cost to the Government

Table 14.1a. Estimated Annualized Cost to the Government (Excluding Emergency Response)

Type of Data	Estimated Persons to be Surveyed	Responses/ respondent	Time to Collect Data	Contractor annual burden hours (rounded to the nearest whole number)	Hourly Wage Rate	Annual Cost to Government to collect data3
Interactive Voice Response Survey (Satisfaction Survey for Callers)	25,000	1	3/60	1,250	-----------	------------
Live Follow-Up Telephone Survey (Follow Up Survey)	3,125	1	7/60	365	$14.00	$5,104.17
Web Survey for E-Mail Inquirers Survey (Satisfaction Survey for E-Mail Inquiries)	330	1	3/60	17	-----------	------------
Postcard Survey for Single Publication Orders Survey (Postcard Survey for Individual Publications)	2,100	1	1/60	35	$5.15	$180.25
Postcard Survey for Bulk Mailing	950	1	1/60	16	$5.15	$81.54
Web Survey for Internet Publication Orders	950	1	1/60	16	-----------	-------------
Web Survey for E-Mailed Publications	1,000	1	1/60	17	-----------	-------------
Customer Service Representative Survey (Key Informant Survey)	100	1	7/60	12	$10.57	$123.32
Special Event/Outreach Surveys (General Public)	25,600	1	5/60	2,133	$10.57	$22,549.33
Special Event/ Outreach Surveys (Professionals)	10,400	1	5/60	867	$10.57	$9,160.67
Total Cost						$37,199.28

The cost of this survey to the CDC will be indirect, but tangible. Costs are absorbed within the service contract.

Below is a summary of the information contained in the table shown above and on the prior page:

• Interactive Voice Response Survey (Customer Satisfaction Survey for Callers): There is no cost to the government for data collection because the survey is completely automated.

• Live Follow Up Telephone Survey (Follow Up Survey): The estimated number of persons participating in this survey each year is 3,125. The survey length is 7 minutes, with a yearly burden of 365 hours. Given the prevailing wage of $14.00 per hour for data collectors, the annual cost to the government to collect data using the Live Follow Up Telephone Survey is $5,104.17.

• Web Survey for E-Mail Inquirers (Customer Satisfaction Survey for E-Mail Inquiries): There is no cost to the government for data collection because the survey is completely automated.

• Postcard Survey for Single Publication Orders (Postcard Survey for Individual Publication Orders): The estimated number of respondents is 2,100 annually. The amount of time estimated to place a postcard in every publication is 1 minute with a yearly burden of 35 hours. Given the minimum wage of $5.15 per hour for MCRB employees who will place the postcard in the publication order, the annual cost to the government for collecting data using the Postcard Survey for Single Publication Orders is $180.25.

• Postcard Survey for Bulk Mailing: The estimated number of respondents is 950 annually. The amount of time estimated to place a postcard in every publication is 1 minute with a yearly burden of 16 hours. Given the minimum wage of $5.15 per hour for MCRB employees who will place the postcard in the publication order, the annual cost to the government for collecting data using the Postcard Survey for Bulk Mailing is $81.54.

• Web Survey for Internet Publication Orders: There is no cost to the government for data collection because the survey is completely automated.

• Web Survey for E-Mailed Publications: There is no cost to the government for data collection because the survey is completely automated.

• Customer Service Representative Survey (Key Informant Survey): The estimated number of respondents is 100 annually. The survey length for completion is 7 minutes. Customer Service Representatives will be asked to complete the survey on an annual basis. The total annual burden is 12 hours. Given the prevailing wage of $10.57 per hour for Customer Service Representatives in the health field, the annual cost to the government for collecting data using the Customer Service Representative Survey is $123.32.

• Special Event/Outreach Surveys (General Public): The estimated number of respondents is 25,600. The survey length for completion is 5 minutes for a total annual burden of 2,133 hours. Given the prevailing wage of $10.57 per hour for Customer Service Representatives in the health field, the annual cost to the government for collecting data using Special Outreach Surveys (General Public) is $22,549.33.

• Special Event/Outreach Surveys (Professionals): The estimated number of professional respondents is 10,400. The survey length for completion is 5 minutes for a total annual burden of 867 hours. Given the prevailing wage of $10.57 per hour for Customer Service Representatives in the health field, the annual cost to the government for collecting data using the Special Outreach Surveys (Professionals) is $9,160.67.

Table 14.1b. Estimated Annualized Cost to the Government (Emergency Response ONLY)

Type of Data	Estimated Persons to be Surveyed	Responses/ Respondent	Time to Collect Data	Contractor annual burden hours (rounded to the nearest whole number)	Hourly Wage Rate	Annual Cost to Government to collect data
Emergency Response Surveys—/Level 1 Emergency—General Public	31,151	1	5/60	2,596	$10.57	$27,438.84
Emergency Response Surveys—Level 2 Emergency—General Public	57,579	1	5/60	4,798	$10.57	$50,717.50
Emergency Response Surveys—Level 3 Emergency—General Public	351,863	1	5/60	29,322	$10.57	$309,932.66
Emergency Response Surveys—Level 4 Emergency—General Public	645,630	1	5/60	53,803	$10.57	$568,692.43
Emergency Response Surveys—Level 1 Emergency--Professional	7,459	1	5/60	622	$10.57	$6,570.14
Emergency Response Surveys—/Level 2 Emergency—Professional	51,821	1	5/60	4,318	$10.57	$45,645.66
Emergency Response Surveys—Level 3 Emergency—Professional	316,678	1	5/60	26,390	$10.57	$278,940.54
Emergency Response Surveys—Level 4 Emergency—Professional	596,504	1	5/60	49,709	$10.57	$525,420.61

• Emergency Response Surveys (General Public): For a Level 1 emergency, the estimated number of respondents is 31,151. The survey length for completion is 5 minutes for a total annual burden of 2,763 hours. Given the prevailing wage of $10.57 per hour for Customer Service Representatives in the health field, the annual cost to the government to sponsor the Emergency Response Surveys (General Public) is $27,438.84

• For a Level 2 emergency, the estimated number of respondents is 57,579. The survey length for completion is 5 minutes for a total annual burden of 4,798 hours.

Given the prevailing wage of $10.57 per hour for Customer Service Representatives in the health field, the annual cost to the government to sponsor the Emergency Response Surveys (General Public) is $50,717.50.

• For a Level 3 emergency, the estimated number of respondents is 351,863. The survey length for completion is 5 minutes for a total annual burden of 29,322 hours. Given the prevailing wage of $10.57 per hour for Customer Service Representatives in the health field, the annual cost to the government to sponsor the Emergency Response Surveys (General Public) is $309,932.66.

• For a Level 4 emergency, the estimated number of respondents is 645,630. The survey length for completion is 5 minutes for a total annual burden of 53,803 hours. Given the prevailing wage of $10.57 per hour for Customer Service Representatives in the health field, the annual cost to the government to sponsor the Emergency Response Surveys (General Public) is $568,692.43.

• Emergency Response Surveys (Professionals): For a Level 1 emergency, the estimated number of respondents is 7,459. The survey length for completion is 5 minutes for a total annual burden of 622 hours. Given the prevailing wage of $10.57 per hour for Customer Service Representatives in the health field, the annual cost to the government to sponsor the Emergency Response Surveys (Professionals) is $6,570.14.

• For a Level 2 emergency, the estimated number of respondents is 51,821. The survey length for completion is 5 minutes for a total annual burden of 4,318 hours. Given the prevailing wage of $10.57 per hour for Customer Service Representatives in the health field, the annual cost to the government to sponsor the Emergency Response Surveys (Professionals) is $45,645.66.

• For a Level 3 emergency, the estimated number of respondents is 316,678. The survey length for completion is 5 minutes for a total annual burden of 26,390 hours. Given the prevailing wage of $10.57 per hour for Customer Service Representatives in the health field, the annual cost to the government to sponsor the Emergency Response Surveys (Professionals) is $278,940.54.

• For a Level 4 emergency, the estimated number of respondents is 596,504. The survey length for completion is 5 minutes for a total annual burden of 49,709 hours. Given the prevailing wage of $10.57 per hour for Customer Service Representatives in the health field, the annual cost to the government to sponsor the Emergency Response Surveys (Professionals) is $525,420.61.

The table below shows the range of the total annual cost to the government to sponsor survey data collection for CDC-INFO, depending upon the size of the emergency.

Table 14.1c. Estimated Annualized Cost to the Government

Type of Data	Estimated Persons to be Surveyed	Responses/ Respondent	Time to Collect Data	Contractor annual burden hours (rounded to the nearest whole number)	Hourly Wage Rate	Annual Cost to Government to collect data
Total Government Costs (NOT including Emergency Response)						$37,199.28
Emergency Response Surveys—Level 1 Emergency—General Public	31,151	1	5/60	2,763	$10.57	$27,438.84
Emergency Response Surveys--Level 1 Emergency—Professionals	7,459	1	5/60	622	$10.57	$6,570.14
Total Government Costs Including All Non-Emergency Data Collection & Emergency Response Data Collection in the event of a Level 1 Emergency						$71,208.26
Total Government Costs (NOT including Emergency Response)						$37,199.28
Emergency Response Surveys—Level 2 Emergency—General Public	57,579	1	5/60	4,798	$10.57	$50,717.50
Emergency Response Surveys—Level 2 Emergency--Professionals	51,821	1	5/60	4,318	$10.57	$45,645.66
Total Government Costs Including All Non-Emergency Data Collection & Emergency Response Data Collection in the event of a Level 2 Emergency						$133,562.44
Total Government Costs (NOT including Emergency Response)						$37,199.28
Emergency Response Surveys—Level 3 Emergency—General Public	351,863	1	5/60	29,322	$10.57	$309,932.66
Emergency Response Surveys—Level 3 Emergency--Professionals	316,678	1	5/60	26,390	$10.57	$278,940.54
Total Government Costs Including All Non-Emergency Data Collection & Emergency Response Data Collection in the event of a Level 3 Emergency						$626,072.48
Total Government Costs (NOT including Emergency Response)						$37,199.28
Emergency Response Surveys—Level 4 Emergency—General Public	645,630	1	5/60	53,803	$10.57	$568,692.43

Emergency Response Surveys—Level 4 Emergency--Professionals	596,504	1	5/60	49,709	$10.57	$525,420.61
Total Government Costs Including All Non-Emergency Data Collection & Emergency Response Data Collection in the event of a Level 4 Emergency						$1,131,312.32

Other direct expenses for one year are based on 50% of the evaluation contract or $689,260. This amount does not include the cost for the Program Evaluation Contractor (PEC) to collect data as described in table Table 14.1a. The additional cost to the PEC of $689,260 represents the cost of processing measures collected electronically and by CDC-INFO staff. In addition, this cost encompasses analysis of the results and reporting of the results.

In addition, the annual cost to the government for CDC Program Officer overseeing the data collection effort is $30,000.

When the dollar amounts in the table above are added to the additional cost to the PEC and CDC Program Officer involvement, the total annual cost to the government to collect survey data for CDC-INFO is $790,468.26 in the event of a Level 1 emergency, to $1,850,572.32 in the event of a Level 4 emergency.

15. Explanation for Program Changes or Adjustments

This is a new data collection.

16. Plans for Tabulation and Publication and Project Time Schedule

Data collection will take place continuously throughout the three years approved by OMB. OMB renewal will be requested at the end of the first three years, in order to collect the data for the remaining three years of the program. Analyses will consist primarily of frequencies and descriptive statistics. The contact center data will be stratified by various factors: age, race and ethnicity. The data collected will be used by CDC staff for conferences, presentations, and publications as well as for evaluation purposes.

Project Time Schedule

Activity	Time Schedule
Data Collection	Immediately after OMB approval and conducted continuously thereafter
Monthly Report	Monthly after OMB Approval
Annual Report	3 months after end of the calendar year
Presentations at Conferences	Ongoing

17. Reason(s) Display of OMB Expiration Date is Inappropriate

Exemption is not being sought.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to certification.

List of Attachments

Attachment A: Federal Register Notice

Attachment B: CDC Program Transition Timeline

Attachment C: Public Health Service Act (42 USC 241) Section 301

Attachment D: Citizen Service Levels Interagency Committee (CSLIC) Report

Attachment E: Brief Interactive Voice Response Survey (English) (Customer Satisfaction Survey for Callers)

Attachment F: Brief Interactive Voice Response Survey (Spanish) (Customer Satisfaction Survey for Callers)

Attachment G: Live Follow-up Telephone Survey (English) (Follow Up Survey)

Attachment H: Live Follow-up Telephone Survey (Spanish) (Follow Up Survey)

Attachment I: Web Survey for E-Mail Inquirers (English) (Customer Satisfaction Survey for E-Mail Inquires)

Attachment J: Web Survey for E-Mail Inquirers (Spanish) (Customer Satisfaction Survey for E-Mail Inquiries)

Attachment K: Postcard Survey for Single Publication Orders (English) (Postcard Survey for Individual Publication Orders)

Attachment L: Postcard Survey for Single Publication Orders (Spanish) (Postcard Survey for Individual Publication Orders)

Attachment M: Postcard Survey for Bulk Mailing (English)

Attachment N: Postcard Survey for Bulk Mailing (Spanish)

Attachment O: Web Survey for Internet Publication Orders (English)

Attachment P: Web Survey for Internet Publication Orders (Spanish)

Attachment Q: Web Survey for E-Mailed Publications (English)

Attachment R: Web Survey for E-Mailed Publications (Spanish)

Attachment S: Customer Service Representative Survey (Key Informant Survey)

Attachment T: Special Event/Outreach Survey (General Public) (English)

Attachment U: Special Event/Outreach Survey (General Public) (Spanish)

Attachment V: Special Event/Outreach Survey (Professionals) (English)

Attachment W: Special Event/Outreach Survey (Professionals) (Spanish)

Attachment X: Emergency Response Survey (General Public) (English)

Attachment Y: Emergency Response Survey (General Public) (Spanish)

Attachment Z: Emergency Response Survey (Professionals) (English)

Attachment AA: Emergency Response Survey (Professionals) (Spanish)

Attachment BB: Historical Evolution of Consumer Response Services Centers

Attachment CC: List of Technologies Used

Attachment DD: “This call may be monitored for quality assurance purposes”

Attachment EE: List of Federal Contacts

References

1 When calculating the total cost to collect data using each of the proposed surveys, the fraction was not rounded until the very last step in the calculation. Taking the Live Follow-Up Telephone Survey as an example, 3,125 respondents x 7/60 = 364.5833 burden hours x $5.15=$1,877.60. If you round at any step earlier in the process you will come up with different costs than what is laid out in the table above.

2 When calculating the total cost to collect data using each of the proposed surveys, the fraction was not rounded until the very last step in the calculation. Taking the Live Follow-Up Telephone Survey as an example, 3,125 respondents x 7/60 = 364.5833 burden hours x $5.15=$1,877.60. If you round at any step earlier in the process you will come up with different costs than what is laid out in the table above.

3 When calculating the total cost to collect data using each of the proposed surveys, the fraction was not rounded until the very last step in the calculation. Taking the Live Follow-Up Telephone Survey as an example, 3,125 respondents x 7/60 = 364.5833 burden hours x $5.15=$1,877.60. If you round at any step earlier in the process you will come up with different costs than what is laid out in the table above.