Memo

4

MEMORANDUM

SUBJECT: Customer Satisfaction Survey: Training and Guidance

EPA OEI Quality Staff Submission: Under EPA Information Collection Request #1711.04, OMB No 2090-0019

FROM: Reggie Cheatham

Director, Quality Staff (2811R)

THRU: Patricia Bonner, Customer Service Director

Office of Policy, Economics, and Innovation (1807T)

TO: Robert Garnett, Desk Officer

Office of Information Collection (2822T)

U.S. EPA’s Office of Environmental Information (OEI) Quality Staff will conduct a Customer Satisfaction Survey to determine how well the Quality System training and guidance meets customer needs and expectations. As stated in EPA Order 5360.1, EPA uses its Quality System to manage the quality of its environmental data collection, generation, and use. The primary goal of the Quality System is to ensure that our environmental data are of sufficient quantity and quality to support the data's intended use. Under the EPA Quality System, EPA organizations develop and implement supporting quality systems. Similar specifications are applied to contractors and may be applied to grantees and other recipients of financial assistance from EPA.

As part of its oversight of the EPA Agency-wide Quality System, the Quality Staff develops training to assist organizations in developing and implementing suitable quality systems. Training is made available to EPA staff and management, as well as a diverse stakeholder community, at two large Quality Staff-hosted events per year. One is a Training Event and the other is an Annual Quality Conference at which time is devoted to training. These events provide a training venue, information exchange, and networking opportunities which enhance and support the effectiveness of Quality Staff-developed customizable training materials for local use.

Quality Staff assistance to organizations in quality system implementation also encompasses development of guidance documents on a range of topics, including: Overview of the Quality System, Data Quality Objectives Process, Decision Error Feasibility Trials, Quality Assurance Project Plans (General, Geospatial, and Modeling), Sampling Designs for Environmental Data Collection, Preparing Standard Operating Procedures, and others.

The EPA Order 5360.1A2 contains a requirement to perform surveys to facilitate implementation of Agency-wide Quality Systems policy. This survey requirement demonstrates Agency commitment to regularly evaluate the extent to which current training and guidance satisfy the customers consistently held to Quality System implementation. OEI Quality Staff summarizes the findings and makes recommendations to improve services and meet future needs more effectively. The Quality Staff last conducted a survey in the spring of 2003.

This Information Collection Request (ICR) will provide access to entire groups of recognized customers previously not included due to Paperwork Reduction Act restrictions: State, Local, and Tribal Governments in addition to Contractors and Grantees. This survey conforms to the Customer Service ICR guidelines. The Quality Staff will use this ICR as a benchmark for periodic assessment of customer satisfaction. This survey will be repeated every three years to ensure Quality Staff produced training and guidance consistently meets customer expectations and continues to achieve high, measured levels of customer satisfaction.

The survey will be administered electronically through a contractor Web site. We will begin the process by making the survey notification to the previously identified EPA quality community through electronic mail. This distribution method will reach the desired community without unduly burdening the electronic mail system with mass-mailing. This method will also reduce the likelihood that potential respondents receive duplicate e-mails. The survey will be available for two weeks. To enhance awareness and participation, pre-notification of survey availability will be provided to management at meetings and through other official channels, on the three Quality Community teleconferences, and via e-mail to the Quality Community Contact list.

The survey should take approximately eight minutes which amounts to a burden of 147 hours every three years, or 43 hours per year. All responses will be stored in a database, which will be used to track and analyze the feedback over time. Feedback collection, analysis, and reporting will require approximately 40 hours of staff time. The resources invested in implementing recommendations will vary annually depending on results.

Approximately 1,100 participants have been identified to receive the survey. Initially, an exceptional response rate of 50% is predicted, providing 550 responses, as much of this customer base has previously lacked direct means to provide feedback on their level of satisfaction with Quality Staff training and guidance products. In subsequent efforts, as the submission becomes routine, a response level between 25‑40% is likely.

Previous survey results and training evaluation summaries indicate that a significant portion of the EPA community does not have adequate resources to address quality systems training and primarily relies on Quality Staff training. This area will not be addressed in the survey but is reasonable to assume will apply to all customers. The previous collection efforts identified within the overall EPA Quality Community were limited in scope and topic. Most efforts focused on achieved training levels of EPA employees as related to Quality Systems. These were targeted for training needs assessment in specific offices and divisions. Historically, the Quality Staff conducted internal (Federal employees only) training-needs assessment surveys. The survey results were intended to refine training and guidance under development and to guide the process of determining new development activities.

Lack of responsiveness and benchmarking make previous efforts within the EPA Quality Community and by EPA Quality Staff useful only as short-term needs assessments. Information collection from the expanded customer group will substantively improve results allowing for benchmarking and longer-term planning to track and improve customer satisfaction levels. Other government agency collections will not serve the purpose for which this collection is designed as the satisfaction of this particular customer base with EPA Quality Staff products is not a subject likely to be addressed by other agencies.

A statistical analysis report will be performed on the survey results. The report will contain a description of the survey, survey results, conclusions, numerical and graphic data summaries, a description of the survey design, summary of potential bias in survey results, description of significance testing and sampling errors, and the effect of non-responses as applicable.

The survey report will be provided to the Quality Staff. This report may be shared within the EPA QA Community – overall summary statistics may be published on the Quality System Web site. Individual responses will be provided only to the Contract Officer Representative as part of the contract deliverables.

This collection is voluntary. There is no requirement to provide this information to gain/qualify for a Right/Privilege/Benefit. This survey will not collect social security numbers or other personal identifiers and does not generate a requirement for system of records.

As the Quality Staff better defines the product design and specifications most likely to satisfy the customer, the burden on other organizations may be reduced. Identification of subgroup satisfaction levels will be shared with Quality Community groups in an effort to reduce the need for similar collections from EPA Regions or Program offices and enhance responsiveness of the Quality Community.

Please contact me at (202) 564-7713 or Katherine Breidenstine at (202) 564-1511 with any further questions and/or concerns. This memorandum transmits the customer satisfaction survey conducted by and funded by the Office of Environmental Information, Quality Staff for the Agency Quality System. The survey will be conducted in October 2006. The attachments contain the format and plan for this survey including planned statistical analysis, and planned use of the resulting information.

I appreciate the cooperation and assistance of Patricia Bonner, OPEI in facilitating this review. Her cooperation and assistance was key to our successful completion of this Customer Satisfaction Survey.

Attachment

EPA Quality System Customer Satisfaction Survey (Text Version of Web-based survey)

EPA Quality System Customer Satisfaction Survey (Web-based survey screen shots)