May 5, 2006
SUPPORTING STATEMENT
INFORMATION COLLECTION REQUEST
RENEWAL
for
CUSTOMER SATISFACTION SURVEYS
of
The United States Environmental Protection Agency
ICR 1711.05
OMB No 2090-0019
Table of Contents
Title Page
1. Identification of the Information Collection 1
1(a) Title of the Information Collection 1
1(b) Short Characterization/Abstract 1
2. Need for and Use of the Collection 1
2(a) Need/Authority for the Collection 1
2(b) Practical Utility/Users of the Data 3
3. Non-duplication, Consultations, and Other Collection Criteria 4
3(a) Non-duplication 4
3(b) Public Notice Required Prior to ICR Submission to OMB 5
3(c) Consultations 5
3(d) Effects of Less Frequent Collection 5
3(e) General Guidelines 5
3(f) Confidentiality 5
3(g) Sensitive Questions 5
4. The Respondents and The Information Requested 5
4(a) Respondents/SIC Codes 5
4(b) Information Requested 5
(I) Data items, including record keeping requirements 5
(II) Respondent Activities 6
5. The Information Collected--Agency Activities, Collection Methodology,
and Information Management 8
5(a) Agency Activities 8
5(b) Collection Methodology and Management 8
5(c) Small Entity Flexibility 10
5(d) Collection Schedule 10
6. Estimating the Burden and Cost of the Collection 10
6(a) Estimating Respondent Burden 10
6(b) Estimating Respondent Costs 12
(I) Labor Costs 12
(II) Capital and Operations and Maintenance Costs 12
(III) Capital/Start-up vs. Operations and Maintenance (O&M) Costs 12
(IV) Annualizing Capital Costs 12
6(c) Estimating Agency Burden and Cost 12
6(d) Estimating the Respondent Universe and Total Burden and Costs 13
6(e) Bottom Line Burden Hours and Cost Tables 15
(I) Respondent Tally 15
(II) The Agency Tally 15
(III) Variations in the Annual Bottom Line 15
(IV) Reasons for Change in Burden 15
(V) Burden Statement 16
Table of Contents (continued)
Title Page
List of Tables
Table 5-1 - Projected Use of Surveys 2006 - 2008 10
Table 6-1 - Burden Table 2006-2008 11
Table 6-2 - Respondent Universe, Total Burden and Costs 13
Table 6-3 - Agency Burden/Cost for Telephone Surveys 13
Table 6-4 - Agency Burden/Cost for Mail Surveys 14
Table 6-5 - Agency Burden/Cost for Customer Feedback Forms/Internet Screens 14
Table 6-6 - Agency Burden/Cost for Focus Groups 15
Table 6-7 -Aggregate Agency Table for Annual Burden/Cost 16
Table 6-8 – Aggregate EPA and Respondent Costs 16
Exhibits - Samples of OMB - Approved EPA Survey Instruments
Exhibit 1 – Telephone Survey Questionnaire 18
Exhibit 2 – NRMRL Feedback Cards, Evaluation Forms, Evaluation Surveys 19
U.S. ENVIRONMENTAL PROTECTION AGENCY
VOLUNTARY CUSTOMER SATISFACTION SURVEYS
TO IMPLEMENT
EXECUTIVE ORDER (E.O.) 12862
1. Identification of the Information Collection
1(a) Title of the Information Collection: Voluntary Customer Satisfaction Surveys
1(b) Short Characterization/Abstract
In accordance with Executive Order 12862, the Environmental Protection Agency is seeking from Office of Management and Budget (OMB) renewal of its generic clearance (OMB Control No. 2090-0019, expiring 06/30/06) for a period of three years. The clearance will be used to conduct two types of customer satisfaction surveys: “qualitative” surveys for identifying customer perceptions for expectations through focus groups or laboratory evaluations; and “quantitative” surveys for establishing general attitudes of EPA customers through a statistical sampling of customers. A customer, as described in E.O. 12862, is considered to be “...an individual or entity who is directly served by a department of an Agency.”
By seeking renewal of the generic clearance for customer surveys, EPA will have the flexibility to gather the views of our customers to better determine the extent to which our services, products and processes satisfy their needs or need to be improved. The generic clearance will speed the review and approval of customer surveys that solicit opinions from EPA customers on a voluntary basis, and do not involve “fact-finding” for the purposes of regulatory development or enforcement.
EPA sponsoring organizations seeking approval to conduct a customer survey will continue to submit their survey instruments with a brief description to the customer service staff in the Office of Policy, Economics and Innovation within the Office of the Administrator, for a screening/assistance review of the questions. Following review, endorsed survey packages will be sent to EPA’s Information Collections Division within the Office of Environmental Information and then to OMB. OMB will continue to review submissions for compliance with the Paperwork Reduction Act on an expedited schedule. EPA will provide OMB an annual summary of surveys conducted in accordance with OMB’s Resource Manual for Customer Surveys (dated October 1993). The EPA estimates that a combination of customer satisfaction surveys (mail, telephone, feedback forms and Internet) and focus group studies will request voluntary responses from approximately 18,735 respondents for an estimated burden of 1,671 hours over the three-year period.
2. Need for and Use of the Collection
2a. Need/Authority for the Collection
Executive Order 12862, dated September 11, 1993, calls upon agencies to take the following actions:
(a) identify the customers who are, or should be, served by the agency;
(b) survey customers to determine the kind and quality of services they want and their level of satisfaction with existing services;
(c) post service standards and measure results against them;
(d) benchmark customer service performance against the best in business;
(e) survey front-line employees on barriers to, and ideas for, matching the best in business;
(f) provide customers with choices in both the sources of service and the means of delivery;
(g) make information, services, and complaint systems easily accessible, and
(h) provide means to address customer complaints.
A March 1995 Presidential memo called upon federal agencies to enhance their customer service improvement efforts. A March 1998 Presidential memo underscored the continuing need to improve customer service and directed agencies to provide expanded opportunities for customers to communicate their needs and expectations. The Governmental Performance and Results Act of 1993 requires that agencies gather and use customer feedback. Finally, President Bush’s Management Agenda underscores the need for citizen-centered service delivery, increased satisfaction with government services, and the ability to prove government is doing a better job through measuring outcomes.
Using OMB’s Resource Manual for Customer Surveys (dated October 1993), which outlines the steps an Agency must take to obtain a generic clearance for Customer Satisfaction Surveys, and provides guidance on obtaining quality survey results, EPA developed its 1997 1999, and 2002 generic information collection requests to enable staff across the Agency to continue sponsoring customer satisfaction surveys. To reflect the Terms of Clearance for the 1997 ICR, Customer Service Program (CSP) staff developed, distributed and posted on the CSP web site a fact sheet clearly stating the restrictions on the use of this clearance. Efforts were validated when the 2000 and 2003 Terms of Clearance supported our efforts to improve screening, encouraging staff to consult with the CSP staff.
During the past three years, EPA has worked cooperatively with OMB to clear survey instruments. CSP staff has also advised many individuals and their contractors that their survey designs could not fit under this ICR. CSP staff worked with others to develop surveys to assist them in gathering information that could serve at least part of their needs through this ICR. If CSP staff could not work with regional and program staff to modify questions to fit the ICR and satisfy the needs of staff and their managers, we rejected their use of the ICR. Our goal has been to ensure that the surveys submitted under this ICR clearly meet the Terms of Clearance that OMB set out when approving it:
“As stated in OMB's 1999 terms of clearance: "The generic ICR is approved to allow the expedited OMB clearance of EPA customer satisfaction surveys that are simple, straightforward, and narrowly focused to:
1. current or former customers of EPA products or services;
2. the level of satisfaction with an actual service or product provided by EPA that they have utilized; and,
3. their recommendations for improving said product or service."
Surveys that target these elements and are submitted to OMB in accordance with this ICR will be reviewed by OMB within 20 working days. EPA shall provide OMB with an annual report outlining the use of this generic clearance, including the number of surveys, the burden imposed, and a brief description of their purposes (a condition of previous clearances). OMB encourages agency staff to consult with EPA's Customer Service Program (CSP) for advice, survey evaluation, and clearance assistance. OMB reserves the authority to disapprove any individual survey that does not meet the conditions outlined in this ICR. This generic clearance does not extend to "fact finding" for the purpose of regulatory development or enforcement. OMB is relying in large part on EPA's internal review and quality control to develop useful customer information. Finally, this generic ICR approval does not, and is not intended to, cover all types of surveys that EPA may wish to do relating to customer satisfaction -- only the narrow range of surveys discussed above. Surveys that do not meet the terms of clearance for this expedited clearance process may be entirely valid and appropriate surveys, but they should be submitted under the normal PRA clearance process. The agency is required to display the OMB control number and inform respondents of its legal significance (see 5 CFR 1320.5(b)).”
To fulfill its broad mandate of protecting human health and the environment, EPA provides a wide variety of voluntary public services ranging from information clearinghouses to educational programs and emergency hot lines. Corresponding to this broad range of services is a diverse universe of EPA customers, loosely defined by E.O. 12862 as “...an individual or entity who is directly served by a department or agency.”
EPA expands this definition to include customers who chose not to participate in an EPA service function, such as persons who were provided the opportunity but did not comment on a permit, participate in a community meeting, join a partnership program, etc. Learning perceptions of our services from those who select not to use them may also assist the Agency in its service innovation efforts. As we continue to redesign our processes and practices, we will be asking customers who use our current services what, from their perspectives, would be the most useful improvements.
Because Agency services and customers are so diverse, the Agency is requesting a generic clearance that will maximize flexibility in the methods used to fulfill the requirements for the Executive Order and expedite OMB review and clearance process of customer satisfaction surveys. EPA maintains a central repository of surveys submitted to OMB in the Regulatory Information Division. In addition, EPA developed a summary of the surveys and collected the analytical reports produced. The CSP staff has shared the findings, analysis and “success stories” following the conduct of surveys so this information can benefit those planning future surveys.
OPEI staff assigned to oversee the use of this ICR will continue to be a resource to individuals considering the development of customer satisfaction measurement programs within their organizations, explaining what the customer satisfaction ICR does and does not cover and how to make the best use of it. EPA ended its National Customer Service Program in 2003. Since then, the former CSP director (Patricia Bonner) has continued to screen and process survey submittals under this ICR. In 2005, a second OPEI staff member (Michelle Mandolia) has been preparing to assume management of all submittals under the ICR and was responsible for development of this renewal request.
2(b) Practical Utility/Uses of the Data
Customer service standards (http://www.epa.gov/customerservice/) provide a basis for performance measurement systems to determine our success at reaching customers, and provide the necessary framework for a management role in the development and use of the survey results. Information gathered from these surveys will continue to assist EPA to build and validate measurement systems. Survey results may be used to identify:
1) service needs and expectations of EPA customers;
2) strengths and weaknesses of EPA services;
3) ideas or suggestions for improvement of EPA services from its customers;
4) barriers to achieving customer service standards; and
5) needed changes to customer service standards.
While the information will not be used for regulatory development, the results of customer surveys could lead to reallocations of resources, revisions in certain Agency processes and policies, and development of guidance related to EPA’s customer services. Ultimately, these changes could result in improvements in services, products and processes the Agency provides to the public, and in turn, the public perception of the Agency.
To ensure proper design of EPA customer feedback and customer satisfaction measurement activities, increase the use and application of customer feedback, and build internal capacity to carry out these activities, the CSP coordinated development of “Hearing the Voice of the Customer - Customer Feedback and Customer Satisfaction Measurement Guidelines.” The “Feedback Guidelines” were first published in November 1998. The CSP has sponsored training workshops on the application of the Guidelines.
A five-stage model for feedback: Plan, Construct, Conduct, Analyze and Act, is the foundation for the Guidelines. The document focuses major attention on the planning phase, with the object being to prevent duplication and poor design, and to eliminate survey work that will not result in actions that can benefit customers and the agency. A long series of detailed questions supplement the Guidelines to further assist the Feedback Advisors and others. The document is available on the Internet (http://www.epa.gov/customerservice/feedback.htm), and is being used by individuals in other federal and state agencies to guide their feedback efforts.
The Guidelines and questions are not our only resources. To help ensure that feedback information used in an appropriate fashion, CSP staff encourage EPA programs to develop surveys consistent with OMB’s Resource Manual for Customer Surveys, EPA’s Survey Management Handbook and to take advantage of survey development training such as that offered by the Joint Program in Survey Methodology (JPSM). The EPA staff managing use of this ICR will continue to facilitate sharing of information gathered from customer satisfaction surveys, and explore ways to aid programs in survey development.
As a result of past survey feedback, sponsors have taken actions to change to revamp our dockets, to streamline processes and improve web sites and regularly issued documents.
3. Non-duplication, Consultations, and Other Collection Criteria
3(a) Non-duplication
EPA service providers develop customer satisfaction surveys to learn how their customers perceive their specific services. Therefore, the information collected will not overlap with other customer satisfaction surveys. Every effort will be made to channel all customer related surveys through this ICR and to prevent misuse of this ICR for program effectiveness surveys.
3(b) Public Notice Required Prior to ICR Submission to OMB
EPA conformed to the requirement for public notice by publishing a preliminary and final Federal Register Notice concerning our intent under this ICR and requesting comment.
3(c) Consultations
To estimate ICR usage during the next three years, EPA staff managing use of this ICR requested input from EPA’s Office of Environmental Information and reviewed ICR use over the previous year. This feedback and information was used to develop the burden estimates described in this document.
3(d) Effects of Less Frequent Collection
This information collection could not be conducted less frequently. EPA will gage customer reactions to and perceptions of services and products the Agency now provides in order to improve them. Programs will not survey all customers, nor will each program survey every year. There will be sufficient time between surveys to allow the actions taken in response to customer comments to show results. There are no technical or legal obstacles to reducing the burden.
3(e) General Guidelines
This ICR complies with OMB’s general guidelines for the collection of information.
3(f) Confidentiality
Not applicable
3(g) Sensitive Questions
No sensitive data will be collected.
4. The Respondents and the Information Requested
4(a) Respondents/SIC Codes
The Executive Order describes a customer as “...an individual or entity who is directly served by a department or agency.” The EPA, by the very nature of its mandate, serves very large and diverse groups that receive or are in some way affected by EPA services. Past EPA customer groups targeted for customer satisfaction surveys include individual citizens, industry/business, states/other governments, and web users. Because several customer groups use the same services, a survey may reach more than one of the designated customer categories. (The code standard industrial code (SIC) for “General Public” is 99.)
4(b) Information Requested
(I) Data items, including record keeping requirements
The Agency will maintain records of the surveys sent to OMB in the ICD. Offices sponsoring the surveys will retain files of the surveys, responses and analysis. Since customer satisfaction surveys seek to gauge public opinions on Agency services, the surveys have not and will not involve respondents in extensive searching of existing sources, or reformatting information to submit to the Agency. The Agency does not anticipate any public record keeping activities under this ICR.
(II) Respondent Activities
EPA customer satisfaction surveys have focused on services (hot lines, dockets, clearinghouse, websites), products (technical assistance, documents, information, training, workshops) and processes (grants, inspections, registrations, permitting).
The surveys conducted under this clearance are of two major types, “quantitative” and qualitative. Respondent activities related to “quantitative” are dependent on the survey method; feedback instrument types and the activities for each follow.
Mail surveys and Customer Feedback Forms (including comment cards, evaluation forms and some web-based surveys)1. Both may involve the following activities:
- Read instructions;
- Search data sources;
- Complete questionnaire;
- Mail questionnaire.
Telephone Surveys
- Listen to instructions;
- Answer questions (oral response)
EPA expects to continue its use of these surveys. Respondent activities related to “qualitative” feedback may include:
Focus Groups or Interviews
Listening to group instructions
Participating in discussions;
Completing any forms or materials provided at the group session.
EPA uses focus groups for evaluating various aspects of its programs, to assist in improving and testing of outreach materials and web sites, and to explore new aspects of service delivery.
Training/Education/Outreach products and services. EPA gathers feedback on its training, outreach products and educational programs through a variety of methods. The Agency distributes a broad array of materials to the public such as public affairs materials, videos, brochures and fact sheets, software, manuals, guidance material, reports, etc. It also holds many meetings, workshops and training sessions. Corresponding to this diverse set of products is a need to make extensive use of a variety of methods to evaluate customer satisfaction. EPA uses feedback forms in publications and on counters in service delivery areas, focus groups, mail and telephone surveys, and, when publications are available on the Internet, the Agency is using short on-line surveys to solicit customer input. Offices also ask for feedback on the usefulness of their web sites.
Many of these evaluation activities can use feedback forms to be completed by attendees after an EPA-sponsored event, or by users of documents, software or web sites. Focus groups are also useful for pre-testing EPA training materials (videos, brochures, etc.) prior to their dissemination to the public. Mail or telephone surveys help EPA identify a need for changes in training/educational programs, outreach products or services to assure their usefulness to a specific audience.
Hot lines/PICs/clearinghouses. Hotline evaluations are conducted on selected samples of hotline users. By their very nature, hotline customers will most often be surveyed by telephone. However, more complex surveys may require face-to-face interviews, focus sessions, or mail questionnaires. In addition, comment cards are used periodically when information packets are mailed by hotline, Public Information Center (PIC) or clearinghouse staff.
Miscellaneous Service Related Activities. The EPA has a broad network consisting of its headquarters and regional offices, laboratories, and field offices that may conduct customer surveys on outreach and other services that they provide. Most mail and telephone surveys are conducted under this “miscellaneous” category.
To reduce respondent burden, EPA has been expanding use of Internet feedback screens and comment blocks to provide increased opportunity for customers to comment on attributes of our services and web sites. Fewer offices each year develop lengthy questionnaires. Focus groups, though they require higher respondent burden, are still used because of the specificity and the depth of responses that offices/regions can obtain from them.
The redesign of EPA’s web pages required comment buttons on all EPA Internet sites. The Agency is therefore receiving and will continue to receive informal feedback and questions that are purely voluntary and not solicited specifically through sets of Agency questions of nine or more individuals outside the Federal government. We plan to continue to manage and act upon such customer information, particularly to improve EPA’s on-line information service on the Internet.
OMB’s Resource Manual for Customer Surveys (dated October 1993) and other relevant guidance documents state that the generic clearance shall be used for “strictly voluntary collections of opinion information from clients that have experience with the program that is the subject of each data collection” and precludes this option for use:
by regulatory agencies to survey regulated entities2;
in any situation where a respondent may perceive that a response will result in risks to his interests through potential penalties or loss of benefits;
for collecting factual information (other than simple identifying information, where needed); or
for collecting data from the general public.3
5. The Information Collected
Agency Activities, Collection Methodology, and Information Management
5(a) Agency Activities.
Agency activities associated with the collection of information include:
Developing survey design, assembling data sources (mailing lists, etc.) and pretesting questionnaire;
Internal EPA review and approval of questionnaire;
Disseminating questionnaire to respondents;
Gathering information from respondents;
Answering respondent questions, follow-up;
Reviewing data;
Recording submissions and analyzing results;
Preparing findings;
Storing and maintaining results
Making results public via annual reports and the Internet.
We do not account for the work of implementing and tracking actions taken as a result of customer feedback.
5(b) Collection Methodology and Information Management.
Prior to initiating any survey, sponsoring programs must seek final approval from OMB. EPA staff managing this ICR will continue to encourage survey sponsors to develop instruments using the 12 step process outlined in OMB’s Resource Manual for Customer Satisfaction Surveys (dated October 1993). The following internal review process, independent of the originating program office, will continue:
To obtain approval, sponsoring programs must submit a clearance package consisting of a memorandum from the program or office director and a copy of the survey instrument through the staff managing this ICR to the Information Collections Division in the Office of Environmental Information that will forward acceptable packages to OMB.
The memorandum must address the following 4:
Survey title, identification of survey originator (Office, point of contact/phone number)
Description and intended purpose of the survey as it relates to EPA customers
Methodology and use of anticipated results
Collection schedule, follow-up plans
Costs and burden to the Agency and respondents, and the number of respondents
- The memorandum will vary in length and detail, depending on the complexity of the survey. ICD staff, experienced with the requirements of the Paperwork Reduction Act (PRA), will review each submission to ensure that it meets the requirements of the PRA and any conditions of the generic approval, and may reject any proposed customer survey that does not meet the criteria outlined in Section 3(b).
- Statistical methods will not be used for many of the collections covered under this generic clearance. However, if a collection does use statistical methods to select a sample, answers to questions 1 through 5 in the section of the OMB guidelines for preparing supporting statements will be provided for that specific survey at the time the survey instrument is sent to OMB for clearance. If statistical design or methodological issues arise, the program will obtain Agency statistical expertise to help make any final determinations as to the statistical validity of the customer survey prior to OMB submittal.
- ICD will submit surveys and attached materials to OMB for an expedited review and determination. On an annual basis, the EPA shall submit a summary of the surveys cleared under the generic clearance to OMB. The summary shall include the survey title, sponsoring office, number of respondents and estimated burden hours.
- Sponsoring organizations within the EPA should maintain records according to each survey schedule. In general, survey results should be maintained for three years or until after follow-up activities have been completed.
- All offices will provide copies of their approved surveys, analytical reports and follow-up actions taken based on survey results to customer service.
- The customer service staff will share results and success stories with other offices and provide feedback to ICD on overall survey results. This base of experiences/lessons-learned could be useful in establishing model surveys for developing customer measurement programs within the EPA and other Agencies.
EPA expects use of five basic types of survey instruments for 2006-2008: feedback (to include comment cards, feedback and short evaluation forms), web based questionnaires, mail surveys, telephone surveys/short interviews and focus groups/long interviews. These are displayed in Table 5-1.
5 (c) Small Entity Flexibility.
Not applicable.
5 (d) Collection Schedule.
This will be dependent upon the needs of each originator of a survey. Schedules for customer surveys will be documented in the package submitted to the Information Collections Division for review and submittal to OMB.
Table 5-1 Projected Use of Surveys 2006-2008
Survey Type |
Total Uses 2006-2008 |
Web based |
48 |
Feedback/Comment/Evaluation Form |
15 |
Mail/detailed evaluation form |
3 |
Telephone |
3 |
Focus Group |
3 |
Total |
72 |
6. Estimating the Burden and the Cost of the Collection
6(a) Estimating Respondent Burden.
The estimate was based on the projected survey plans of EPA programs as summarized in Table 6-1. Table 6-2 summarizes respondent burden over the three years by survey type.
Table 6-1
EPA CUSTOMER SATISFACTION SURVEYS
INFORMATION COLLECTION REQUEST
BURDEN TABLE 2006 - 2008
Feedback Instrument |
Number of uses each year |
(Number of uses each year x number of people responding to each)=Total People Responding annually
|
(Respondent time x number of people responding)=total annual hours |
Total people responding 2006-2008
|
Total hours 2006-2008 |
Web based/e-mail based surveys |
15 |
(15 x 200)= 3,000 people |
(5 min. x 3,000)= 250 hours |
9,000 people |
750 hours |
OEI online surveys |
1 |
(1 x 2,640)= 2,640 people |
(5 min. x 2,640)= 220 hours |
7,920 people |
660 hours |
Feedback/Comment/ Evaluation Form |
5 |
(5 x 100)= 500 people |
(5 min. x 500)= 42 hours |
1,500 people |
126 hours |
Mail survey/detailed evaluation form |
1 |
(1 x 60)= 60 people |
(10 min. x 60)= 10 hours |
180 people |
30 hours |
Telephone survey |
1 |
(1 x 30)= 30 people |
(10 min. x 30)= 5 hours |
90 people |
15 hours |
Focus groups |
1 |
(1 x 15)= 15 people |
(2 hours x 15)= 30 hours |
45 people |
90 hours |
TOTALS |
24 |
6,245 people |
557 hours |
18,735 people |
1,671 hours |
Recent feedback from EPA programs suggests continuing interest in using customer satisfaction surveys as part of the overall long-term strategy of these organizations. The EPA estimates 557 hours of respondent burden on the part of 6,245 individuals each year from 2006 to 2008.
The EPA program staff planning to use this generic clearance know that burden should be as low as possible in keeping with the Paperwork Reduction Act. Survey designs will be simple, convenient, easy to respond to, and clear in content and purpose. Few long surveys will be designed; most surveys will be of limited scope and require only a short time to complete. Many comments card/feedback forms will be used, and programs will continue to increase their use of web based feedback.
Section 5(b) describes the types and number of uses for five types of survey instruments. If programs succeed in their expanded use of Internet for customer satisfaction surveys, burden could be further reduced. EPA may achieve additional reductions by eliminating some planned surveys through sharing results of completed surveys across the Agency.
6(b) Estimating Respondent Costs
I Labor Costs
Since the respondents represent such a diverse group, EPA based wage estimates on the Bureau of Labor Statistics (BLS) of the U.S. Department of Labor weekly earnings of wage and salary workers as reported on January 19, 2005, in the BLS news release “Usual Weekly Earnings of Wage and Salary Workers: Fourth Quarter 2005.” The weekly earnings are $659.00; this computes to $16.50 per hour for a 40 hour week.
There is no need for “developing, acquiring, or utilizing technology and systems for the purpose of collecting, validating or verifying information,” “....disclosing and providing information,” “adjusting the existing ways to comply with any previously applicable instructions or requirements,” “training personnel to be able to respond to a collection of information,” “searching data sources,” nor a need for the respondents to keep records. Burden activities include only a few steps: reviewing instructions, responding, and sending (e-mail or mail) responses when the surveys are not performed in person or over the telephone.
Table 6-2 displays the annual burden estimates for respondents and total estimated respondent costs.
II Capital and Operations and Maintenance Costs
Not applicable.
III Capital/Start-up vs. Operating and Maintenance (O & M) Costs
Not applicable.
IV Annualizing Capital Costs
Not applicable.
6 (c) Estimating Agency Burden and Cost.
Tables 6-3 through 6-7 provide the annual estimates for agency burden associated with developing, disseminating customer surveys and analyzing the results. Wage estimates were divided into three categories of labor: Management (GS-15), Technical (GS-13), and Clerical (GS-7).5
Table 6-2 Respondent Universe, Total Burden and Costs
Survey Type |
3 years Surveys |
Respondents
|
Total Hours |
Total Cost |
|
3 |
180 |
30 |
$456 |
Telephone |
3 |
90 |
15 |
$228 |
Feedback cards, evaluations + Web-based |
63 |
18,420 |
1,536 |
$23,347.2 |
Focus Groups |
3 |
45 |
90 |
$1,368 |
Totals |
72 |
18,735 |
1,671 |
$25,399.2 |
OPEI and Information Collection staffs will be sharing information and survey instruments across the Agency. Feedback Advisors will also use this information to assist people. Costs per instrument should continue to be reduced as Agency staff members gain experience with feedback through Internet, and with developing and analyzing surveys of other types. However, since these cost reductions cannot be accurately estimated, aggregate annual costs that follow do not reflect these cost reductions to the Agency.
6(d) Estimating the Respondent Universe and Total Burden and Costs Burden
Table 6-1 provides information on each survey by instrument type, number of respondents expected, and burden hours requested per survey. Table 6-8 summarizes the total burden and costs for respondents, and the Agency. Activities have been grouped to reflect the various types of surveys and the total respondents expected for each instrument type. In all cases, the activities performed remain only the time required to read, respond and transmit the survey instruments. Burden estimates were calculated using the median weekly earnings of the nation's full‑time wage and salary workers in the fourth quarter of 2005, $659.00, or $16.50 per hour for a 40 hour week.
Table 6-3. Agency Burden/Cost for Telephone Surveys
Activities |
Manager @ $58 |
Burden Technical @ $42 |
Hours Clerical @ $20 |
Total Hrs. |
Total Cost |
Developing survey Obtaining EPA approval Gathering information Reviewing data; follow-ups Analyzing results Storing and maintaining results Preparing survey findings |
1.5 1.0 0.0 0.0 2.0 0.0 1.0 |
40.0 4.0 60.0 16.0 80.0 4.0 80.0 |
20.0 1.0 20.0 8.0 0.0 5.0 8.0 |
61.5 6.0 80.0 24.0 82.0 9.0 89.0 |
$2,167 $246 $2,920 $832 $3,476 $268 $3,578 |
Totals hours Category costs |
5.5 $319 |
284.0 $11,928 |
62.0 $1,240 |
351.5
|
$13,487 |
Table 6-4. Agency Burden/Cost for Mail Surveys and Evaluation Forms
Activities |
Manager @ $58 |
Burden Technical @ $42 |
Hours Clerical @$20 |
Total Hours
|
Total Cost |
Developing survey Obtaining EPA approval Gathering information Reviewing data Analyzing results Storing and maintaining results Preparing survey findings |
1.5 1.0 0.0 0.0 2.0 0.0
5.0 |
80.0 4.0 40.0 8.0 40.0 2.0
40.0 |
8.0 1.0 16.0 8.0 0.0 3.0
8.0 |
89.5 6.0 56.0 16.0 42.0 5.0
53.0 |
$3,607 $246 $2,000 $496 $1,796 $144
$2,130 |
Totals hours Category costs |
9.5 $551 |
214.0 $8,988 |
44.0 $880 |
267.5
|
$10,419 |
Table 6-5. Agency Burden/Cost for Customer Feedback Forms/Internet Screens*
Activities |
Manager @ $58 |
Burden Technical @ $42 |
Hours Clerical @ $20 |
Total Hours |
Total Cost |
Developing feedback instruments Obtaining EPA approval Gathering information Reviewing data Analyzing results Storing and maintaining results Preparing survey findings |
1.0
1.0 0.0 0.0 2.0 0.0
2.0 |
20.0
4.0 20.0 8.0 20.0 2.0
20.0 |
2.0
1.0 16.0 8.0 0.0 3.0
8.0 |
23.0
6.0 36.0 16.0 22.0 5.0
30.0 |
$938
$246 $1,160 $496 $956 $144
$1,116 |
Total hours Category costs |
6.0 $348 |
94.0 $3,948 |
38.0 $760 |
138.0
|
$5,056 |
* Internet feedback forms, comment cards, short publication/meeting/workshop evaluation forms and short web-based surveys are grouped into this one category.
Table 6-6. Agency Burden/Cost for Focus Groups
Activities |
Manager @ $58 |
Burden Technical @ $42 |
Hours Clerical @ $20 |
Total Hours |
Total Cost |
Developing Focus Sessions Obtaining EPA approval Conducting Focus Groups Reviewing data Analyzing results Storing and maintaining result Preparing findings |
1.5 1.0 0.0 0.0 1.0 0.0 3.0 |
40.0 4.0 8.0 4.0 20.0 2.0 20.0 |
40.0 2.0 8.0 4.0 0.0 3.0 8.0 |
81.5 7.0 16.0 8.0 21.0 5.0 31.0
|
$2,567 $266 $496 $248 $898 $144 $1,174
|
Totals hours Category costs |
6.5 $377 |
98.0 $4,116 |
65.0 $1,300 |
169.5
|
$5,793 |
6(e) Bottom Line Burden Hours and Cost Tables
I Respondent Tally See Table 6-1.
II The Agency Tally
Table 6-7 provides the Agency Tally estimates. The total EPA tally for the three-year period is $407,625.
III Variations in the Annual Bottom Line
EPA burden hour projections are: 3,686.5 hours annually.
IV Reasons for Change in Burden
Within the Agency, fewer organizations than in past years have decided to do customer satisfaction surveys. We anticipate this trend to continue during the next three years, and expect more organizations to use web-based or short surveys. Offices and regions will be using a variety of techniques, but will repeatedly use the same survey instruments. The number of respondent burden hours will continue to drop as more organizations use web-based surveys and feedback options, rather than longer and more formal survey instruments. Respondent burden hours are: 557 hours annually.
Table 6-7. Aggregate Agency Table for Annual Burden/Cost
Survey Collection Type |
Annual # of Collections |
Annual Cost |
Annual Hours
|
Telephone
Feedback (cards, web-based, e-mail & evaluation forms)
Focus Groups |
1
1
21
1 |
$13,487
$10,419
$106,176
$5,793
|
351.5
267.5
2,898
169.5 |
Totals |
24 |
$135,875 |
3,686.5 |
V Burden Statement
The following statement applies overall to the planned surveys for the next three years:
The annual public reporting and recordkeeping burden for this collection of information is estimated to average 5.4 hours per response, including the time for reviewing instructions, gathering information, and completing and reviewing the collection of information. Burden means the total time, effort, or financial resources expended by persons to generate, maintain, retain, or disclose or provide information to or for a Federal agency. This includes the time needed to review instructions; develop, acquire, install, and utilize technology and systems for the purposes of collecting, validating, and verifying information, processing and maintaining information, and disclosing and providing information; adjust the existing ways to comply with any previously applicable instructions and requirements; train personnel to be able to respond to a collection of information; search data sources; complete and review the collection of information; and transmit or otherwise disclose the information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB control numbers for EPA's regulations are listed in 40 CFR part 9 and 48 CFR chapter 15.
To comment on the Agency's need for this information, the accuracy of the provided burden estimates, and any suggested methods for minimizing respondent burden, including the use of automated collection techniques, EPA has established a public docket for this ICR under Docket ID Number EPA-HQ-OA-2006-0074, which is available for online viewing at www.regulations.gov, or in person viewing at the OEI Docket in the EPA Docket Center (EPA/DC), EPA West, Room B102, 1301 Constitution Avenue, NW, Washington, D.C. The EPA Docket Center Public Reading Room is open from 8:30 a.m. to 4:30 p.m., Monday through Friday, excluding legal holidays. The telephone number for the Reading Room is (202) 566-1744, and the telephone number for the OEI Docket is (202) 566-1752. An electronic version of the public docket is available at www.regulations.gov. This site can be used to submit or view public comments, access the index listing of the contents of the public docket, and to access those documents in the public docket that are available electronically. When in the system, select “search,” then key in the Docket ID Number identified above. Also, you can send comments to the Office of Information and Regulatory Affairs, Office of Management and Budget, 725 17th Street, NW, Washington, D.C. 20503, Attention: Desk Officer for EPA. Please include the EPA Docket ID Number EPA-HQ-2006-0074 and OMB Control Number 2090-0019 in any correspondence.
Table 6-8 Aggregate EPA and Respondent Costs
Surveys EPA hours EPA costs Respondent hours Respondent costs
2006 |
24 |
3,686.5 |
$135,875 |
557 |
$8,466.4 |
2007 |
24 |
3,686.5 |
$135,875 |
557 |
$8,466.4 |
2008 |
24 |
3,686.5 |
$135,875 |
557 |
$8,466.4 |
Total |
72 |
11,059.5 |
$407,625 |
1,671 |
$25,399.2 |
Three year total respondents: 18,735
EXHIBITS
Samples of past OMB-Approved EPA Survey Instruments
Two different OMB approved survey instruments follow as Exhibits 1 and 2. The first is an approved telephone survey questionnaire. The second is a set of several feedback cards and surveys administered by various methods.
All future surveys under this ICR are to include, on or near the first page of the survey, a burden statement specific to that survey explaining the number of hours/minutes per year per respondent and what that burden entails (e.g.: Respondent burden for this survey is estimated to be five minutes for reading and responding to the questions.) as well as the following paragraphs.
Burden means the total time, effort, or financial resources expended by persons to generate, maintain, retain, or disclose or provide information to or for a Federal agency. This includes the time needed to review instructions; develop, acquire, install, and utilize technology and systems for the purposes of collecting, validating, and verifying information, processing and maintaining information, and disclosing and providing information; adjust the existing ways to comply with any previously applicable instructions and requirements; train personnel to be able to respond to a collection of information; search data sources; complete and review the collection of information; and transmit or otherwise disclose the information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB control numbers for EPA's regulations are listed in 40 CFR Part 9 and 48 CFR Chapter 15.
Send comments on the Agency's need for this information, the accuracy of the provided burden estimates, and any suggested methods for minimizing respondent burden, including through the use of automated collection techniques to the Director, Collection Strategies Division, U.S. Environmental Protection Agency (2822T), 1200 Pennsylvania Ave., NW, Washington, D.C. 20460; and to the Office of Information and Regulatory Affairs, Office of Management and Budget, 725 17th Street, NW, Washington, DC 20503, Attention: Desk Officer for EPA. Include the EPA ICR number 1711.04 and OMB control number 2090-2119 in any correspondence.
All future surveys under this ICR are to include the following OMB number and expiration date information at the top right hand corner of the survey’s first page:
OMB CONTROL NO: 2090-0019
EXPIRATION DATE: 06/30/2009
Exhibit 1
Telephone Survey Questionnaire
Survey Questionnaire
The purpose of the survey is to learn how well we met the needs and expectations for EPA’s National Network for Environmental Management Studies Program. We’d like to ask you six questions regarding your experience with program services. It should take less than five minutes, and any additional comments are welcome. If you have any questions or if additional ideas arise after the survey, you are welcome to call me back.
1. Please rate your satisfaction with the relevance of your NNEMS fellowship work experience to your field of study. (1 to 6, where 1 is very dissatisfied and 6 is very satisfied)
2. Please rate the degree to which your experience as a NNEMS fellow enabled you to decide whether to continue a career related to protecting the environment or human health. (1 to 6, where 1 is minimal influence and 6 is large influence)
3. a. Please rate your satisfaction with the application process for the NNEMS fellowship. (1 to 6, where 1 is very dissatisfied and 6 is very satisfied)
b. If your satisfaction rating was lower than 4, please explain any process problem(s) and provide any suggestions you may have to improve the process.
Please rate your satisfaction with the following aspects of your working with EPA. (1 to 6, where 1 is very dissatisfied and 6 is very satisfied)
Staff:
a. provided appropriate initial guidance on project design
b. treated with courtesy
c. provided access to people and materials
d. enabled me to understand the mission, structure and functions of the Agency
e. provided sufficient periodic review to ensure timely completion of the project
f. answered questions accurately
g. provided clear guidance
h. were easily accessible
Please rate the degree to which you would recommend participation in the NNEMS fellowship program to others. (1 to 6, where 1 is not at all and 6 is highly recommend)
Do you have any suggestions for how EPA could improve the NNEMS program?
Public reporting burden for this collection of information is estimated to average five minutes per response, including the time for reviewing instructions, gathering information, and completing and reviewing the collection of information. Send comments on the Agency’s need for this information, the accuracy of the provided burden estimates, and any suggestions for reducing the burden, including the use of automated collection techniques to the Director, Information Collections Division, Office of Environmental Information, United States Environmental Protection Agency (Mail Code 2822T), 1200 Pennsylvania Avenue , NW, Washington, D.C. 20460; and to the Office of Information & Regulatory Affairs, Office of Management & Budget, 725 17th Street, NW, Washington, DC 20503, Attention: Desk Officer for EPA. Include the EPA ICR number and the OMB control number in any correspondence.
Exhibit 2
NRMRL Feedback Cards, Evaluation Forms, Evaluation Surveys
CDs Response Cards The card will contain 13 questions, on one side of the card, franking and mailing address on the other.
REVIEWER INFORMATION
1. With what type of organization are you affiliated (check as may as apply)?
a. Regulated facility or business e. Trade association
Industry sector f. Nonprofit organization
Consulting company g. School or university
Government h. Other (please specify)
_________
OPTIONAL: Job Title: _____________________________________________________
Please provide your name and phone number so that we can contact you if we have any questions about your responses. ____________________________________________________________
EFFECTIVE COMMUNICATION/CLARITY OF INFORMATION
On a scale of 1 to 6 where 1 is very dissatisfied and 6 is very satisfied please indicate your level of satisfaction with the following
Readability
2. a. Readability (i.e., flow, writing style, and presentation of concepts) of the CD content?
6 Very satisfied 3 Somewhat dissatisfied
5 Satisfied 2 Dissatisfied
4 Somewhat satisfied 1 Very dissatisfied
Don’t know
b. COMMENT:______________________________________________
3. a. Understandability of the CD content (i.e., the ability of the CD to convey the information
in a way that is easy to grasp and comprehend)?
6 Very satisfied 3 Somewhat dissatisfied
5 Satisfied 2 Dissatisfied
4 Somewhat satisfied 1 Very dissatisfied
Don’t know
b. COMMENT:_______________________________________________________
4. a. Format (attractiveness, use of graphics) of this CD?
6 Very satisfied 3 Somewhat dissatisfied
5 Satisfied 2 Dissatisfied
4 Somewhat satisfied 1 Very dissatisfied
Don’t know
b. COMMENT:_______________________________________________________
Content
5. a. How satisfied are you that this CD met its stated objectives?
6 Very satisfied 3 Somewhat dissatisfied
5 Satisfied 2 Dissatisfied
4 Somewhat satisfied 1 Very dissatisfied
Don’t know
b. COMMENT:_____________________________________________________
6. Please rate the CD as to how informative (i.e., instructive) the discussions within each of the sections are, with 6 = Very informative and 1 = Not informative
Section Very Not Comment
Informative Informative
a. 6 5 4 3 2 1 _________________________________________
b. 6 5 4 3 2 1 _________________________________________
c. 6 5 4 3 2 1 _________________________________________
d. 6 5 4 3 2 1 _________________________________________
e. 6 5 4 3 2 1 _________________________________________
f. 6 5 4 3 2 1 _________________________________________
7. a. How satisfied are you with the accuracy of the technical information (knowledge regarding the subject matter) included in the CD?
6 Very satisfied 3 Somewhat dissatisfied
5 Satisfied 2 Dissatisfied
4 Somewhat satisfied 1 Very dissatisfied
Don’t know
b. COMMENT: _______________________________________________________________
Organization and Clarity
8. a. How satisfied are you with the organization (presented in a logical order) of the CD?
6 Very satisfied 3 Somewhat dissatisfied
5 Satisfied 2 Dissatisfied
4 Somewhat satisfied 1 Very dissatisfied
Don’t know
b. COMMENT: ___________________________________________________________
9. a. How clearly (plain and evident) is information communicated in this CD?
6 Very clearly 3 Somewhat unclearly
5 Clearly 2 Unclearly
4 Somewhat clearly 1 Very unclearly
Don’t know
b. COMMENT: _________________________________________________________
Graphics (Figures, Tables and Pictures)
10. a. How much did the graphics contribute to your overall understanding of the material presented on the CD?
Very much A little
Somewhat Not at all
b. COMMENT: _________________________________________________________
Other Recommendations:
11. We welcome any other comments you have about this CD.
______________________________________________________________________
______________________________________________________________________
______________________________________________________________________
12. a. Do you expect to refer to the information from this CD again?
Yes No
b. Why or why not? ________________________________________________________
13. a. Would you recommend this CD to others?
Yes No
b. Why or why not? ________________________________________________________
Public reporting burden for this collection of information is estimated to average four minutes per response, including the time for reviewing instructions, gathering information, and completing and reviewing the collection of information. Send comments on the Agency’s need for this information, the accuracy of the provided burden estimates, and any suggestions for reducing the burden, including the use of automated collection techniques to the Director, Information Collections Division, Office of Environmental Information, United States Environmental Protection Agency (Mail Code 2822T), 1200 Pennsylvania Avenue , NW, Washington, D.C. 20460; and to the Office of Information & Regulatory Affairs, Office of Management & Budget, 725 17th Street, NW, Washington, DC 20503, Attention: Desk Officer for EPA. Include the EPA ICR number and the OMB control number in any correspondence.
OMB #2090-0019 Expires 3/31/06
Documents Response Cards The card will contain 13 questions, on one side of the card, with franking and mailing address on the other.
REVIEWER INFORMATION
1. With what type of organization are you affiliated (check as may as apply)?
a. Regulated facility or business e. Trade association
Industry sector f. Nonprofit organization
Consulting company g. School or university
Government h. Other (please specify)_________
OPTIONAL: Job Title: _____________________________________________________
Please provide your name and phone number so that we can contact you if we have any questions about your responses._________________________________________________________
EFFECTIVE COMMUNICATION/CLARITY OF INFORMATION
On a scale of 1 to 6 where 1 is very dissatisfied and 6 is very satisfied please indicate your level of satisfaction with the following
2. a. Readability (i.e., flow, writing style, and presentation of concepts) of the document?
6 Very satisfied 3 Somewhat dissatisfied
5 Satisfied 2 Dissatisfied
4 Somewhat satisfied 1 Very dissatisfied
Don’t know
b. COMMENT: _____________________________________________________________
3. a. Understandability of the document content (i.e., the ability of the product to convey the information in a way that is easy to grasp and comprehend)?
6 Very satisfied 3 Somewhat dissatisfied
5 Satisfied 2 Dissatisfied
4 Somewhat satisfied 1 Very dissatisfied
Don’t know
b. COMMENT: ____________________________________________________________
4. a. Format (attractiveness, use of graphics) of this document?
6 Very satisfied 3 Somewhat dissatisfied
5 Satisfied 2 Dissatisfied
4 Somewhat satisfied 1 Very dissatisfied
Don’t know
b. COMMENT:__________________________________________________
Content
5. a. How satisfied are you that this document met its stated objectives?
6 Very satisfied 3 Somewhat dissatisfied
5 Satisfied 2 Dissatisfied
4 Somewhat satisfied 1 Very dissatisfied
Don’t know
b. COMMENT: ____________________________________________________________
6. Please rate the document as to how informative (i.e., instructive) the discussions within each of the sections are, with 6 = Very informative and 1 = Not informative
Section Very Not Comment
Informative Informative
a. 6 5 4 3 2 1 _________________________________________
b. 6 5 4 3 2 1 _________________________________________
c. 6 5 4 3 2 1 _________________________________________
d. 6 5 4 3 2 1 _________________________________________
e. 6 5 4 3 2 1 _________________________________________
f. 6 5 4 3 2 1 _________________________________________
7. a. How satisfied are you with the accuracy of the technical information (knowledge regarding the subject matter) included in the document?
6 Very satisfied 3 Somewhat dissatisfied
5 Satisfied 2 Dissatisfied
4 Somewhat satisfied 1 Very dissatisfied
Don’t know
b. COMMENT: ____________________________________________________________
Organization and Clarity
8. a. How satisfied are you with the organization (presented in a logical order) of the document?
6 Very satisfied 3 Somewhat dissatisfied
5 Satisfied 2 Dissatisfied
4 Somewhat satisfied 1 Very dissatisfied
Don’t know
b. COMMENT: ____________________________________________________________
9. a. How clearly (plain and evident) is information communicated in this document?
6 Very clearly 3 Somewhat unclearly
5 Clearly 2 Unclearly
4 Somewhat clearly 1 Very unclearly
Don’t know
b. COMMENT: ____________________________________________________________
Graphics (Figures, Tables, and Pictures)
10. a. How much did the graphics contribute to your overall understanding of the material presented in the document?
Very much A little
Somewhat Not at all
b. COMMENT: ___________________________________________________________
Other Recommendations:
11. We welcome any other comments you have about this document.
______________________________________________________________________
______________________________________________________________________
______________________________________________________________________
12. a. Do you expect to refer to the information from this document again?
Yes No
b. Why or why not? ________________________________________________________
13 a. Would you recommend this document to others?
Yes No
b. Why or why not? ________________________________________________________
Public reporting burden for this collection of information is estimated to average four minutes per response, including the time for reviewing instructions, gathering information, and completing and reviewing the collection of information. Send comments on the Agency’s need for this information, the accuracy of the provided burden estimates, and any suggestions for reducing the burden, including the use of automated collection techniques to the Director, Information Collections Division, Office of Environmental Information, United States Environmental Protection Agency (Mail Code 2822T), 1200 Pennsylvania Avenue , NW, Washington, D.C. 20460; and to the Office of Information & Regulatory Affairs, Office of Management & Budget, 725 17th Street, NW, Washington, DC 20503, Attention: Desk Officer for EPA. Include the EPA ICR number and the OMB control number in any correspondence.
OMB #2090-0019 Expires 3/31/06
The Focus Group Evaluation Responses. The forms contain 8 questions and will be collected by facilitators who carry out the Focus Work, during the focus group meeting. Focus groups, in this context, are gathered to evaluate directly, one or another of ORD’s technology transfer products.
The facilitator will present the product, for example a document or brochure, or guidance manual, to the group, giving them time to read it or look at it in some detail. Some brief discussion may be held in terms of the intended audience or motivation for the production of the product. The facilitator will not give an opinion of the product or its content, but may answer questions about its probable audience and use.
Then the facilitator will ask the group to fill out the questionnaire. The intent of the process is to get first hand responses to the product, with an eye to improving it and future similar products. For this reason, the group will be as diverse in background as the setting allows. Larger National meetings will offer the most diversity.
SECTION I : REVIEWER INFORMATION
1. With what type of organization are you affiliated (check as may as apply)?
a. Regulated facility or business e. Trade association
Industry sector f. Nonprofit organization
Consulting company g. School or university
Government h. Other (please specify)_________
OPTIONAL: Job Title: _______________________________________________
Please provide your name and phone number so that we can contact you if we have any questions about your responses._________________________________________________________
SECTION II: EFFECTIVE COMMUNICATION/CLARITY OF INFORMATION
On a scale of 1 to 6 where 1 is very dissatisfied and 6 is very satisfied please indicate your level of satisfaction with the following
Content
2. a. How satisfied are you that this Focus Group met its stated objectives?
6 Very satisfied 3 Somewhat dissatisfied
5 Satisfied 2 Dissatisfied
4 Somewhat satisfied 1 Very dissatisfied
Don’t know
b. COMMENT: ____________________________________________________________
3. a. How satisfied are you with the accuracy of the technical information (knowledge regarding the subject matter) included in the Focus Group?
6 Very satisfied 3 Somewhat dissatisfied
5 Satisfied 2 Dissatisfied
4 Somewhat satisfied 1 Very dissatisfied
Don’t know
b. COMMENT: ____________________________________________________________
4. Is there information that should not have been included in this focus group? If so, please specify.
________________________________________________________________
5. Is there additional information that should be included in this focus group but was not? If so, please specify?
________________________________________________________________
Organization and Clarity
6. a. How satisfied are you with the organization (presented in a logical order) of the focus group?
6 Very satisfied 3 Somewhat dissatisfied
5 Satisfied 2 Dissatisfied
4 Somewhat satisfied 1 Very dissatisfied
Don’t know
b. COMMENT:
_______________________________________________________________
_______________________________________________________________
Other Recommendations:
7. We welcome any other comments you have about this focus group.
______________________________________________________________________
______________________________________________________________________
______________________________________________________________________
SECTION III: USEFULNESS OF THE FOCUS GROUP
8. a. Would you recommend this kind of focus group to others?
Yes No
b. Why or why not? ________________________________________________________
Public reporting burden for this collection of information is estimated to average four minutes per response, including the time for reviewing instructions, gathering information, and completing and reviewing the collection of information. Send comments on the Agency’s need for this information, the accuracy of the provided burden estimates, and any suggestions for reducing the burden, including the use of automated collection techniques to the Director, Information Collections Division, Office of Environmental Information, United States Environmental Protection Agency (Mail Code 2822T), 1200 Pennsylvania Avenue , NW, Washington, D.C. 20460; and to the Office of Information & Regulatory Affairs, Office of Management & Budget, 725 17th Street, NW, Washington, DC 20503, Attention: Desk Officer for EPA. Include the EPA ICR number and the OMB control number in any correspondence.
Meeting Evaluation Forms will be given to each meeting attendee in their handout materials and collected on their way out of the meeting or sessions. The form contains 11 questions.
SECTION I : REVIEWER INFORMATION
1. With what type of organization are you affiliated (check as may as apply)?
a. Regulated facility or business e. Trade association
Industry sector f. Nonprofit organization
Consulting company g. School or university
Government h. Other (please specify)_________
OPTIONAL: Job Title: ____________________________________________________
Please provide your name and phone number so that we can contact you if we have any questions about your responses. ______________________________________________________________
SECTION II: EFFECTIVE COMMUNICATION/CLARITY OF INFORMATION
On a scale of 1 to 6 where 1 is very dissatisfied and 6 is very satisfied please indicate your level of satisfaction with the following
Readability
2. a. Readability (i.e., flow, writing style, and presentation of concepts) of the Meeting Handout Materials?
6 Very satisfied 3 Somewhat dissatisfied
5 Satisfied 2 Dissatisfied
4 Somewhat satisfied 1 Very dissatisfied
Don’t know
b. COMMENT: ___________________________________________________________
3. a. How satisfied are you that this Meeting met its stated objectives?
6 Very satisfied 3 Somewhat dissatisfied
5 Satisfied 2 Dissatisfied
4 Somewhat satisfied 1 Very dissatisfied
Don’t know
b. COMMENT: ____________________________________________________________
Content
4. a. How satisfied are you with the accuracy of the technical information (knowledge regarding the subject matter) included in the Meeting?
6 Very satisfied 3 Somewhat dissatisfied
5 Satisfied 2 Dissatisfied
4 Somewhat satisfied 1 Very dissatisfied
Don’t know
b. COMMENT: ____________________________________________________________
Organization and Clarity
5. a. How satisfied are you with the organization (presented in a logical order) of the Meeting?
6 Very satisfied 3 Somewhat dissatisfied
5 Satisfied 2 Dissatisfied
4 Somewhat satisfied 1 Very dissatisfied
Don’t know
b. COMMENT: ____________________________________________________________
6. a. How clearly (plain and evident) is information communicated in this Meeting?
6 Very clearly 3 Somewhat unclearly
5 Clearly 2 Unclearly
4 Somewhat clearly 1 Very unclearly
Don’t know
b. COMMENT: ____________________________________________________________
7. We welcome any other comments you have about this Meeting.
______________________________________________________________________
______________________________________________________________________
______________________________________________________________________
SECTION III: AWARENESS AND UNDERSTANDING
8. a. How satisfied are you that this Meeting meets your need for information about the issue/subject matter?
6 Very satisfied 3 Somewhat dissatisfied
5 Satisfied 2 Dissatisfied
4 Somewhat satisfied 1 Very dissatisfied
Don’t know
b. COMMENT: ____________________________________________________________
9. a. How satisfied are you that the information presented in this Meeting increased your
awareness of the issue/subject matter?
6 Very satisfied 3 Somewhat dissatisfied
5 Satisfied 2 Dissatisfied
4 Somewhat satisfied 1 Very dissatisfied
Don’t know
b. COMMENT: ____________________________________________________________
10. a. Do you expect to refer to the information from this Meeting again?
Yes No
b. Why or why not? ________________________________________________________
11. a. Would you recommend this Meeting to others?
Yes No
b. Why or why not? ________________________________________________________
Public reporting burden for this collection of information is estimated to average four minutes per response, including the time for reviewing instructions, gathering information, and completing and reviewing the collection of information. Send comments on the Agency’s need for this information, the accuracy of the provided burden estimates, and any suggestions for reducing the burden, including the use of automated collection techniques to the Director, Information Collections Division, Office of Environmental Information, United States Environmental Protection Agency (Mail Code 2822T), 1200 Pennsylvania Avenue , NW, Washington, D.C. 20460; and to the Office of Information & Regulatory Affairs, Office of Management & Budget, 725 17th Street, NW, Washington, DC 20503, Attention: Desk Officer for EPA. Include the EPA ICR number and the OMB control number in any correspondence.
Reviewer Forms are included with the other forms sent to a document reviewer. They will be pre-franked and addressed for return to USEPA. The form contains 10 questions.
1. With what type of organization are you affiliated (check as may as apply)?
a. Regulated facility or business e. Trade association
Industry sector f. Nonprofit organization
Consulting company g. School or university
Government h. Other (please specify)_________
SECTION II: EFFECTIVE COMMUNICATION/CLARITY OF INFORMATION
On a scale of 1 to 6 where 1 is very dissatisfied and 6 is very satisfied please indicate your level of satisfaction with the following
Readability
2. a. Readability (i.e., flow, writing style, and presentation of concepts) of the product you reviewed?
6 Very satisfied 3 Somewhat dissatisfied
5 Satisfied 2 Dissatisfied
4 Somewhat satisfied 1 Very dissatisfied
Don’t know
b. COMMENT: ____________________________________________________________
3. a. Understandability of the product you reviewed (i.e., the ability of the product to convey the information in a way that is easy to grasp and comprehend)?
6 Very satisfied 3 Somewhat dissatisfied
5 Satisfied 2 Dissatisfied
4 Somewhat satisfied 1 Very dissatisfied
Don’t know
b. COMMENT: _____________________________________________________
Content
4. a. How satisfied are you that this product you reviewed met its stated objectives?
6 Very satisfied 3 Somewhat dissatisfied
5 Satisfied 2 Dissatisfied
4 Somewhat satisfied 1 Very dissatisfied
Don’t know
b. COMMENT: ____________________________________________________________
5. a. How satisfied are you with the accuracy of the technical information (knowledge regarding the subject matter) included in the product you reviewed?
6 Very satisfied 3 Somewhat dissatisfied
5 Satisfied 2 Dissatisfied
4 Somewhat satisfied 1 Very dissatisfied
Don’t know
b. COMMENT: ____________________________________________________________
Graphics (Figures, Tables and Pictures)
6. a. How satisfied are you with the understandability of our “graphics” (i.e., ability to convey the intended meaning in a way that is easy to grasp and comprehend)?
6 Very satisfied 3 Somewhat dissatisfied
5 Satisfied 2 Dissatisfied
4 Somewhat satisfied 1 Very dissatisfied
Don’t know
b. COMMENT: ____________________________________________________________
7. a. How much did the graphics contribute to your overall understanding of the material presented in the product you reviewed?
Very much A little
Somewhat Not at all
b. COMMENT: ___________________________________________________________
Other Recommendations:
8. We welcome any other comments you have about the product you reviewed.
______________________________________________________________________
______________________________________________________________________
______________________________________________________________________
9. a. Do you expect to refer to the information from the product you reviewed again?
Yes No
b. Why or why not? ________________________________________________________
10. a. Would you recommend the product you reviewed to others?
Yes No
b. Why or why not? ________________________________________________________
Public reporting burden for this collection of information is estimated to average two minutes per response, including the time for reviewing instructions, gathering information, and completing and reviewing the collection of information. Send comments on the Agency’s need for this information, the accuracy of the provided burden estimates, and any suggestions for reducing the burden, including the use of automated collection techniques to the Director, Information Collections Division, Office of Environmental Information, United States Environmental Protection Agency (Mail Code 2822T), 1200 Pennsylvania Avenue , NW, Washington, D.C. 20460; and to the Office of Information & Regulatory Affairs, Office of Management & Budget, 725 17th Street, NW, Washington, DC 20503, Attention: Desk Officer for EPA. Include the EPA ICR number and the OMB control number in any correspondence.
OMB #2090-0019 Expires 3/31/06
Workshop Evaluation Surveys will be handed out at each workshop and collected at the end of the session. The form contains 10 questions.
SECTION I : REVIEWER INFORMATION
1. With what type of organization are you affiliated (check as may as apply)?
a. Regulated facility or business e. Trade association
Industry sector f. Nonprofit organization
Consulting company g. School or university
Government h. Other (please specify)_________
OPTIONAL: Job Title: _______________________________________________
Please provide your name and phone number so that we can contact you if we have any questions about your responses. ______________________________________________________________
SECTION II: EFFECTIVE COMMUNICATION/CLARITY OF INFORMATION
On a scale of 1 to 6 where 1 is very dissatisfied and 6 is very satisfied please indicate your level of satisfaction with the following
Readability
2. a. Readability (i.e., flow, writing style, and presentation of concepts) of the workshop materials?
6 Very satisfied 3 Somewhat dissatisfied
5 Satisfied 2 Dissatisfied
4 Somewhat satisfied 1 Very dissatisfied
Don’t know
b. COMMENT: ______________________________________________________
3. a. Understandability of the workshop materials (i.e., the ability of the product to convey the information in a way that is easy to grasp and comprehend)?
6 Very satisfied 3 Somewhat dissatisfied
5 Satisfied 2 Dissatisfied
4 Somewhat satisfied 1 Very dissatisfied
Don’t know
b. COMMENT: _____________________________________________________________
4. Please rate the workshop materials as to how informative (i.e., instructive) the discussions within each of the sections are, with 6 = Very informative and 1 = Not informative
Section Very Not Comment
Informative Informative
a. 6 5 4 3 2 1 _________________________________________
b. 6 5 4 3 2 1 _________________________________________
c. 6 5 4 3 2 1 _________________________________________
d. 6 5 4 3 2 1 _________________________________________
e. 6 5 4 3 2 1 _________________________________________
f. 6 5 4 3 2 1 _________________________________________
5. a. Accuracy of the technical information (knowledge regarding the subject matter) included in the workshop materials?
6 Very satisfied 3 Somewhat dissatisfied
5 Satisfied 2 Dissatisfied
4 Somewhat satisfied 1 Very dissatisfied
Don’t know
b. COMMENT: ____________________________________________________________
Other Recommendations:
6. We welcome any other comments you have about this workshop.
______________________________________________________________________
______________________________________________________________________
______________________________________________________________________
SECTION III: AWARENESS AND UNDERSTANDING
7. a. How satisfied are you that this workshop meets your need for information about the issue/subject matter?
6 Very satisfied 3 Somewhat dissatisfied
5 Satisfied 2 Dissatisfied
4 Somewhat satisfied 1 Very dissatisfied
Don’t know
b. COMMENT: ____________________________________________________________
8. a. How satisfied are you that the information presented in this workshop increased your awareness of the issue/subject matter?
6 Very satisfied 3 Somewhat dissatisfied
5 Satisfied 2 Dissatisfied
4 Somewhat satisfied 1 Very dissatisfied
Don’t know
b. COMMENT: ____________________________________________________________
9. a. Do you expect to refer to the information from this workshop again?
Yes No
b. Why or why not? ________________________________________________________
10. a. Would you recommend this workshop to others?
Yes No
b. Why or why not? ________________________________________________________
Public reporting burden for this collection of information is estimated to average two minutes per response, including the time for reviewing instructions, gathering information, and completing and reviewing the collection of information. Send comments on the Agency’s need for this information, the accuracy of the provided burden estimates, and any suggestions for reducing the burden, including the use of automated collection techniques to the Director, Information Collections Division, Office of Environmental Information, United States Environmental Protection Agency (Mail Code 2822T), 1200 Pennsylvania Avenue , NW, Washington, D.C. 20460; and to the Office of Information & Regulatory Affairs, Office of Management & Budget, 725 17th Street, NW, Washington, DC 20503, Attention: Desk Officer for EPA. Include the EPA ICR number and the OMB control number in any correspondence.
1 Customer feedback forms/comment cards/evaluation forms are considered to be short, 5 to 15 question forms that typically accompany, and seek feedback for a specific service (such as a training course, or “over the counter” service) or product (such as a manual, software, etc). Internet (web based) surveys also fit into this category. Mail surveys may involve more extensive questionnaires and may require more rigorous statistical sampling methodology to evaluate a certain group or groups’ perceptions about a service the Agency offers.
2 EPA interprets this to preclude any EPA purposes of regulatory development or enforcement.
3 EPA interprets this to mean random sampling of the general public in a “market research” mode.
4 For customer feedback forms and short questionnaires, a one page memorandum should be sufficient. Mail or telephone surveys making use of statistical sampling must include the statistician’s name/phone, and a brief description of the statistical aspects of the survey, such as the statistical approach, population coverage, survey design, precision requirement, and pretests/pilot tests.
5 Agency hourly wages estimates were made using the 2006 figure, step 5 for each grade.
| File Type | application/msword |
| Author | EPA |
| Last Modified By | MDSADM10 |
| File Modified | 2006-05-22 |
| File Created | 2006-04-10 |