Three years customer survey results

The following are brief descriptions of the surveys conducted during the past three years
under the generic clearance for customer surveys, DOE-887, (OMB No. 1901-0302).
DOE-887(67) USE OF GENERIC CLEARANCE FOR THE ENERGY
INFORMATION ADMINISTRATION’S WEB SITE CUSTOMER SURVEY
The time period that the survey was collected was August 3-August 24, 2006, three
weeks. The survey was fielded on our website so it was presented to many thousands of
customers. EIA received 4,830 responses in three weeks. The response rate cannot be
determined because there is no accurate denominator.
This was a short but meaningful survey for EIA. The survey showed that 19% of EIA
users were first-time visitors, but that 55% visited the site daily or weekly. EIA’s major
customer groups are business/industry (39%), research/consulting (14%), finance (10%),
private citizens (10%), and lower percentages for government, students and teachers, the
media, and other groups.
One interesting finding is that 34% of those who responded to our survey live outside the
United States. EIA had some indirect evidence that our international customer base was
significant, but this was the first time we collected quantitative data on this customer
aspect.
Customers were asked how satisfied they were with the quality of the information on
EIA’s website: 93% said they were satisfied or very satisfied.
A wide variety of comments, kudos, and suggestions were received from the last question
which asked “Is there anything else they wanted to tell us?” Some common themes
include:
 Customer wanted EIA data to be timelier, updated more regularly and collected
more frequently.
 Some said the site is still difficult to navigate, although it’s easier than it was
before the redesign.
 More than 50 customers said they wanted more international data. (Further
analysis showed that two-thirds of these requestors were international customers.)
 Previously, many customers complained about the poor results from EIA’s search
engine. After EIA launched its new search engine and redesigned its website,
only 9 of the 4,830 respondents even mentioned EIA’s web site search function.
Some additional results:



Regular users were more satisfied with EIA’s site than first-time users.
International users were slightly less satisfied than domestic users, although both
rating were very high.

DOE-887(68) USE OF GENERIC CLEARANCE FOR THE FEDERAL ENERGY
MANAGEMENT PROGRAM (FEMP) CUSTOMER FEEDBACK SURVEY OF
YOU HAVE THE POWER CAMPAIGN MATERIALS
The time period that the survey was collected was August 1-September 30, 2006. During
the timeframe that the survey was posted, the FEMP Web site received 47,692 visitors,
any of whom could have completed the survey as it was available to any visitor to the
site. In addition, an email was sent to approximately 350 You Have the Power
(YHTP)coordinators letting them know about the survey and encouraging their
participation. Fifty-nine person participated in the survey. Based on the YHTP mailing to
coordinators, our response rate was about 1.5%.
It is difficult to apply the results of this survey to the broader audience because the
response was so limited. However, among those responding they found the YHTP
information and resources to be very useful and the overwhelming majority of
respondents were somewhat satisfied (18 respondents) to very satisfied (39 respondents)
with the YHTP materials and campaign. FEMP also received useful information on how
the materials are used and information on how many people see the YHTP materials
within their agencies. Lastly, FEMP wanted to make sure that it was reaching its target
audience of federal energy and facility managers and all but three respondents identified
themselves as working for a federal agency. At least 37 identified themselves as either an
energy manager, facility manager, or resource efficiency manager indicating that FEMP
is reaching its intended audience.

DOE-887(69) USE OF GENERIC CLEARANCE FOR THE ENERGY
INFORMATION ADMINISTRATION ANNUAL ENERGY OUTLOOK (AEO)
AND INTERNATIONAL ENERGY OUTLOOK (IEO) CUSTOMER SURVEYS
The time period that the IEO customer survey was collected was during January, 2007.
The AEO customer survey was conducted during March 2007. The IEO survey was sent
to 318 operating e-mail addresses. The AEO was sent to 450 operating e-mail addresses.
For the IEO, 61 persons participated. For the AEO, 62 persons participated. The
response rate for the IEO was 19.2%; for the AEO 13.8%. The low figure for the AEO
was somewhat due to system problems associated with this web survey.
Since EIA has been conducting these surveys for several years now with no changes in
the survey design; comparisons could be made with previous survey results. It was found
that there were no statistically significant differences between the previous responses and
the current responses for any substantive question for either the AEO or IEO. Since the
level of satisfaction was relatively high previously, EIA found the current responses
satisfactory.
The office that produces the AEO and IEO were providing with a written report
summarizing these findings and lists of narrative comments by respondents (while

maintaining the anonymity of the respondent). That office then considered making
changes based upon the comments and other findings.
DOE-887(70) USE OF GENERIC CLEARANCE FOR THE DOE NATIONAL
RENEWABLE ENERGY LAB USER FEEDBACK SURVEY FOR THE
HYDROGEN ANALYSIS MODELING TOOLS ON THE DOE HYDROGEN
PROGRAM WEB SITE.
The survey was collected from January 22, 2007 through February 2, 2007. There were
647 persons contacted and 11 person participated. The response rate was 1.7 percent.
The DOE Hydrogen Program's hydrogen analysis production modeling tools, which
assess the hydrogen cost for central and forecourt hydrogen production technologies,
were first released in 2005. The questionnaire was designed to solicit user feedback in
order to identify the most pertinent update needs as well as the areas for usability
improvement. This feedback played a key role in preparing for a comprehensive update
of these tools and their accompanying case studies.
DOE-887(71), “USE OF GENERIC CLEARANCE FOR THE OFFICE OF
ENERGY EFFICIENCY AND RENEWABLE ENERGY WEB SITE CUSTOMER
SURVEY
The survey was collected during the time period from May 1 to May 14, 2007. No
one was individually contacted for this survey. Instead, it was advertised in three ways:
(1) a feature on the EERE Home Page (2) a popup on 14 EERE sites, and (3) via EERE’s
weekly electronic newsletter EERE Network News. We calculated that the survey
invitation was displayed 46,647 times -- 28,139 times through the EERE home page
feature and 17,509 times through the pop-up box on 14 EERE subsite home pages. (We
did not include the number of subscribers to the EERE Network News because many of
those readers also visit the site and we did not want to double count.) However, the
**actual** display rate is likely less because this simple calculation does not account for
double-counting on the EERE home page (i.e., when a user was exposed to both the
feature and the pop-up at the same time).
456 people completed the survey. The survey form itself was accessed 888 times.
This translates to a 2% response rate based on home page traffic, a 3% rate based on the
pop-up window, and a 1% rate based on the total of the two. These numbers were
significant departures from the 10%–22% response rates we experienced in 2004. This is
likely due to the fact that we did not offer a gift to people who completed the survey this
year.
Summary of results:
Overall, survey responses were once again positive. Seventy-four percent of respondents
indicated satisfaction with the site content and its functionality. In addition, the 2007
results also showed remarkable consistency with past survey results. Although variations
are evident from year to year, the historical comparisons demonstrate an overall stability
of user opinion regardless of site changes or growth.

However, there is room for improvement, and several themes emerged. Survey results
suggest the following actions to improve the site:


Decrease site complexity - users appear to be having increasing difficulties
finding their desired information. This is likely related to a number of factors.
Among them is the massive growth of content and the outdated site architecture.
 Improve search engine results - The search engine currently employed by the
EERE Web site has a strong history of valid results and good performance.
However, as the site grows, the usefulness of its search results is diminishing.
Although the search engine’s functionality has not changed, users are becoming
frustrated with the increasing mass of documents returned in search results.
Largely, this is a result of site growth.
 Make content more timely to catch and hold user interest - New information
drives traffic and encourages repeat visits. Although new information is
constantly available on the site, it is often difficult to locate.
 Continue efforts to encourage action - Although survey results show improvement
in this area, more can be done to encourage action. Certain areas of the site, such
as the Consumer subsite, have incorporated features and functionalities to assist
site users. However, other areas, most notably the program sites, have lagged
behind. Few pages in these areas are focused on action.
EERE is now embarking on a redesign and is using these findings to direct what items
need to be addressed in the redesign.

DOE-887(72) USE OF GENERIC CLEARANCE FOR THE ENERGY
INFORMATION ADMINISTRATION’S WEB SITE CUSTOMER SURVEY
The time period the survey was collected was July 16-August 6, 2007. The survey was
fielded on our website so it was presented to many thousands of users. EIA received
5,049 responses. The response rate cannot be determined because we do not have an
accurate denominator.
Summary of results:
We confirmed, as we had learned from past surveys, that our largest customer group is
business and industry, with strong additional segments from citizens, finance, academia,
research and consulting, and government. About half of those who responded to the
survey said they come to our website weekly, while 17% identified themselves as firsttime visitors. A total of 69% said they live in the United States, confirming our results
from the 2006 survey that about 1/3 of our web customers are international.
Ninety-one percent of respondents said they were satisfied or very satisfied with the
quality of the information on our site. Students and citizens, although still very satisfied
with the quality of our information, were relatively less satisfied than business and
industry, finance, and the media. First-time users are less satisfied than repeat users,
probably because they have more difficult finding what they are looking for.

A little more than half (56%) of respondents said they easily found what they were
looking for on our website, with 21% saying they found their information with some
effort. Only 6% said they didn’t find what they were looking for. Our customers told us
they are starting to use Web 2.0 technologies, with about 20-25% responding that they
had used wikis, blogs and RSS somewhere on the internet.
The most frequent open-ended comment was a compliment or kudo. Many fewer
respondents than in past surveys had negative comments about our site navigation, our
internal search function, and the timeliness of our information. Suggested improvements
included more writing for the web and more customer-friendly presentations. Customers
had many specific suggestions for EIA, including things such as collecting more types
and providing more detail and forecasts for many different energy statistics, providing
more maps and graphs showing information about the energy industry (supplies,
transportation, demand), and having more explanations and details (especially for
acronyms and technical terms).

DOE-887(73) USE OF GENERIC CLEARANCE FOR INDUSTRIAL
TECHNOLOGIES PROGRAM SURVEY OF TRAINING AND SOFTWARE
RECIPIENTS
Telephone surveys were conducted between August 1 and September 4, 2007. Two
separate sets of surveys were conducted, one with Training recipients and one with
Software recipients. For the Training survey, we started with a set of 933 eligible
individuals. This number was eventually reduced to 807 because a number of individuals
were removed from the sample after someone else from their facility was interviewed
about the same topic. Approximately one third of the eligible individuals could not be
contacted, either because they no longer worked for the company contacted, the
telephone number was wrong or disconnected, or the call was answered by a machine or
voice mail. For the Software survey, we started with 606 eligible individuals and that
number was eventually reduced to 594 for the same reason given for Training. Close to
one third of the eligible individuals could not be contacted due to the same circumstances
described above for Training. Surveys were completed by 347 Training recipients and
206 Software recipients. The 347 completed Training surveys represent 43% of the 807
eligible individuals. The 206 completed Software surveys represent 35% of the 594
individuals who were eligible for that survey.
Summary of results:
Training recipients reported that the training they received was beneficial overall and that
it was well worth the registration fee. The specific items to which they gave the highest
ratings were the performance of the instructor and the helpfulness of the course materials
and handouts. The scores given by software recipients were not as high as those given by
those who received training, but they did express agreement with the statement that the
DOE software was beneficial overall.

Regarding the helpfulness of possible additional features or formats, training recipients
gave their highest ratings to more customized assistance, more hands-on exercises, more
examples of projects that utilize the technologies addressed, and continuing education
courses, with the lowest scores going to more financial information and web-based short
courses. For software recipients, the highest marks were given to adding capability to
simulate systems dynamically and allowing more flexibility in system design options,
while the lowest score went to offering a metric version of the software.

DOE-887(74) USE OF GENERIC CLEARANCE FOR THE FEDERAL ENERGY
MANAGEMENT PROGRAM (FEMP) WEBSITE CUSTOMER SURVEY
The FEMP web site survey was conducted from August 1, 2007 – September 30, 2007.
The survey was posted on the FEMP home page and was voluntary. During this time
period 9,567 people visited the FEMP home page and 175 people took the survey. This is
a response rate of 1.829% of total visitors to the site’s home page.
Summary of results:
From the responses received from the survey, we are able to clarify who are the active
users of the site, and what they use the site for. Their responses also provided insight into
areas and aspects of the site that should remain a higher priority, in regards to updates,
due to frequent use and applicability to the needs of energy managers. The results of the
survey will continue to be used to improve and make the web site more user-friendly.
DOE-887(75) USE OF GENERIC CLEARANCE FOR THE DOE OFFICE OF
ENERGY EFFICIENCY AND RENEWABLE ENERGY WEB SITE CUSTOMER
SURVEY
The survey was collected from March 25 to April 22, 2008. This survey was conducted in
order to gather the opinions of the readership of the EERE Network News (ENN), a
weekly newsletter run by the U.S. Department of Energy's Office of Energy Efficiency
and Renewable Energy (EERE). No one was individually contacted for this survey.
Instead, it was advertised in the HTML and RSS versions of ENN on March 26 and April
9, and was available on the ENN Web site from March 25 to April 22. Statistics are not
available for the number of people who regularly read ENN through the Web site or RSS
feed. However, ENN's email subscription numbered 16,298 on April 23, 2008. 874
people took the survey. Out of the 874 respondents, 830 self identified as subscribers to
the email version of ENN. Out of its 16,298 subscribers, this totals a response rate of
approximately 5%.
Summary of results:
The information from this survey is being used to gauge the opinion of ENN's readership
toward the quality and content of the ENN Newsletter. The survey also revealed valuable
information about how ENN is used by its readership—what they use the information for
and why it's important to them in their personal and professional lives. While the overall
responses were positive, the responses also suggest some improvements or developments

that could be made to the newsletter. One of the most clear-cut findings was that many of
ENN's RSS users were interested in having multiple, specialized RSS feeds.
The information is being used internally to guide the future development of the ENN
Newsletter. A report is currently being developed that summarizes the responses to the
survey and presents conclusions based on the findings. When this report is complete, it
will be posted on the EERE site and ENN readers will be invited to read the responses.

DOE-887(76) USE OF GENERIC CLEARANCE FOR THE EIA 2008
CONFERENCE CUSTOMER SURVEY
The survey was collected from April 11 – May 2. The number of persons contacted were
1,050, but 440 actually participated. This yielded a response rate of 42 percent.
Summary of results:
The conference received very good ratings from respondents. The conference topics were
relevant and met expectations. Many respondents of this survey indicated interests in
specific topics and pointed out areas for improvements. The information collected from
this survey will be used for the planning and improvement of Annual EIA Energy
Conference.

DOE-887(77) USE OF GENERIC CLEARANCE FOR INDUSTRIAL
TECHNOLOGIES PROGRAM ENERGY MATTERS E-MAGAZINE CUSTOMER
FEEDBACK SURVEY
The survey was posted on the ITP Web site from April 15, 2008 through May 15, 2008,
and contained 16 questions. An e-mail message inviting participation in the survey was
sent to approximately 4,000 readers who subscribe to Energy Matters. In addition, a
feature button linking to the survey was posted on the ITP home Web site, as well as the
main Energy Matters and BestPractices Web sites. A news article announcing the survey
and encouraging participation was also posted on the ITP home page, as well as a
reminder announcement in the E-Bulletin. There were 156 responses to the survey (4% of
total distribution list). This is an appropriate response rate; the 1999 survey had a
response rate of 5%, and the 2002 survey response rate was 4.5%.
Summary of results:
The results of the survey demonstrated a favorable view of Energy Matters. Following
are highlights of the results:


Approximately 94% of the respondents subscribe to Energy Matters. This shows a
high level of readership, and that most of the respondents came from the subscriber
base. As they are regular subscribers, they are familiar with Energy Matters and

better able to provide historical perspective and solid opinions about what type of
information they need to meet their energy efficiency challenges.


More than 95% of respondents found Energy Matters to be "very" or "somewhat"
useful, and many provided specific reasons as to why they found it useful. Almost
70% of respondents reported that they read "every" or "several features" of each
issue.



The demographics also suggest that Energy Matters is successful in reaching its
intended audience. Forty-one percent of the respondents are engineers, with
management personnel following at 23%. All of the key industries are represented,
with representatives from the Chemicals industry registering as the highest
percentage. The highest overall percentage was the "Other" category, which included
mostly industry-related professionals.



Almost 85% of the respondents indicated that the information in Energy Matters is
presented at an appropriate technical level.



It is particularly important to gauge the level of interest in various topics. The topic
that received the highest interest level was energy management strategies, followed
by technology R&D, software tools, case studies, and system optimization tips. This
feedback confirms that Energy Matters meets the need as a hands-on, practical
resource for the industrial audience. ITP will consider this feedback as it continues to
develop appropriate articles for its audience.



The actions taken as a result of reading Energy Matters are particularly compelling in
showing the readers find value and credibility in the information they read. The
survey results indicate that readers are directly applying the information they obtain
from the e-magazine. A high percentage of respondents revealed that they have used
information in formulating energy management strategies, downloaded software
tools, implemented new technologies, or alerted their management personnel to
articles. In addition, Energy Matters has added value as a pass-along resource: 46%
of respondents replied that they cite, reprint, or reuse information, while 23% forward
the link to others.



The majority of respondents (92%) indicate that they like the new online format.

DOE-887(78), “USE OF GENERIC CLEARANCE FOR THE ENERGY
INFORMATION ADMINISTRATION’S WEB SITE CUSTOMER SURVEY”
The survey was collected during the time period from July 21-August 6, 2008. The
survey was fielded on our website, so it was presented to many thousands of users. EIA
received 5,635 responses. The response rate cannot be determined because we do not
have an accurate denominator.

Summary of results:
We confirmed, as we had learned from past surveys, that our largest customer group is
business and industry, with strong additional segments from citizens, finance, academia,
research and consulting, and government. About a third of those who responded to the
survey said they come to our website weekly, while an additional third identified
themselves as first-time visitors. We had a larger share of first-time visitors (mainly
citizens) this year than in previous years, possibly because gasoline prices were $4 per
gallon at that time and citizens were looking on the Internet for the first time for
explanations. A total of 76% said they live in the United States, slightly higher than in
previous years but confirming our results from the 2006 and 2007 surveys that a large
slice of our web customers are international.
Ninety percent of respondents said they were satisfied or very satisfied with the quality of
the information on our site. Students and citizens, although still very satisfied with the
quality of our information, were relatively less satisfied than business and industry,
research and consulting, and finance. First-time users are less satisfied than repeat
users, probably because they have more difficult finding what they are looking for. A
total of 82% of first-time visitors said they were very likely or somewhat likely to come
back to our website. Customers who found what they were looking for on our website
were more satisfied with the quality of the information than those who did not find what
they were looking for.
About half (49%) of respondents said they easily found what they were looking for on
our website, with 22% saying they found their information with some effort. Only 7%
said they didn’t find what they were looking for. Nearly 80% said the level of detail on
the website was just right, with 11% saying it was not detailed enough.
Respondents most frequently selected the words informative, objective, expert, thorough,
and policy neutral to describe EIA. Terms least selected to describe EIA (all at 1% or
less) were long-winded, old fashioned, boring, and non-responsive.
When asked where the largest share of U.S. imported oil comes from, 48% said Canada
(the correct answer), but 24% said Saudi Arabia. Even with a list of countries to choose
from, 17% of respondents said they didn’t know.