Energy Efficiency and Renewable Energy Website Usability Test

Generic Clearance for the Collection of Qualitative Feedback on Agency Service Delivery

EERE Ux test materials for OMB 2_22_2012

Energy Efficiency and Renewable Energy Website Usability Test

OMB: 1910-5160

Document [docx]
Download: docx | pdf

OMB Control Number: 1910-5160

Expiration Date: 6/30/2014



OMB Burden Disclosure Statement

Public reporting burden for this collection of information is estimated to average 15 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this estimate or any other aspect of this information, including suggestions for reducing this burden, to Information, Records, and Resource Management, MA-41-GTN, Paperwork Reduction Project (1910-5160), U.S. Department of Energy, Washington, DC 20874-1290; and to the Office of Management and Budget (OMB), Paperwork Reduction Project (1910-5160),Washington, DC 20503.

Energy Efficiency and Renewable Energy Website Usability Test

Method:

  • We will administer our remote, unmoderated usability test of the EERE website using User Zoom, an online usability testing tool (a short video showing how the tool works is available here: http://www.userzoom.com/products/online-usability-testing). No personally identifiable information will be collected during this test.

  • Potential test participants will receive an email with a link to the usability test. Specific instructions for taking the test are provided by User Zoom at the beginning of the testing process.

  • Each voluntary participant will be asked to answer:

    • 3 pre-test questions

    • 4 exercises where they will look for particular information on the EERE website and then answer a follow up question

    • 4 post-test questions about their experience using the site.

Note that all participants will answer the same pre- and post-test questions. However, the set of 4 exercises each participant is asked to complete will be randomly generated from a bank of all the exercises that are appropriate for that participant.

  • The test will take each participant an estimated 15 minutes to complete.

Pre-Test Questions:

These questions will be asked of each participant at the beginning of the test.

  1. What best describes the reason you typically visit the EERE website?

I want to…

You may be a…

Reduce my energy bills or use green energy at home.

  • Interested member of the public

  • Homeowner

  • Student

Save energy and therefore money at my organization or company.

  • Energy manager

  • Operations supervisor

  • Engineer

  • Business owner

  • Fleet/Transportation manager

  • Government employee (not from Department of Energy)

Help my clients save money and energy.

  • Consultant

  • Energy auditor

  • Engineering advisor

  • Energy analyst

  • Building contractor

Do cutting edge energy research and have an impact on the field.

  • Scientist

  • Inventor

  • Researcher

  • Graduate student

Inspire and educate others about energy efficiency and renewable energy technologies.

  • Teacher or professor

  • Green advocate

  • Non-profit organization

Invest in the emerging technologies in energy efficiency and renewable energy.

  • Entrepreneur

  • Business owner or executive

  • Investor

I want to: _________________________________
_________________________________

Other __________________________

  1. How would you describe your level of knowledge about energy efficiency and renewable energy?

|-----------------------------------------------------------------------------------|

Beginner (0) Expert (10)



  1. How often do you visit this website?

    1. Daily

    2. Every few days

    3. Weekly

    4. Every few weeks

    5. Monthly

    6. Every few months

    7. Yearly

    8. This is my first time

Exercises:

This is the bank of exercises from which 3-4 will be automatically selected for each participant depending on their answer to the first test question (which asks about their typical reason for visiting the site).

All groups

Task:

Find ….

Scenario

Destination

Answer

Interested public

Innovators

Energy implementer

Advisors

Ambassadors

Investors

  1. Contact information

You want to email someone for information regarding hydrogen storage.


Find an email address for someone you could correspond with.

http://www1.eere.energy.gov/site_administration/feedback.html

Complete web site contact form


x

x

x


x

  1. Financial incentives

*start at energy.gov

You are interested in grants for energy efficiency and renewable energy research and development.


Find out what grants are currently available.


http://www1.eere.energy.gov/financing/business.html

SunShot

User gets to “Financial Opportunities by Audience” page


x

x

x

x

x

  1. Tools / Calculators

You want to purchase a new car that will help you save money on gas and be better for the environment. You are trying to decide between two cars.


Find out if there is a tool to help compare the two options.

http://www.afdc.energy.gov/afdc/calc/


x

x

x

x

x

x

  1. About EERE

*start at energy.gov

You want to learn more about the Office of Energy Efficiency & Renewable Energy.


Find out what initiatives the Office of Energy Efficiency & Renewable Energy is currently working on.

https://www1.eere.energy.gov/office_eere/initiatives.html


x

x

x

x

x

x

  1. Event information/Educational activities

You recently read a news story about an event in Washington D.C. where students compete to build the best solar-powered houses.


Find out when the Department will be hosting its next event.

http://energy.gov/solar-decathlon then

http://www.solardecathlon.gov/about.html

2013

x

x

x

x

x

x

  1. Maps > Wind

You are researching the best places for wind turbine farms.


Find out which U.S. state or region would make a good location for a wind farm.


http://www.windpoweringamerica.gov/wind_maps.asp


Wind Map

X

X

X

X

X

X

  1. Multimedia

You’re interested in learning more about wind power.

Find a video on how wind turbines work.

http://www1.eere.energy.gov/multimedia/video_wind_turbines.html

or http://www1.eere.energy.gov/wind/wind_animation.html

or

http://www.energysavers.gov/your_home/electricity/index.cfm/mytopic=10501


x

x

x

x

x

x

  1. Local data


*start at energy.gov

Find out how much solar energy your town or city can generate.


Energy.gov

Or


http://www1.eere.energy.gov/analysis/maps.html > http://www.nrel.gov/gis/images/map_pv_national_lo-res.jpg

Enter zip code in masthead

x

x

x

x

x

x

  1. News > Tidal

Lately you’ve been hearing about the potential for hydropower.


Find recent information on the potential of Wave and Tidal Energy production.

http://www1.eere.energy.gov/water/news_detail.html?news_id=18017

DOE Reports show major potential for wave and tidal energy this century.

x

x

x

x

x

x

  1. Email updates/Progress alerts

You want to stay in touch with what’s new in energy efficiency and renewable energy technologies.


Find out where to sign up for email updates.

http://www.eere.energy.gov/ - email sign up on home page


http://apps1.eere.energy.gov/news/progress_alerts_archive.cfm


https://public.govdelivery.com/accounts/USEERE/subscriber/new?topic_id=USEERE_2

Two answers, but they take you to two different newsletters.

x

x

x

x

x

x

  1. Career > General

Find out about job opportunities in green technologies.

http://www1.eere.energy.gov/education/clean_energy_jobs.html >


http://www1.eere.energy.gov/solar/careermap/


http://www1.eere.energy.gov/office_eere/careers/job_vacancies.html


X

X

X

X

X

X

  1. Rebates > Appliances


You are considering purchasing a new dishwasher for your home in Los Angeles.


Find out if you can get money back for purchasing an energy-efficient appliance in California.

http://www.energysavers.gov/financial/rebates/state_CA.cfm

Yes

X






  1. Tax credits > Funding for homeowners


You recently installed a geothermal heat pump in your home. You are interested in seeing what tax credits are available.


Find out how much of a tax credit you can get.

http://www1.eere.energy.gov/office_eere/faqs.html#home

Answer: 30% of the cost of the system

X






  1. Science fair project


You’re helping a student research ideas for a science fair. The student is interested in making energy from plants.


Find a science fair project idea for making energy from grass.

http://www1.eere.energy.gov/biomass/for_students.html

Answer: ‘Which grass produces more biomass?’

X




X


  1. Reducing energy use at home

You want to use less energy in your home. Find out how this site recommends you assess how much energy you currently use.

http://www.energysavers.gov/your_home/energy_audits/index.cfm/mytopic=11160
http://www.energysavers.gov/your_home/energy_audits/index.cfm/mytopic=11170 http://www.energysavers.gov/your_home/energy_audits/index.cfm/mytopic=11180


X

X

X

X

X

X

  1. Reducing energy use in the community/Green Schools

You want to help schools in your town use less energy.


Find out what information is available to help schools use less energy.

http://www1.eere.energy.gov/education/school_buildings.html


X


X

X

X


  1. Lesson Plan

You want to introduce grade school students to concepts about solar energy.


Find something on the site to help you build a curriculum that includes solar energy.

http://www1.eere.energy.gov/education/lessonplans/plans.aspx?id=267

Yes, they have solar lesson plans for grades K-4

X




X


  1. Internship

You are helping a college student look for an internship in clean energy research and gain practical work experience.


Find out about internships opportunities with the Office of Energy Efficiency and Renewable Energy.

http://www1.eere.energy.gov/office_eere/careers/internships_fellowships.html


Yes, they have student volunteer internships at the EERE headquarters in Washington DC.

X




X


  1. Training > Installing Solar

You want to learn more about training opportunities for those seeking to install solar technology.

Find information about training opportunities for the installation of solar technologies.

http://www1.eere.energy.gov/solar/educator_resources.html



X




X


  1. Vehicles

You are interested in learning more about hybrid cars that are powered by electric motors and fuel.


Find an explanation of how hybrids work.


http://www.eere.energy.gov/basics/vehicles/hybrid_electric_vehicles.html


http://www.fueleconomy.gov/feg/hybridtech.shtml



http://www.afdc.energy.gov/afdc/vehicles/electric_basics_hev.html


X




X


  1. Funding Opportunities for Inventors

You are interested in developing renewable energy technologies and want to know if you can get funding for your project.


Find out the types of funding that the Office of Energy Efficiency and Renewable Energy offers to those who develop clean energy technologies.

http://www1.eere.energy.gov/financing/inventors.html

  • Small Business Innovation Research Program (SBIR)

  • Small Business Technology Transfer program (SBTR)

  • Advance Research Projects Agency-Energy (ARPA-E)


X


X


X

  1. Energy Use in Industry

You have an idea for a project that may help petroleum refiners save energy. You are interested in learning more about this industry and its energy practices.


Find a synopsis of current trends in the petroleum refining industry.

http://www1.eere.energy.gov/industry/petroleum_refining/pdfs/profile.pdf


http://www1.eere.energy.gov/industry/petroleum_refining/analysis.html

Find and open the energy and environmental profile for the petroleum industry.


X




X

  1. Energy Plus

Your town is building a new City Hall, and you’ve been asked ensure the building is energy efficient.


Find a tool that can help you model the new building’s use of energy and water.


http://apps1.eere.energy.gov/buildings/energyplus/

The software tool is Energy Plus and can be downloaded from the website



X

X



  1. Emerging Technologies

You want to learn about the latest technologies in energy efficiency and renewable energy. Find a list of emerging technologies that the Department of Energy labs and their partners are working on.

http://techportal.eere.energy.gov/category.do/categoryID=17


Yes, there is a table of emerging technologies with descriptions.


X




X

  1. Patent

You are interested in filing a patent application for a geothermal project. You want to learn about existing patents.


Find a list of geothermal patents

http://techportal.eere.energy.gov/ , http://techportal.eere.energy.gov/search.do


Yes, you can perform a search for patents in the Innovation portal


X


X


X

  1. Find information on a specific technology (CHP)


You work in a paper plant and are looking for technologies that recover waste heat from manufacturing processes to generate electricity.


Use the site to find the latest research in this field.

http://www1.eere.energy.gov/industry/distributedenergy/rd.html


Locate the Distributed Energy research and development page.



X

X



  1. Fuel Cell technology

You have heard that fuel cell technologies are useful for a variety of reasons.


Find out more about how fuel cell technologies are being used.

http://www1.eere.energy.gov/hydrogenandfuelcells/applications.html

DOE is applying fuel cell technology to specialty vehicles, emergency backup power, and prime power for critical loads.

X





X

  1. Training > Cool Roofs

You want to learn more about the benefits of cool roofs.

Find a previously recorded online training about cool roofs.

http://www1.eere.energy.gov/buildings/webinar_archives.html#coolroofs_20100422

Download the presentation slides, download the fact sheet, or listen to the Webinar recording

X


X

X

X


  1. Wind trends

You are researching trends in wind energy and you want to find information about installed project costs and wind power prices.


Find the latest report that outlines wind energy trends.

http://www1.eere.energy.gov/library/default.aspx?Page=9

2010 Wind Technologies Market Report



X

X


X

  1. Energy consumption

Find out how energy consumption has changed in the US over the past several decades.

http://www1.eere.energy.gov/ba/pba/intensityindicators/total_energy.html



X


X



  1. Energy Saving Assessment

You represent a small manufacturing plant and would like to get some help evaluating the plant’s current energy use in order to save money and power.


Find out if the plant qualifies for an energy audit.

http://www1.eere.energy.gov/industry/bestpractices/iac_eligibility.html

IAC eligibility requirements



X

X



  1. Events > Implementer

You manufacture solar panels, and want to explore potential markets on American Indian lands. You’ve heard about an upcoming conference that would allow you to network with American Indian organizations.


Find out when and where the conference is.

http://www1.eere.energy.gov/wip/events.html

Answer: RES2012 February 27 - March 1, 2012, Las Vegas, Nevada



X

X



  1. Case Studies

You’ve been asked to look for ways to cut electrical costs for lighting in the parking garage at the Department of Labor. You want to find out how others may have solved this problem.


Find an example of how another group has implemented a street lighting project.

http://www1.eere.energy.gov/femp/technologies/ssl_casestudies.html





X

X



  1. Awards

Your represent a business that has made a great deal of progress reducing its energy use. You want to promote the plant's achievement to the public. Find out if DOE offers any awards or recognition to highlight the plant's achievements.

http://www1.eere.energy.gov/industry/saveenergynow/recognition.html




X

X



  1. Cross Technology

You run a utility in Colorado and are considering adding renewable energy technologies to your portfolio of offerings. Find out which technologies would be most feasible to offer in your area.


Varies; would like to see how they approach a question that requires researching multiple technologies.



X




  1. EERE Programs

You’ve been asked to upgrade a shopping mall to reduce its energy use. Find out if the Department of energy offers a program that can provide funding and assistance to help you with your project.


http://www1.eere.energy.gov/buildings/commercial_initiative/partnerships.html

Commercial Building Initiative partnerships



x




  1. Sunshot and solar innovation

You want to learn more about the history of the solar technologies industry.

Find information about key milestones in the development of the solar industry.


http://www1.eere.energy.gov/solar/sunshot/timeline.html

Solar Innovation timeline

x

x

x

x

x

x

  1. Biodiesel

You’re interested in learning more about biodiesel.


Find out what biodiesel is.

http://www.afdc.energy.gov/afdc/fuels/biodiesel.html


http://www.eere.energy.gov/basics/renewable_energy/biodiesel.html


x

x

x

x

x

x

  1. Weatherization grant guidance

You received a grant from EERE to weatherize businesses in Maryland and now you need to close your grant.


Find the rules for closing your grant.

http://www1.eere.energy.gov/wip/pdfs/wap_closeout_guidance.pdf

Rules are available in the PDF



x

x



Post-Exercise Question (Sauro, 2009):

This question will be asked after each exercise is either completed or abandoned.

Success (for users who said they successfully completed the task)

Very difficult






Very easy

Overall this task was:



Abandon (for users who quit before finishing)

I did not finish this task because:

    1. I could not find the answer

    2. I felt this task did not apply to me

    3. I already knew the answer

    4. I ran out of time

    5. I was interrupted

    6. Other

Post-Test Questions:

These questions will be asked of each participant after they have completed the 5 exercises, and will be used to better understand their reaction to the site as they reflect on their experience using it.


  1. System Usability Scale (Sauro, 2009) (Brook, 1996)

Strongly Disagree

Somewhat Disagree

Neutral

Somewhat Agree

Strongly Agree

I think I would like to use this website frequently.

I found the website unnecessarily complex.

I thought the website was easy to use.

I think that I would need the support of a technical person to be able to use this system.

I found the various functions in this website were well integrated.

I thought there was too much inconsistency in this website.

I would imagine that most people would learn to use this website very quickly.

I found the website very cumbersome to use.

I felt very confident using the website.

I needed to learn a lot of things before I could get going with this website.


  1. American satisfaction ratings (Claes Fornell, 1996) (www.theacsi.com)

  1. Is there any feedback you’d like to share with us on your experience?

Shape1



  1. How likely are you to recommend this website to others? (Reichheld, 2003)

File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
File Modified0000-00-00
File Created0000-00-00
© 2024 OMB.report | Privacy Policy