2020 RECS Cognitive Research

OMB Control Number 1905-0186

2020 RECS Pretesting – Generic Clearance

Crowdsourcing Protocol (Instrument 2)

Instrument 2 (Crowdsourcing)

This task is part of a voluntary research study about household energy use. Your responses will be kept private and confidential.

In this task, you will sort images of household heating and cooling equipment into categories. Our goal is to learn about people’s familiarity with this equipment. The entire task should take about 4 minutes to complete. You will receive $0.59 after you complete the task and your HIT is approved.

You may not recognize some of the equipment in the images, but we would like you to take your best guess as you categorize the images. Please do not look up any information to help you complete this task. Since our goal is to learn about how familiar or unfamiliar people are with this equipment, we would rather you categorize an image incorrectly than seek outside information to guide you.

Heating Equipment Images
ASK	All respondents
The images below display various types of home heating equipment. Please sort each image according to the type of heating equipment you think it is. If you are unsure, take your best guess. CATEGORIES: Central furnace Central heat pump Ductless heat pump, also known as a “mini-split” Steam or hot water system with radiators or pipes Built-in electric units installed in walls, ceilings, baseboards, or floors Built-in room heater burning gas, oil, or kerosene Wood or pellet stove Portable electric heaters Other (please specify) DISPLAY PHOTOS OF EACH TYPE OF HEATING EQUIPMENT. RESPONDENTS WILL SORT THE PHOTOS INTO THE EQUIPMENT CATEGORIES. HALF OF THE RESPONDENTS WILL SEE A VERSION OF THE QUESTION THAT DISPLAYS PEPCO’S IMAGES FOR EACH CATEGORY; THE OTHER HALF WILL ONLY SEE THE EQUIPMENT NAME.
NEXT		END
NOTE
PROBES		None

Air Conditioning Equipment Images
ASK	All respondents
The images below display various types of home cooling equipment. Please sort each image according to the type of cooling equipment you think it is. If you are unsure, take your best guess. CATEGORIES: Central air conditioner Central heat pump Ductless heat pump, also known as a “mini-split” Window or wall air conditioner Portable air conditioner Evaporative or swamp cooler DISPLAY PHOTOS OF EACH TYPE OF COOLING EQUIPMENT. RESPONDENTS WILL SORT THE PHOTOS INTO THE EQUIPMENT CATEGORIES. HALF OF THE RESPONDENTS WILL SEE A VERSION OF THE QUESTION THAT DISPLAYS PEPCO’S IMAGES FOR EACH CATEGORY; THE OTHER HALF WILL ONLY SEE THE EQUIPMENT NAME.
NEXT		END
NOTE
PROBES		None

This is the end of the task.

This is your MTurk verification code: [DISPLAY RANDOM 8-DIGIT ALPHANUMERIC STRING]

Please enter the verification code in your HIT on MTurk to receive approval. Remember to copy your code before you close the current browser tab or window.

Thank you for your participation!

Instrument 4 (Crowdsourcing)

This survey is part of a voluntary research study to find out about your household energy use. In addition to asking you questions about your energy use, this web survey includes several open-ended questions specifically asking you to provide comments regarding how you came up with your answers to the questions. Your responses to these open-ended questions will be used to revise the survey questions. There are no right or wrong answers to the open-ended questions. Your feedback is extremely helpful for us to better refine the survey questionnaire. Your responses will be kept private and confidential.

The entire survey should take about 30 minutes to complete. You will receive $5.00 after you complete the survey and your HIT is approved.

TYPEHUQ
ASK	All respondents
Which best describes your home? Mobile home Single-family house detached from any other house Single-family house attached to one or more other houses (for example: duplex, row house, or townhome) Apartment in a building with 2 to 4 units Apartment in a building with 5 or more units
NEXT	BEDROOMS, NCOMBATH, NHAFBATH, OTHROOMS
NOTE	Question is included to characterize participants as representing single family versus multi-family households.
PROBES	None

BEDROOMS, NCOMBATH, NHAFBATH, OTHROOMS
ASK	All respondents
How many of the following types of rooms are in your home? Include rooms in finished basements and finished attics. If none, please enter “0.” ____ Bedrooms (BEDROOMS) ____ Full bathrooms (A full bathroom includes a sink with running water, a toilet, and a bath or shower.) (NCOMBATH) ____ Half bathrooms (A half bathroom includes a sink with running water and either a toilet, a bath, or a shower.) (NHAFBATH) ____ Other rooms (Include living or family rooms, kitchens, home offices, etc. Do not include hallways, closets, or rooms you already counted above.) (OTHROOMS) {ALLOW RANGE 0-30 FOR EACH. REQUIRE RESPONSE FOR NUMBER OF BEDROOMS BECAUSE IT TRIGGERS THE SOFT CHECK IN SF2.}}
NEXT	SF1
NOTE	Question is included to trigger the soft check on SF2.
PROBES	None

SF1
ASK	All respondents
A home’s size is typically reported in “square feet.” Do you know your home’s square footage? Yes No
NEXT		If SF1 = 1: SF2 Else: SF3
NOTE
PROBES		None

SF2
ASK	If SF1 = 1
What is the square footage of your home? We would rather you select ‘Don’t know” than provide a guess. ________square feet Don’t know “SOFT CHECK” WILL APPEAR FOR ALL RESPONSES OUTSIDE OF THE 25th-75th PERCENTILES BASED ON NUMBER OF BEDROOMS (SHOWN BELOW): 0 BEDROOMS: 450-800 SQ FT 1 BEDROOM: 600-800 SQ FT 2 BEDROOMS: 800-1200 SQ FT 3 BEDROOMS: 1200-2000 SQ FT 4 BEDROOMS: 1800-2800 SQ FT 5+ BEDROOMS: 2000-4000 SQ FT HALF WILL RECEIVE SOFT CHECK A: You reported that your home is [FILL] square feet. This number is [larger/smaller] than we expected based on other information about your home. Please verify your response. My response is correct. I would like to continue to the next question. My response is incorrect. I would like to edit my response. I am unsure. I would like to change my response to “Don’t know.” IF SOFT CHECK = 2, ASK FOR SQUARE FOOTAGE AGAIN. INSTRUMENT SHOULD RECORD BOTH ORIGINAL RESPONSE AND UPDATED RESPONSE. HALF WILL RECEIVE SOFT CHECK B: Please verify your response. My response is correct. I would like to continue to the next question. My response is incorrect. I would like to edit my response. I am unsure. I would like to change my response to “Don’t know.” IF SOFT CHECK = 2, ASK FOR SQUARE FOOTAGE AGAIN. INSTRUMENT SHOULD RECORD BOTH ORIGINAL RESPONSE AND UPDATED RESPONSE.
NEXT		If SF2 = DK, SF3 Else: SF4
NOTE
PROBES		How accurate do you think your response is? (Extremely accurate, Very accurate, Somewhat accurate, Not very accurate, Not at all accurate) IF RECEIVED SOFT CHECK A: The survey noted that your original response was [larger/smaller] than we expected based on other information about your home. Please tell us what you thought when you saw this message. How do you feel about this message?

SF3
ASK	If SF1 = 0 or SF2 = DK
We understand that you might not know how many square feet your home is. A home’s size is closely related to its energy use, so we would like you to provide an estimate. Please tell us which category best describes the square footage of your home. Less than 500 square feet 500-999 square feet 1,000-1,499 square feet 1,500-1,999 square feet 2,000-2,499 square feet 2,500-2,999 square feet 3,000-3,499 square feet 3,500-3,999 square feet 4,000 square feet or more Don’t know (Select this option only if you are unable to provide a guess.)
NEXT		IF SF1=1, SF4 ELSE: VACANT
NOTE
PROBES		How did you decide which response to select? Please describe your thought process. Is anything about this question confusing or unclear? Please explain. What do you think of the response categories? Do you think they should be changed, or are they okay? Please explain.

SF4
ASK	If SF1 = 1 & SF2 ≠ DK
What sources of information did you use to determine your home’s square footage? Select all that apply. I checked my property tax documents I checked my lease or mortgage documents I remember from when I leased or bought my home I asked someone else who knows I checked an internet site (e.g., Zillow) Other (please specify): _________________________________________
NEXT		VACANT
NOTE
PROBES		None

VACANT, VACANTLEN
ASK	All respondents
During the past year, was your home vacant for one or more months? Yes No Don’t know VACANTLEN (ASK IF VACANT = 1) For how many months was your home vacant during the last year?
NEXT	STOVEN, STOVE, OVEN
NOTE
PROBES	IF VACANT = 1: For what reason was your home vacant? IF VACANTLEN IS ANSWERED: How confident are you that the number of months you reported your home was vacant is accurate? (Extremely confident, Very confident, Somewhat confident, Not very confident, Not at all confident) Did you have any difficulty answering [this question/these questions]? Please explain. Suppose you moved into your home in the last year. Do you think [this question is/these questions are] asking only about vacancy after you moved into your home, or do you think [it includes/they include] both the time before and after you moved in? (Only after I moved in; Both before and after I moved in)

STOVEN, STOVE, OVEN
ASK	All respondents
PRESENT ITEMS ON THE SAME PAGE AND INCLUDE IMAGE OF EACH ITEM. STOVEN How many stoves/ranges do you have in your home? STOVE How many separate cooktops without an oven do you have in your home? OVEN How many separate wall ovens do you have in your home?
NEXT	If STOVEN>0: STOVENIND Else if STOVE>0: STOVEIND Else: TVCOLOR
NOTE
PROBES	None

STOVENIND
ASK	If STOVEN>0
Does your [If STOVEN>1: most used] stove/range have an induction cooktop? Yes No
NEXT		If STOVE>0: STOVEIND Else: TVCOLOR
NOTE
PROBES		How confident are you that your answer is correct? (Extremely confident, Very confident, Somewhat confident, Not very confident, Not at all confident) How familiar are you with induction cooktops? (Extremely familiar, Very familiar, Somewhat familiar, Not very familiar, Not at all familiar) IF FAMILIAR: In your own words, what is an induction cooktop? If you are not sure, please do not look it up—we are interested in what you think it is.

STOVEIND
ASK	If STOVE>0
Is your [If STOVE>1: most used] separate cooktop an induction cooktop? Yes No
NEXT		TVCOLOR
NOTE
PROBES		How confident are you that your answer is correct? (Extremely confident, Very confident, Somewhat confident, Not very confident, Not at all confident) ASK ONLY IF STOVENIND WAS NOT ASKED: How familiar are you with induction cooktops? (Extremely familiar, Very familiar, Somewhat familiar, Not very familiar, Not at all familiar) IF FAMILIAR: In your own words, what is an induction cooktop? If you are not sure, please do not look it up—we are interested in what you think it is.

TVCOLOR
ASK	All respondents
How many televisions are used in your home? If none, please enter “0.” ____ Televisions {{ALLOW RANGE 00-30}
NEXT	If TVCOLOR>0 : TVUSE1A, TVUSE1B Else: DESKTOP, LAPTOP, TABLET, PCPRINT, SMARTPHN, CELLPHN
NOTE	Question is included to satisfy skip logic for TV usage (TVUSE1A/TVUSE1B).
PROBES	None

TVUSE1A, TVUSE1B
ASK	If TVCOLOR>0
What is the main use of your most used television? Watching live programming Watching recorded or on-demand programming from a set-top box 3 Streaming content via built-in smart TV apps 4 Streaming content via a separate streaming device connected to the television 5 Playing video games Other (please specify): What is the main use of your most used television? Watching live, recorded, or on-demand programming 2 Streaming content via built-in smart TV apps or a separate streaming device 3 Playing video games 4 Other (please specify):
NEXT		DESKTOP, LAPTOP, TABLET, PCPRINT, SMARTPHN, CELLPHN
NOTE
PROBES		Are any of the answer options confusing or unclear? Do you use a television for any purposes not listed in the question? (EXPLAIN) Which answer would you choose if your main use was watching DVDs?

DESKTOP, LAPTOP
ASK	All Respondents
How many of each of the following are used in your home? If none, please enter “0.” ________ Desktop computer (DESKTOP) ________ Laptop computer (LAPTOP) {ALLOW RANGE 00-30 FOR ALL}
NEXT	If DESKTOP > 0: TIMEON1 Else if LAPTOP > 0: TIMEON1 Else if TABLET > 0: TIMEON1 Else: HEATHOME
NOTE	Question is included to satisfy skip logic for computer usage (TIMEON1). Additional devices are included in the cognitive testing version of this item.
PROBES	None

TIMEON1
ASK	If DESKTOP+LAPTOP>0
Thinking about the [If DESKTOP+NUMLAPTOP>1: most used] computer, how many hours each day is it used? 1 Less than 1 hour 2 1 to 3 hours 3 4 to 6 hours 4 7 to 10 hours 5 More than 10 hours
NEXT		LGTINNUM
NOTE
PROBES		Which of the following items do you think this question is asking about? (Desktop computer; Laptop computer; Tablet computer or e-reader [for example: iPad or Kindle]; Printer, scanner, fax machine, or copier; ‘Smart phones’ [for example, iPhone or Android]; Other cellular phones)

LGTINNUM
ASK	All respondents
Approximately how many light bulbs are installed inside your home? Include light bulbs in ceiling fixtures and fans, table and floor lamps, as well as those used infrequently, such as in hallways, closets, and garages. For fixtures with multiple bulbs, count each bulb separately. Fewer than 20 light bulbs 20 to 39 light bulbs 40 to 59 light bulbs 60 to 79 light bulbs 80 or more light bulbs Don’t Know
NEXT		ILTUSE
NOTE		Question is included to provide context for ILTUSE.
PROBES		None

ILTUSE
ASK	All respondents
How many of the light bulbs inside your home are turned on for… Between 1 and 4 hours per day? ______ Between 4 and 8 hours per day? ______ More than 8 hours per day? _______
NEXT	OLTUSE
NOTE
PROBES	How easy or difficult was it to answer this question? (Very easy, Somewhat easy, Neither easy nor difficult, Somewhat difficult, Very difficult) How accurate do you think your response is? (Extremely accurate, Very accurate, Somewhat accurate, Not very accurate, Not at all accurate) Please explain why this question was easy or difficult to answer.

OLTUSE
ASK	All respondents
Does your home have any outside lights? For apartments, only consider light bulbs connected to or controlled by your unit. Yes No
NEXT	If OLTUSE = 1: OLTWHEN Else: EV1
NOTE	Question is included to ease into OLTWHEN. The probe findings from OLTUSE also apply to OLTWHEN.
PROBES	When you thought about “outside lights,” did you think about holiday lights? (Yes/No)

OLTWHEN
ASK	All respondents
When do you use outdoor lights? ______________________________________________________
NEXT	OTHWORK
NOTE
PROBES	Does your use of outdoor lights vary from day to day, or do you use them consistently each day? Please explain. Does your use of outdoor lights vary from season to season, or do you use them consistently regardless of the season? Please explain.

OTHWORK, OTHACT
ASK	All respondents
50% RECEIVE VERSION 1; 50% RECEIVE VERSION 2 VERSION 1 Is there any other kind of activity occurring in your home that uses a lot more energy than would usually be used in a home? 1 Yes (please specify) (OTHACT) No VERSION 2 Are there any other devices used in your home that use a lot more energy than would usually be used in a home? Examples include hair dryers, air purifiers, water softeners, sump pumps, well pumps, medical devices, engine block heaters, power tools, woodworking or hobby equipment, aquariums, extreme holiday lights, etc. 1 Yes (please specify) (OTHACT) 0 No
NEXT	POWEROUT
NOTE
PROBES	None

POWEROUT, WHYPOWEROUT
ASK	All respondents
During the past year, has your home had a power outage lasting at least 24 hours? Yes (please briefly explain why): No [IF POWEROUT=1] Which of the following best describes why this power outage occurred? If you experienced multiple power outages during the past year, please select the reason for the longest outage. Natural disaster or weather event 2 Unable to pay electric bills 3 Electricity utility had a planned or unplanned blackout 4 Other (please specify):
NEXT	END
NOTE
PROBES	Did you have any difficulty answering either of these questions? Please explain.

This is the end of the task.

This is your MTurk verification code: [DISPLAY RANDOM 8-DIGIT ALPHANUMERIC STRING]

Please enter the verification code in your HIT on MTurk to receive approval. Remember to copy your code before you close the current browser tab or window.

Thank you for your participation!