Updated Pretest Findings

Attachment 1
Summary of Pre-Test and Cognitive Interview Findings

The Volpe Center conducted a pre-test and cognitive interview of the eVIUS questionnaire to test question understanding and clarity. The Bureau of Transportation Statistics (BTS) and the Volpe Center (referred to jointly as the survey team) recruited five pre-testers. The pre-testers agreed to complete the online survey and to meet with Volpe Center staff to debrief on their survey experience.

The five pre-testers were comprised of one plug-in hybrid electric vehicle (PHEV) owner and four battery electric vehicle (BEV) owners, including a 2022 Mini Cooper, 2023 Tesla, 2020 Kia Mira, and 2021 Ford Mach E. Two of the pre-testers were Volpe Center staff and the other three were contacts provided by the Bureau of Transportation Statistics. In terms of demographics, the pre-testers included four males and one female, and their age range was between 35 and 74.

For the pre-test, the Volpe Center team programmed the questionnaire in Qualtrics, so that the survey experience would be similar to the full-scale eVIUS, which will be administered online. However, a couple of the questions could not be tested. For example, eVIUS will present survey participants with information about the make, model and year of the sampled vehicle and will ask them to confirm the address of the registrant (i.e., based on sampling frame data). This information was not available for the pre-test participants, so these questions were excluded.

The survey link was sent to the pre-testers starting on August 3, 2023, and the five survey debriefs (one hour each) were conducted Friday, August 4 through Friday, August 11, 2023. During each debrief, the Volpe Center team walked through the survey question by question, using a PDF showing each survey question, as programmed, and the pre-tester’s responses. The Volpe Center team encouraged pre-testers to share their feedback on the questions and/or response options, including any points of confusion or other issues.

Overall, the survey pre-testers reported that the survey was clear and easy to understand, and they did not have any major issues with the survey questions. Pre-testers reported that the survey took about 15 minutes to complete. Table 1 provides a summary of the pre-test feedback. Most suggestions involved minor wording edits that the Volpe Center team implemented. Edits that were not accepted included recommendations for new questions (e.g., fuel economy and previous EV ownership). The suggested questions were not priority topics, and the survey team did not want to add to respondent burden.

Based on pre-test feedback, one survey question - factors influencing the purchase decision (Q7) - was cut. It was unclear if the question was asking about the factors’ influence relative to purchasing a gas-powered vehicle or other EV/PHEV makes and models. Moreover, there was some confusion around a couple of the response options. Given this was not a priority question, the survey team decided it could be cut.

For another question – which asked the respondent to report the EV’s/PHEV’s range, in miles (Q19) – the question was redesigned due to pre-test feedback. The intent of the question had been to gather information on vehicle range that could be piped into follow-up questions. However, as originally worded, there was confusion over whether the question was referencing the “sticker” or “actual” vehicle range, and a respondent mentioned that some may also respond based on the number of miles they feel comfortable driving (which likely would be lower than either the sticker or actual range). Since data on vehicle range can be obtained from the Environmental Protection Agency (i.e., “sticker”), it was determined that this question was not necessary and should be redesigned to collect data on perceptions related to charging (i.e., how many miles the driver feels comfortable driving the EV/PHEV before recharging). This new question was moved to the end of this question section. In addition, in the follow-up questions, rather than piping in the value obtained from this question, respondents are asked about “trips that exceed the range of the vehicle” (i.e., words are used rather than the number).

The only other substantial edit to the questionnaire was the addition of a screener which asks respondents if they ever used a gas-powered vehicle instead of their EV/PHEV for a trip that exceeded their EV/PHEV’s total range. The original question (Q20) asked how many times a gas-powered vehicle was used instead of their EV/PHEV for a trip that exceeded their EV/PHEV’s total range. While “0” was included as a response option, the pre-tester felt that the question wording assumed that a gasoline powered vehicle was an option, which was not the case in his situation. The new screener question resolves this issue.

Detailed feedback from the pre-test, including how each comment was resolved is presented in Table 1.

Table 1. Summary of Pre-Test Feedback

Survey Question	Issue Raised by Pre-testers	Resolution
Introduction/Definitions	Simplify definitions - possibly put more technical information in hover boxes	Simplified definitions and put technical information in hover boxes.
	Show images of what charger looks like	Plan to add an image for further clarification
	Consider adding a question on previous EV ownership.	Suggestion not accepted, as this is not a priority and would add to respondent burden. However, BTS will ask the vendor (from whom the sample data is being acquired) for previous EV ownership. This information will only be available for the registered owner (not for others in the household).
	Consider adding a question on fuel economy.	Suggestion not accepted. While this would be interesting, this is not a priority question and would add to respondent burden.
7. [ALL] How influential were each of the following factors in your decision to purchase or lease this [electric/plug-in hybrid electric] vehicle? Please select one response for each item. Environmental reasons Federal tax credits State or local incentives (e.g., tax credits or exemptions, rebates, HOV lane access etc.)   Lower cost to operate or maintain than gasoline/diesel vehicles Battery range  Ease of charging at home Access to public charging Other	Unclear whether interpreting vs. gas powered or vs. other EVs (interpreted differently by respondents). One respondent suggested breaking into 2 questions.	Given the key measurement issue with this question (i.e., unclear if respondents were considering these factors relative to purchasing a gas-powered vehicle or a different EV/PHEV make or model), and the fact that this was not a priority question, the BTS and Volpe teams determined that this question should be cut.
	Could add a new response option related to “early adopter/likes new technology.”
	Include "Total Range"; this is relevant for PHEVs only.
	For “access to public charging” – some considerations were positive and others were negative (“it’s complicated”)
9. [OWNED VEHICLE IN 2023] How many months in 2023 was this [electric/plug-in hybrid electric] vehicle operated?	Simplify the language; change "operated" to "driven"	Changed “operated” to “driven”
10. [OWNED VEHICLE IN 2023] Are (or were) you the primary operator of the vehicle (i.e., drive at least half of the annual miles of this vehicle)?	Simplify the language; Change “operator” to “driver”.	Changed “operator” to driver”
15. [Q3=YES (VEHICLE IN POSSESSION)] [Q6 = PERSONAL OWN/LEASE] In addition to this vehicle, how many other personal vehicles did your household own/lease in 2023?	Person included his motorcycle but wasn't sure if he should.	Added a note to clarify that motorcycles should be included.
	One respondent who purchased her PHEV in 2023 also had two gas powered vehicles in 2023, so she listed both - BUT she has since gotten rid of them. Need to reframe the question so vehicles that were replaced would NOT be reported.	For those who still have their vehicle, ask this question “as of January 2024.” For those who disposed of their vehicle, ask this question relative to when they disposed of EV/PHEV.
16. [OWNED VEHICLE IN 2023] What is the current odometer reading of this [electric/plug-in hybrid electric] vehicle?	Encourage respondents to check their odometer for the reading; this will lead to more accurate data.	Added language to question asking respondents to please check their odometer.
	Upfront info should clarify that survey asks for odometer reading, so the respondent is prepared.	Also included information upfront indicating that survey asks for odometer reading.
19. [OWNED VEHICLE IN 2023] Approximately how many miles can this [electric/plug-in vehicle] be driven before needing to be recharged [PHEV: or refueled] (i.e., the vehicle’s range)?	Unclear if the survey is asking for the “sticker” range or “actual” range, which may be lower. In addition, there is the issue of how many miles the driver feels comfortable driving the EV before recharging (which is likely lower than the “actual” range).	Given these issues, the BTS and Volpe teams decided to cut this question, and to add a new question related to how many miles the driver feels comfortable driving the EV/PHEV before recharging. Since the number reported in this question was piped into other questions in this section, the follow-on questions were edited to no longer rely on this number. Instead, text was added referring to trips that “exceeded the range of this EV/PHEV.”
20. [OWNED VEHICLE IN 2023] Thinking about other round trips taken in 2023 that exceeded [INSERT MILEAGE FROM Q19], approximately how many times, if any, was a gasoline/diesel powered non-plug-in vehicle used instead of this [electric/plug-in hybrid electric] vehicle] due to concerns about charging enroute?	Person who only has EVs got a little tripped up – the question assumes the respondent wants to take a gas-powered vehicle (which may not be the case).	Add a screener; this would be a more neutral formulation (yes/no).
23. Public charging stations – Level 1 or Level 2 Response options: Every day; A few times per week; About once a week; A few times per month; Monthly or less often; Never (but have access); No charging access or availability; Don’t know	One respondent wasn't sure where he had access, so he responded "Don't know" - even though “Never” would have been a more appropriate response. The parenthetical - “(but have access)” - confused him.	In the response category “Never (but have access to)” – the parenthetical was deleted, so the response category is “Never.”
Public charging stations – DCFC or Tesla Supercharger	PHEVs can't use DCFC.	Remove fast charger response option for PHEVs only.
25. [Q23 = IF ACCESS AT “YOUR PRIVATE RESIDENCE”] At your private residence, do you/your household use a private charging space or a shared charging space (e.g., shared with other households in a multi-family dwelling). Please select all that apply.	Shared charger – one respondent misunderstood the response options and thought “shared charging space” referred to being shared among vehicles in same household.	Added a phrase to the response option clarifying that shared means shared among different households.
26. [Q23 = IF EVER CHARGE AT PUBLIC CHARGING STATION] For local travel, at what public charging locations do you typically recharge your electric vehicle? Please select all that apply.	People might put their workplace as a public charging station.	Respondents should NOT include their workplace unless it is a public charging place. Added a definition for “public” to mitigate measurement error.
	Add “shopping malls” to the retail store response option; shopping malls are a common place for public charging.	Added shopping malls.
27.  [Q23 = IF EVER CHARGE AT PUBLIC CHARGING STATION] Which of the following factors are most important when choosing a public charging station? Please select up to 3 factors.	Could ask this question using a scale (rather than “check all that apply”) to obtain more detailed information on the importance of each factor.	Suggestion not accepted. While this is a good suggestion, it would increase respondent burden.
	Allow to check 5, since first three will be the same for many.	Changed instruction to allow up to five selections.
28. [OWNED VEHICLE IN 2023] Where would you like to see more public charging stations? Please select up to three locations.	Add “shopping malls” to the “retail store” response option (same as feedback offered above).	Added shopping malls to the “retail store” response option.

6