Fuel Economy_Supporting Statement B_rev 012712

Supporting Statement for Fuel Economy, GHG, Other Emissions, and Alternative Fuels Education Program
Quantitative Research Plan

Section B

January 27, 2012

Contents

B. Collections of Information Employing Statistical Methods 3

B1. Describe the potential respondent universe and any sampling or other respondent selection method to be used. Indicate the expected response rates for the collection as a whole. 4

B2. Describe the procedures for the collection of information 6

B3. Describe methods to maximize response rates and to deal with issues of non-response. 7

B4. Describe any tests of procedures or methods to be undertaken. 7

B5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency. 8

2. Collections of Information Employing Statistical Methods

NHTSA is seeking approval for one survey instrument to gauge awareness and understanding of information related to fuel economy, alternative fuels, GHG and other emissions and to test potential advertising in order to inform a consumer education program focused on these topics.

This survey will be conducted one time with a sample of 1,500 drivers from all 50 states and the District of Columbia who meet the following criteria:

• Respondents must be 18 years or older.

• Respondents must currently possess a valid driver’s license.

• Respondents must currently own or lease a vehicle.

• Respondents must be the primary or a shared decision maker for vehicle purchases in their household.

The purpose of this survey is to explore awareness of, understanding of and engagement with the above issues, understand motivations for behavior change, test messaging and explore potential communications channels for a consumer education campaign.

Data Analysis Plan

During data analysis, we will use a variety of statistical techniques to assess awareness and understanding of fuel economy related messages. The purpose of this analysis is to inform NHTSA on what consumers know about fuel economy and what topics they are most interested in so that NHTSA’s education campaign can prioritize messages to be most effective and engaging.

In the message testing portion of the survey, respondents will be randomly presented with one of five online advertisement storyboards for which they will rate it on believability, relevance, uniqueness and effectiveness of its call to action (i.e., the resulting likelihood to visit SaferCar.gov). To analyze the effectiveness we will use ANOVA to test for variance between evaluations on key measures so that we can assess which advertisement stands out as most effective.

After this initial evaluation, respondents will be shown the other advertising storyboards in a random order and will evaluate these on the same measures. As a result, we will be able to analyze the data both in terms of unbiased rating (i.e., comparing responses of those who evaluated each storyboard first) and overall rating (i.e., total respondent evaluations for each storyboard).

1. Describe the potential respondent universe and any sampling or other respondent selection method to be used. Indicate the expected response rates for the collection as a whole.

For reasons explained in Part A of this supporting statement, NHTSA will conduct an online survey to explore consumer knowledge and interest in fuel economy related content as well as test potential advertising that will drive consumers to seek out this information. This methodology was chosen as it provides respondents with the opportunity to evaluate images, which is essential for the message testing portion of the study. Since these graphical treatments contain English-language messages, the survey will only be available in English to ensure respondents understand the messages being displayed. Issues of non-response due to this methodology are addressed in section B

The sample will be built using members of an online panel managed by Authentic Response (www.authenticresponse.com), which provides NHTSA with a potential respondent universe of 2 million U.S. consumers. This panel has been built using a passive panel sourcing process. Authentic Response works with publishers and online service providers to embed recruitment efforts into their websites. Potential panel members will only be admitted into the panel after a validation process is complete. For quality purposes, all panel members are de-duplicated at the registration stage and a continuous system of validation checks is in place to prevent professional survey takers from entering studies.

Upon panel registration, members are asked demographic and lifestyle questions so that surveys can be targeted. Using this information, the survey will be e-mailed to U.S. panel members ages 18 and older. We expect approximately 20,000 invitations to be sent to randomly selected panel members on a rolling basis throughout the fielding of the study in order to obtain the 1,500 respondents necessary to complete the study.

Before entering the questionnaire, each respondent will be screened based on the criteria listed below. If qualified, the respondent then enters the survey and begins answering questions relevant to the needs of this study. A full text of the draft survey questionnaire is included in this package as Appendix D.

Upon completion, the respondent will be redirected to the panel provider’s website. This survey will be a one-time implementation, and respondent identification information will be removed prior to data analysis to preserve anonymity.

The sample will be built to consist of percentages of US drivers ages 18+ by age, gender and geographic location proportionate to the data provided in the Federal Highway Administration’s 2009 Highway Statistics. We will impose hard quotas for age ranges, gender, and regions and soft quotas for states within each region.

Table 5. Region Definitions

Region	States
Deep South	Alabama, Arkansas, Florida, Georgia, Louisiana, Mississippi, South Carolina
Farm Belt	Iowa, Kansas, Missouri, Nebraska, North Dakota, South Dakota
Great Lakes	Illinois, Indiana, Michigan, Minnesota, Ohio, Wisconsin
Mid Atlantic	Delaware, District of Columbia, Maryland, New Jersey, New York, Pennsylvania, West Virginia
Mountain	Arizona, Colorado, Idaho, Montana, Nevada, New Mexico, Utah, Wyoming
New England	Connecticut, Maine, Massachusetts, New Hampshire, Rhode Island, Vermont
Outer South	Kentucky, North Carolina, Oklahoma, Tennessee, Texas, Virginia
Pacific	Alaska, California, Hawaii, Oregon, Washington

The sample will be gathered through the following steps:

1. Respondents will first have to qualify as adults ages 18 and older.

2. Next, respondents must qualify as someone who does not work or have a family member working in the automotive industry, an ethanol or alternative fuel producer, or the marketing or market research industry.

3. Next, respondents must qualify as a self-reported licensed driver.

4. Next, respondents must qualify as a vehicle owner or lessee.

5. Finally, respondents will be screened to ensure they are the primary or shared decision maker in their household when it comes to vehicle purchase decisions.

Based on the Federal Highway Administration’s 2009 Highway Statistics, there are approximately 205.8 million licensed drivers ages 18 and older¹. Since we are using a convenience sample for this study, we plan to use the below statistics to set gender, age and region quotas for our sample of 1,500.

Table 6. Expected Population and Sample Distribution by Key Demographics

Gender	Total Population	% of Licensed Drivers 18+	Expected Distribution of Sample (n)
Male	102,320,242	49.7%	746
Female	103,461,215	50.3%	755
Age
18-24	23,538,942	11.4%	172
25-34	36,326,817	17.7%	266
35-44	38,158,133	18.5%	278
45-54	41,665,892	20.2%	303
55-64	33,156,841	16.1%	242
65+	32,934,832	16.0%	240
Region
Deep South	33,614,591	16.3%	245
Farm Belt	10,456,411	5.1%	77
Great Lakes	35,321,339	17.2%	258
Mid Atlantic	31,899,373	15.5%	233
Mountain	14,792,314	7.2%	108
New England	10,718,373	5.2%	78
Outer South	36,390,094	17.7%	266
Pacific	32,588,962	15.8%	237

Estimated Incidence

Based on past research conducted on behalf of NHTSA, we anticipate 85% of general consumers qualified as vehicle owners who are primary or shared decision makers for vehicle purchases.

NHTSA will not present data from this survey as representative of the general population. However, it is important for the agency to develop an understanding of the size of the target audience as the consumer information program is developed.

2. Describe the procedures for the collection of information

The procedure for the collection of information for this study is as follows:

• Survey will be sent via e-mail to online panel members identified as vehicle owners ages 18+.

• The questionnaire will take approximately 20 minutes to complete. All panel members who are vehicle owners over the age of 18 have an equal chance of receiving the survey invitation, but only those who qualify based on screening criteria will have the opportunity to complete.

• A total sample size of 1,500 will be interviewed. Respondents will complete the survey on their own time, and will have an option to contact the panel provider if they experience any technical problems with the survey.

• Data tables, including important cross-tabulations, will be prepared along with a final report of the key findings and strategic recommendations.

The sampling approach described above will yield a convenience sample. This sampling methodology results in a study that only includes respondents who have access to computers and the Internet, and are literate. According to Pew Research Center’s Internet and American Life Project², 77% of US adults access the Internet. Because the study is not a probability-based sample, there is no statistical basis to derive unbiased estimates representative of the target population, U.S. passenger vehicle owners or lessees, or to estimate sampling error.

For the purposes of this study, it is sufficient that the sample be a convenience sample as long as it is diverse in terms of drivers’ gender, age and region. The key component of this study is that the respondents be randomly assigned to different conditions. NHTSA believes that the study design will provide the quantitative estimates to effectively inform the development of a consumer education campaign. In addition, this methodology is a cost- and time-effective way of conducting the research necessary to inform this program and is the most feasible methodology that allows for the testing of graphical online advertising storyboards.

Additionally, planned communications channels will include web sources and the main source of this fuel economy-related content will be a web property like SaferCar.gov. We will not ask respondents to rank sources, as there may be an inherent bias toward web-sources with these respondents, but we will be obtaining evaluations from a group of consumers who are able to access these sources once published. Therefore, the difference between respondents and non-respondents will likely not impact the guidance of consumer information dissemination.

3. Describe methods to maximize response rates and to deal with issues of non-response.

As detailed in B1 and B2, we are utilizing a convenience sample. In order to ensure diversity in respondents, we will set quotas to ensure the sample consists of percentages proportionate to the US driver target audience by gender, age and region based on the data provided in the Federal Highway Administration’s 2009 Highway Statistics. Since a goal of this study is to make comparison s between different messages, it is essential that respondents are randomly assigned to view these different conditions.

4. Describe any tests of procedures or methods to be undertaken.

Pilot Test

NHTSA and its contractors will conduct a test of the online survey to ensure all functionality is working properly before the study is sent to potential respondents.

Once proper functionality of the survey program is confirmed, NHTSA will conduct cognitive testing of the survey instrument with consumers in the Richmond metro area. The purpose of this testing is to ensure the questionnaire wording is clear and comprehendible for consumers and will help ensure that we are collecting quality data in order to meet our objectives. Cognitive testing will be conducted by trained research moderators and will provide the participant the ability to take the survey on a computer and discuss how they are interpreting questions and whether or not they understand what is being asked.

Based on these results, minor edits to the survey instrument may be made to improve the clarity and comprehension of survey question language.

Soft Launch Period

The survey will experience a ‘soft launch’ period in which a limited number of potential respondents (approximately 5% of the sample) are sent invitations to respond. During this period, we will evaluate the demographic data, completion rate, and time-spent in the survey. Any problems discovered during this test period will be addressed before the survey is sent out to a larger population.

Data Collection and Analysis

Responses will be collected in a computerized database. Data processing will consist of tabulation of quantitative and coded open-ended responses using UNCLE software. Data analysis will be conducted by NHTSA and its contractors, StrategyOne. Since individual differences are randomly distributed across conditions, NHTSA plans to use standard statistical techniques to test observed effects between different designs. Summary statistics will be analyzed to determine whether or not significant differences exist between the reported effectiveness of the messaging being tested.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The company selected as a contractor for this study is StrategyOne. StrategyOne is the market research arm of Edelman, Inc. This team has extensive experience in both qualitative and quantitative research practices. More information can be found at their website (http://www.strategyone.com). The contact information for this team is as follows:

Jason McGrath Vice President 202.326.1810	StrategyOne 1875 Eye Street NW Washington, D.C. 20006
Ellen Rienzi Senior Project Manager 202.350.6687
Tiffany Scott Research Assistant 202.336.7976

1 http://www.fhwa.dot.gov/policyinformation/statistics/2009/dl20.cfm

2 The Pew Research Center’s Internet & American Life Project, November 23-December 21, 2010. n=2,303 adults 18 and older, including 748 reached via cell phone. Interviews were conducted in English and Spanish. Margin of error is +/- 2%.

NHTSA Fuel Economy, GHG, Other Emissions

and Alternative Fuels Education Program Quantitative ICR Package

January 27, 2012