Elwha Focus Group Summary

ATTACHMENT A. SUMMARY OF QUALITATIVE RESEARCH

Beginning in February 2012, we conducted focus groups and cognitive interviews over 13 evenings with residents of Seattle, Spokane, and Port Angeles (Washington) and Portland (Oregon). We used these focus groups to determine which attributes we should include in our survey and how best to describe them to the general public. We experimented with many graphics and choice question formats to determine which approach communicated concepts the most clearly without putting excess cognitive burden on respondents. Table A.1 summarizes the date, location, number of participants, and main goals for each round.

Table A.1. Focus group and cognitive interview summary
Date	Location	Number of participants	Main goal of groups
February 28, 2012	Seattle, WA	19	To determine what terms and concepts should be used and present two approaches to describing ecosystem restoration.
March 22, 2012	Seattle, WA	18	To test the scenario for areas needing clarification and identify gaps or superfluous material.
April 5, 2012	Seattle, WA	14	To test simplified introductory material and participant understanding of graphs.
April 17, 2012	Seattle, WA	19	To further refine graphs and scenario and test payment vehicle and ranking question.
May 15, 2012	Seattle, WA	16	To introduce reservoir site revegetation and test a new choice question format.
May 29, 2012	Portland, OR	20	To test a reorganized, shortened instrument and new choice question format.
July 10–11, 2012	Spokane, WA	36	To test a description of a keystone species, new graphics, and several versions of the choice question.
July 26, 2012	Seattle, WA	20	To test a new version of the choice question and changes to the description of the attributes.
August 7–8, 2012	Seattle, WA and Portland, OR	40	To test new formatting and graphics and alternative versions of the choice question.
September 5–6, 2012	Seattle, WA and Port Angeles, WA	39	Cognitive interviews to ensure that wording and graphics are clear and that the cognitive burden is not too high, and to test alternative versions of the choice question.

Qualitative research topics

Several themes evolved during the qualitative research phase of this study, including:

1. The role of dams in the survey

2. The best way to communicate ecological concepts

3. The attributes we included and excluded

4. The levels and measure of the attributes

5. Whether graphs were helpful or detrimental to respondents’ understanding

6. The payment vehicle

7. The format for the choice question.

The role of the dams in this survey

When we began conducting focus groups in February 2012, the removal of the dams on the Elwha River was well underway and well publicized, particularly in the Pacific Northwest, where we expected this survey to be administered. Given the likelihood that a significant proportion of respondents had heard about dam removal activities on the Elwha River, we did not want to risk developing an unrealistic baseline scenario in which the dams would not be removed. We expected that many respondents who were aware of the dam removal might find the survey confusing or untrustworthy if we told them that the dams might not be removed. To avoid this potential scenario rejection, we told participants that the dams were being removed, but that they could help to decide how quickly the ecosystem would recover after they were removed.

The subject of the dams had to be managed carefully. In early versions of the instrument, we presented considerable detail about when and why the dams were constructed and why they were now being torn down. We found that this placed too much emphasis on the dams themselves, leaving participants focused on questions related to the dams, when our goal was for them to consider restoration after dam removal. We also found, however, that too little information (e.g., simply stating, “The dams are being torn down”) left respondents with too many lingering questions. A short section of roughly a quarter-page seemed to give most respondents enough information without overemphasizing the dams.

Communicating ecological concepts

Our goal for this survey is to elicit respondent preferences for restored ecological services. To accomplish this, we had to communicate the roles that returning fish and trees play in the ecosystem. In the current version of the survey, we accomplished this by introducing and defining terms and creating informative, non-technical diagrams.

Through the qualitative research process, we learned that participants are more familiar with the concept of an ecosystem and its interconnected components than we anticipated. We did not encounter any participants who were unfamiliar with the term “ecosystem” or did not understand the concept once we defined it. Similarly, we introduced readers to the term “keystone species.” While very few participants were familiar with this term before reading the survey, all participants questioned were able to define it accurately, in their own words, after reading the survey.

We supported the notion of salmon as a keystone species using an illustration of a food web, with salmon at the center and arrows connecting it to plants and animals that depend on it for food and fertilization. Participants in the focus groups and cognitive interviews informed us that this diagram, displayed following the introduction of the term “keystone species,” helped them to consider salmon restoration for its effects on the ecosystem, rather than just on species numbers.

Although the revegetation of reservoir sites does not have as many direct linkages to other ecosystem components as do salmon, it will affect many bird and animal species. We describe this verbally and provide a supporting diagram showing the vegetation and types of animals associated with a mature forest.

Identifying relevant attributes

One of the first questions we had to address was which ecosystem services to include in the survey. Given the ecological importance of returning anadromous fish, we planned to include salmon and steelhead restoration as one of the survey’s attributes. Initially we hoped to include the restoration of important ecological processes such as nearshore estuary and beach nourishment, but the first focus groups revealed that participants’ prior understanding was so limited that it was not feasible to include these processes in this survey.

The earliest versions of the instrument thus included only salmon and steelhead restoration, and a description of their role in the ecosystem. In the initial phase, two restoration programs – fish stocking and habitat improvements – were the attributes being considered. Respondents were given the choice of doing nothing, doing one of the restoration programs, or doing both programs.

Although this approach worked well, it resulted in a survey focused only on ecosystem services related to the restoration of anadromous fish. To expand participants’ areas of consideration into different components of the ecosystem, we included revegetation of the former reservoir sites as another possible activity.

In the early focus groups, we used the individual restoration activities, alone or bundled together, as the attributes among which participants could choose. To make it more realistic and interesting, however, we changed our approach to ask them to identify the recovery time path they prefer for the restoration of salmon and the restoration of forests and wildlife. Some participants may prefer to have more ecosystem services in the long run, while others may focus more on the most restoration in the shortest period of time. We plan to use this approach for the pretest survey.

Ultimately, this will allow us to estimate a participant’s willingness to pay for a range of recovery paths for salmon as a keystone species and forests as wildlife habitat.

Measuring the attributes

Using feedback from focus group participants, we identified the most effective way to measure the attributes. For both salmon and forest and wildlife restoration, we found that comparing restored levels to historical levels is the most meaningful for participants.

For salmon restoration, we began with the annual number of spawning fish, but participants wanted to have a sense of whether this was relatively few or many fish. We then measured restoration as the percentage of the river’s current carrying capacity for salmon, but found this concept difficult to convey. We thus settled on the percentage of historical levels of fish. Because of pressure from commercial and recreational fishing and environmental factors beyond the scope of restoration efforts, the percentage of returning fish will not reach 100%. Focus group participants seemed to accept this fact, but we will test the effect of the maximum achievable level on participants’ preferences by varying it during the pretest survey.

We measure forest and wildlife restoration as the percentage of forests and wildlife that are restored to their previous condition, i.e., as they were before the dams were built. The survey states that 100% recovery means that the forests and wildlife at the old reservoir sites will return to their previous state, i.e., before the dams were built.

Using graphs

Graphs can be helpful in summarizing information, particularly restoration levels over time. However, we were concerned that graphs can be confusing for some respondents and may result in them reading the survey less carefully. Over the course of several focus groups, we found ways to make the graphs clearer and more intuitive, and to provide information to those who do not like to use graphs.

More people found the graphs helpful once we limited the information in them. To do this, we reduced the number of curves, limiting them to those that did not cross. In other words, the program with the smallest increase in salmon at five years also had the smallest increase in salmon at 100 years. We also limited the vertical lines to signposts at the particular time intervals we had discussed in the text. These steps allowed the graphs to convey the information more quickly.

To further increase the accessibility of the graphs, we added icons (i.e., a fish for salmon restoration and a tree for forests and wildlife) where each line crossed the highlighted time interval. Inside the icon we showed the number of salmon or the percentage of forest restoration in that year. This helped to make the graphs less intimidating and quickly understandable for more respondents.

Even with these refined graphs, we still found participants who were reluctant to use them. To summarize information for them, we included a table below the graph that uses the same colors and time intervals. This table has a similar format to the choice question, which helps to familiarize participants with the layout.

Payment vehicle

We are using a surcharge on electricity bills as the payment vehicle in this survey. With some refinement of the description of why and how participants would pay, we found that most found it reasonable that they would be asked to pay, and that a surcharge on their electric bill was acceptable.

We did not experiment with a sales or income tax because we hoped to implement the survey in Washington, which has no income tax, and Oregon, which has no sales tax. Previous stated preference research has demonstrated that respondents may not believe they would pay for programs through higher consumer goods prices, as they could simply buy fewer or different products.

Choice question formats

The qualitative research phase helped us to develop a choice question format that presents a sufficient amount of information and a sufficient number of choices to make the question useful without overwhelming respondents. While we have found that the format we submitted works well with focus group participants, we plan to test an alternative version in the pretest to ascertain whether the formats have statistically discernible effects on responses.

With two service categories (i.e., salmon restoration and forest and wildlife restoration) and three alternatives for each (i.e., no further actions, limited actions, and extensive actions), we had nine possible combinations of programs from which participants could choose. In the first focus groups when we tested choice question formats, we tried presenting respondents with all nine combinations and asked them to identify their most and least preferred. While some gave us well-reasoned explanations for their choices, for many this seemed to be an overwhelming task. This made us concerned that survey respondents would not carefully consider their answers. Alternatively, we presented participants with a subset of three or four alternatives, always including the “do nothing more” option and varying the combinations of “limited” and “extensive” alternatives. While the cognitive burden was much lower using this approach, many participants objected when the alternative that they preferred was not offered.

To allow participants the most flexibility, we split the choice question into two parts: one for salmon restoration and one for forest and wildlife restoration. The total cost to participants would be the sum of their two selections. By experimenting with different formats, we were able to have participants make their selection independently but understand that they would have to pay for the sum of the two selections. This approach is novel in the stated preference literature. We will test its reliability during the pretest by splitting the sample: half of the respondents will receive this format and half will receive the more traditional format in which attributes are bundled.

We also experimented with the appropriate level of information to include in the choice questions. Too little information may lead to participants not taking the question seriously, or accurately remembering what their choices implied. Too much information may lead to participants being overwhelmed and not carefully reading the table. In several focus groups, we presented participants with different versions of the choice question, containing different amounts of information. Some had only the costs associated with each alternative; others had the number of years until the maximum level would be achieved; and others presented the levels of restoration at three or four time intervals. We found that most participants preferred having the time path summarized for them so that they did not have to reference earlier parts of the survey. We are therefore using that approach in the pretest instrument.

To reduce the amount of information in the choice question, we experimented with showing percentages or levels of salmon. We found that participants were divided as to which approach they preferred. In response to participant feedback and suggestions from other researchers in the field, we chose to include both numbers in the table.

Page A-3