2484supp01 - Appendix 1

Appendix 1: Review Comments and Federal Register Comments Received

June 10, 2013

To: Matt Weber

From: Kerry Smith

Subject: Arizona River Valuation Survey

As promised, I went through the survey. It is a very interesting project. I was really impressed by all of the documentation developed in planning the survey. There were a number of other aspects I liked in the research design. However, I will focus my comments on the issues where I felt there was some need for changes. I have listed them in order of importance:

1. The payment vehicle is not explained well or justified. Which taxes will go up and why? Why will the water be reduced in the “expected future”? You suggest on page 6 it is to reduce the amount of water used by the river ecosystem. What is the alternative use for the water that is not going to the ecosystem services? This is not explained. How will the proposed changes be maintained and why does it cost more to maintain a system that is the current baseline (without the water quality improvements)? These issues are not adequately explained. Until they are, as a reader I don’t know how or why this ends up increasing my taxes indefinitely. People know that taxes depend on income –how do you know what they will pay before you know their income? I realize in a mail question it cannot be dynamic in the way envisioned by this comment. Nonetheless it suggests the tax payment vehicle must be explained carefully.

2. There are too many choice panels. I would explain the number respondents get and have no more than two or three. Respondents are not told each panel is independent of the one before. Since it is expected to be mailed, you need to allow in your experimental design that they will read ahead. It is essential that they cannot find the same option or a better option at a cheaper cost. With most conventional experimental design software programs, the blocking process for selecting choice sequences will not include this constraint. I have had to manually inspect all design panels in previous studies to be sure “better” options are not included as part of the randomization process.

I realize that this affects the assumption of randomization but respondents read ahead. It is the bundle of choice sets you give them that have to be considered as an integrated whole.

3. I felt the choice set panels were too dense. I also felt “expected future” needs to be more carefully identified as what will happen if they do nothing. This is too implicit for me. Also, when is the expected future? What is the time horizon for this to happen? The timing and details on page 6 are too vague.

4. A key element is using these conjoint or CV questions is that respondents believe their answers will be consequential. This follows from the Carson-Groves argument in ERE and Vossler and others recently in the AEJ supporting it. Your discussion on page 1 gives the impression opinions will be used in selecting “best” options. I believe the statements need to be stronger – results based on respondents’ votes will be presented to policymakers and their decisions consider constituents desires. Explanation must be consistent with alternative uses for the water that are motivating the change from the status quo.

5. For me, the description of ecosystem services and situation with and without flow is inadequate. All one knows is the new miles of river and forest acres that is it. A little text that there are other “stuff” affected but little details.

6. The term “Full body contact” in my opinion is terrible. Explain what you mean in plain English. Water will never be deep enough for swimming, but one could wade, raft and not be concerned if you felt in and went underwater. You also need to be specific about current water quality, fish live there, some other species are supported, “no full body contact” is inadequate.

7. As you note reminding respondents what to consider on page 12 – telling them the questions display “different tradeoffs” is inappropriate. Questions display alternative choices – don’t tell them to make tradeoffs. We assume they do but that is an assumption on our part.

You say there are three voting questions. I saw four plus an example. None of the explanation on this page was adequate. It really needs to be re-worked.

8. You indicate in your discussion of recreation trips per year. You need to give the context in the text – trips in a typical year or trips last year; where do they go most often?

9. Question 14 is too direct – you are using economic explanations rather than what people might use to explain their motivations.

(10) I would want varialbes to characterize respondents:

• education

• employment status

• own a home

• boat ownership

• more income categories (and leave out “estimated”)

• define what you mean by income and then ask them to give their best assessment

I realize most of my suggestions are written in fairly direct terms. I don’t mean them to be critical of you—I just wanted to emphasize their importance. I also wrote this quickly.

Finally, I did not get into issues with the application of that conjoint framework for valuation and the implicit assumptions about estimating WTP versus MWTP for changes in an attribute. You did not have much detail on this in what you sent. It might be worth sketching what you hope to derive from the model for policy analysis before finalizing the survey.

Arizona Stream Issue Questionnaire

Issues and Questions

John Hoehn

June 23, 2013

The questionnaire overall is very well put together. The text is readable and information. The sequencing

of sections and questions is logical. The diagrams and graphics are clear and helpful in understanding the

text and issues.

The key questionnaire is how your respondents react to and understand the issues in the questionnaire. I

assume that you’ve either tested or have plans to test the questionnaire with respondents like those in your

sample. With that assumption out of the way, I do have some comments on the questionnaire. None are

game changers but they are offered as issues to consider before testing and finalizing the questionnaire.

Here they are:

1. Page 3. What is treated wastewater? What’s meant by “this was just a method of disposing of”?

I think I understand, but I’m not sure. I suppose treated wastewater is sewage that’s been through some type of treatment facility? How much treatment? Is it discolored and smelly? Does it contain harmful bacteria, viruses or chemicals that may hurt plants, animals or me if I come into contact with it? Is there another term that could be used— recycled water maybe?

2. Pages 3 to 12 are a little much in terms of explanation overload. The 10%/90% information

seems unnecessary to selection the options. Page 5 doesn’t seem relevant either. Page 6 could be edited to reduce the amount of info and focus attention on options and issues in a choice table. The Expected Future lays out what you get for no additional cost. Options A and B are also possible if people are wtp. The Expected Future and the Options differ in the following:

a. Miles of streambed flooded

b. Acres of cottonwood/willow ecosystem

c. Whether the water is safe enough for full body contact

d. How the N and S are affected

e. Cost

The cost issue is somewhat confusing to me. How does one get more water by simply payingmore? Water has to come from someplace—will it raise water prices where I live? Can you offer a little more explanation?

3. Page 7. It seems like what’s represented is with and without water flow. Why not just label the

scenes accordingly—With and Without?

4. Page 9 Why is a sample table necessary? Respondents have probably already read ahead and

know what’s coming. Respondents typically skim through a questionnaire before deciding to take the time to answer it.

5. Page 13-15 Option B in page 15 seems like a real bargain compared to option A on page 16 when I’m indifferent to north or south flows.

This raises a question—is the N/S question issue really important enough for the added

complexity. Water for the south seems cheap which seems a little surprising with Tucson in the

N. Why is water for S so much cheaper than water in N?

Would it be better to focus the cost/service tradeoffs on total flow miles, forest acres and body contact while leaving the N/S question as a separate allocation question? Are there returns to

scope or scale in provision that make the N/S question significant? If not, it might be better to go

with a simpler structure by eliminating separate rows for N and S flows, acres and body contact.

Hi Matt,

As we discussed on the phone just now, Emily and I are very supportive of your work on the “Willingness to Pay” survey focused on the Santa Cruz River. The results of this work will be very helpful for Sonoran Institute’s Santa Cruz River Initiative. Understanding community values regarding natural resource management is critical for prioritizing on the ground actions, crafting policy, leveraging additional funds.

We found this version of the survey to be much improved from the earlier draft you shared. We have some specific comments/edits regarding the language in the survey that I list here in order by page:

• Page 2 – the italicized intro language makes it seem like these are real options that authorities are considering. We suggest a rewording of the third sentence. Instead of “Your responses will help authorities to select the best option.”…we suggest something more like: Your responses will help authorities to better understand public opinion regarding management of the river.

• Page 6 – the language is misleading regarding the cut back to river flows in the “expected future”. Seems river flows would be reduced because managers think the river uses too much. Rather flows will be reduced because effluent may be used for other purposes. Instead of “In the future, river flows are expected to be cut back to reduce the amount of water used by the river ecosystem.” ….perhaps something more like this: In the future, river flows are expected to be cut back as effluent is used for other purposes rather than released into the riverbed. To me this is more accurate.

• Page 6 – reference to voting in the Flow & Forest description implies that the outcome is really up to the survey respondents.

• Page 9 – provide an explanation as to why certain boxes within the questions are shaded. We assumed these were items that did not change from the expected future. This should be more explicitly stated.

• Page 12 – there is a typo in the first sentence. There are 5 voting questions, not 3.

• Page 12 – the language in the third bullet referencing voting again implies that these are real options that people are voting for. Perhaps reword to say something like: Review and answer each question carefully. Understanding your opinion will help authorities to best manage the river.

• Page 18 – minor formatting typo under question 9, first bullet. The text “Never chose” is not the same font as the subsequent questions.

Thank you for the opportunity to review and comment on this work. We would be happy to discuss this work further with you.

Claire and Emily

Claire Zugmeyer

Ecologist

The Pima County Regional Wastewater Reclamation Department appreciates the opportunity to comment on the proposed EPA public opinion survey (EPA-HQ-ORD-2013-0282-0001) regarding future options for the water quality, water quantity, and other environmental benefits of the effluent-dependent reaches of the Upper and Lower Santa Cruz River.  Our comments will focus on the Lower Santa Cruz River as that is the reach managed by various Pima County Government entities and utilized for effluent discharge and aquifer recharge by this Department.

The survey approach is to present the public with several potential future outcomes for the water quality, water quantity and habitat (“forest cover”) of a reach of the River (with photos and graphic illustrations) and ask that survey recipients choose from several incremental, per-household tax amounts that participants would be willing to pay for these outcomes.  Although we are very interested in public perception and opinion related to water quality of this effluent-dependent portion of the Lower Santa Cruz (LSC) River, we nevertheless have significant concerns about the survey methodology and approach, background information provided, and the alternatives presented to the public in this survey.  Highlights of our concerns are as follows:

• The survey approach assumes that the outcomes listed are feasible given the amount of funding stated.  In fact, the legal, scientific and physical circumstances of these effluent-dependent flows combine to make many of the outcomes highly unlikely (if not nearly physically impossible), such as the river to meet “full-body contact” water-quality standards.  For more information about the complex and unique issues of water quality and water quantity for effluent-dependent waters in the arid west, we recommend that the survey personnel review the Arid West Water Quality Research Project, which took place from 1995 to 2007 and which was funded and supervised by EPA Region IX. If necessary, we can provide copies of reports from this effort.

• The survey does not  provide adequate background to the public as to the nature and impact of the over $600 million in water-quality improvements to its two major metropolitan Water Reclamation Facilities (WRFs) that Pima County is currently constructing in coordination with the Arizona Department of Environmental Quality. These facilities provide the effluent flow in this reach of the Lower Santa Cruz River and will have state-of-the-art water-quality treatment processes and advanced odor control for which Pima County ratepayers are totally responsible for paying through increased sewer user charges. We can provide more details about this effort.

• The future alternatives presented in the survey neglect the economic and environmental benefits of using the water beneficially off-channel as reclaimed water or in-channel for aquifer augmentation.

• The survey approach includes sending the survey to selected residents in the Phoenix and Tucson Metropolitan areas, but the effluent-dependent flows of the Santa Cruz do not extend to the Phoenix areas and those residents are generally unaware of the River at all. Furthermore, people living along the Upper Santa Cruz River in Santa Cruz County are not surveyed. Therefore, half of the targeted survey recipients who have no connection to the Santa Cruz will be surveyed, while others with a direct connection will not.

In closing, the Department would recommend that, prior to proceeding further with this survey, EPA staff discuss the survey formally and fully with the Pima County Government management and staff to reconcile these and other discrepancies. We feel that addressing these concerns will lead to an improved survey and research study.