0756 SS Part B _11.May.2018.clean

SUPPORTING STATEMENT

Preliminary Case Study Assessing Economic Benefits of Marine Debris Reduction

Regional Pilot Study

OMB CONTROL NO. 0648-xxxx

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g. establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample are to be provided in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved.

The potential respondent universe consists of visitors to each coastal study location who are 18 years of age or older. Onsite intercept surveys at local beaches will be used to find respondents for the primary mail survey. The onsite intercept survey includes demographic questions, attitudinal questions about marine debris, and questions about participation in single or multiple-day trips. The last question asks respondents if they would be willing to participate in a future mail survey. For those who decline to participate in the onsite interview entirely, we will ask their age, and record their gender and the reason they did not participate. For those who participate in the onsite intercept survey and agree to participate in the mail survey, we will record their name and mailing address. For those who complete the intercept survey but do not agree to participate in the mail survey, we will record their ZIP code in lieu of their name and mailing address.

We anticipate that 67% of those approached will agree to participate in the onsite survey and 33% will agree to receive a mail survey. We will thus need to approach 3,432 potential respondents to obtain the desired 1,144 addresses to administer the mail survey. Assuming a 35% response rate for the mail survey, we expect to receive 400 completed surveys (Table 4).

Table 4. Expected number of intercept and mail surveys for each study area
Sample area	Number of onsite intercept surveys	Expected number of survey mailingsa	Expected number of completed surveys
Great Lakes	572	286	100
East Coast	572	286	100
Gulf of Mexico	572	286	100
West Coast	572	286	100
Total	2,288	1,144	400
a. Surveys will be mailed to those who complete the onsite intercept survey and agree to participate in the mail survey.

An OMB-approved pretest was conducted in Orange County, CA, from September 2017 through January 2018. The response rates for each stage of the pretest survey are shown in Table 5. We used the American Association for Public Opinion Research (AAPOR) RR1 definition for calculating response rates. The response rate for the onsite survey was 36.2% and the response rate for the mail survey was 17.4%. The cumulative response rate for the two stages was 6.3%.

Table 5. Onsite, mail, and cumulative response rates for the pretest
Stage of survey	Number	Percent
Onsite survey
Sampled beachgoers	777	100.0%
At least two mail-survey demographics known	743	95.6%
Five mail-survey demographics known	504	64.9%
Valid addresses obtained	281	36.2%
Mail survey
Valid addresses	281	100.0%
Returned surveys	49	17.4%
Cumulative response rate		6.3%

The necessity of obtaining and analyzing pretest results in time to begin the full survey in the summer of 2018 led us to conduct the pretest mail survey on a schedule that overlapped with the holiday season, which may have affected response rates for the mail survey. If we conduct the full mail survey in September 2018, we expect the final response for the mail stage of the survey to be 35%, based on previous experience. Using our assumption of a 33% response rate for the onsite survey (slightly lower than in the pretest), this would result in a cumulative response rate of 11.6%.

A two-stage onsite and mail survey has the disadvantage that respondents drop out at both stages, leading to a low cumulative response rate. This two-stage sampling approach was essential given the need to contact a sample of all people who visit the beach, which includes people taking overnight trips from throughout the county. A nationwide mail survey to identify beachgoers from the general population would be prohibitively expensive.

The two-stage survey also has advantages with regard to nonresponse that may offset the potential effects of low response rates. Specifically, the first stage allows us to capture some information about the great majority of people in the sample. Specifically, we will have the ability to reweight the mail survey respondents to match nearly the entire sample with respect to both gender and age, since both of these variables will be recorded for 96% of the onsite sample. We will also have the ability to match on five demographic variables for 65% of the sample, namely, those who complete the onsite survey. This will allow us to reduce any bias that may result from nonresponse in a way that specifically focuses on the population of interest, namely, people who go to the beach. This kind of adjustment would not be possible in a one-stage mail survey since characteristics of beachgoers are not available from any outside control source, such as U.S. Census data.

To further adjust for nonresponse, we have revised the onsite survey to include four attitudinal questions from the mail survey. This will allow us to reweight the mail survey observations so that responses to the attitudinal questions, along with the five demographic variables, can be matched to the 65% of the sample that responded to the onsite survey. The specific questions will include asking respondents the importance of three beach characteristics: free parking, no crowds, and no garbage or manmade debris. The onsite survey will also include the question of whether respondents feel garbage or manmade debris is a problem on local beaches.

The pretest yielded estimates with reasonable precision for Orange County beachgoers. The estimated change in trips from doubling debris loads was a 14.0% decrease in trips, with a confidence interval of 13.4% to 16.4%. The change in trips from a decrease in debris to “almost none” was an increase of trips by 4.0%, with a confidence interval of 3.4% to 5.5%. Confidence intervals were estimated using a jackknife variance procedure. The targeted number of 100 completed surveys in a given study area should yield greater precision than the 49 observations obtained in the pretest.

2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Statistical methodology for stratification and sample selection

As described in Part A, we will begin sampling procedures by selecting multiple beaches in each of the four study locations. The beaches selected will represent various types of beach experiences available, including more- and less-developed beaches. Because this approach relies on judgment to achieve representativeness rather than probability-based sampling, there is some uncertainty involved. The degree of accuracy needed is discussed below.

When sampling at the selected beaches, we wish to ensure that approximately 50% of respondents take multiple-day (or overnight) trips. The reason for this target is that a sufficient number of respondents taking multiple-day trips is important for characterizing switching by beachgoers between regions, which is a critical aspect of the Deepwater Horizon model (English et al., In review). It is also important for estimating the number of overnight hotel stays, which is necessary for the regional economic analysis. Conversely, targeting a sufficient number of local beachgoers who are likely to take more trips and visit more beaches in their region, is important for characterizing garbage and marine debris at as many beaches in each region as possible. If, during this initial onsite sampling, it appears that the proportion of those taking overnight trips is significantly less than or greater than 50%, we will adjust the sampling rates accordingly. When evaluating whether an adjustment is needed, we will assume that anyone engaged in a day trip at the time of the onsite survey takes only day trips to the area being sampled. When evaluating the adjustment we will also account for the effects of choice-based sampling, described below. To illustrate the type of adjustment that could be made, consider the case where only 25% of respondents take overnight trips during the initial sampling of a given region. This would mean there are three times as many people taking day trips as there are taking overnight trips. To reach the target of an even split, we would adjust the sampling fraction to one-third for those taking day trips.

Onsite sampling is a form of choice-based sampling, where the choices of selected respondents affect their probability of entering the sample. In this study there are three components of choice-based sampling. The first is the length of time a respondent spends at the beach on a given day. The second is the number of days the respondent spends at the beach during a given trip, which is one for day trips but could be more for overnight trips. The third is the number of trips the respondent takes to the given sampling area during the year. Data on the length of time at the beach and the number of days spent at the beach for a given trip will be collected during the onsite interviews. These questions will be asked with respect to the trip taking place at the time of the interview, and the responses will be viewed as a random draw from all trips the respondent takes. Data on the number of trips a respondent takes to the relevant sampling area during the course of the year (September 2017–August 2018) will be collected in the mail survey.

Estimation procedure

As discussed in Part A, the primary research goal is to quantify the relationship between marine debris and the number of trips to beaches in the study areas. Any change in beach visits caused by potential changes in marine debris, expressed as a percentage, will be used as input to economic models. The models will estimate the impacts of marine debris on the value of recreation and the regional economy. A secondary goal is to compile response statistics on questions in the survey that do not involve a change in trips, such as what types of garbage and debris respondents typically see on the beach, respondents’ demographic characteristics, and other questions.

For all survey results, the statistical estimation procedure will use a weighted average of a respondent’s answers, where weights account for the sample selection factors described above. For example, a respondent who spent three days at the beach on his or her overnight trip, took two trips during the year, and spent four hours at the beach on the day he or she was intercepted, will have a sampling weight that is the product of one-third, one-half, and one-fourth. If the sampling rate for those taking day or overnight trips is adjusted, the weights used in the estimation will also include a factor that is the inverse of the sampling rate at the time of the onsite interview. The sampling weights will be used to compile respondent statistics at the study areas only. The Program does not intend to extrapolate its study results to the national level.

The model estimation procedure relies on a nationwide travel cost model of coastal recreation. The model was developed by experts for NOAA and other Federal and State trustees in the Deepwater Horizon oil spill assessment (English et al., In review). A travel cost model involves a system of demand functions, where price is the cost to individuals of traveling to a given site and quantity is the number of trips individuals take to the site. In the Deepwater Horizon model, travel cost is calculated using an average of airfares and driving costs, depending on the distance traveled and the proportion of people traveling by air or car for any given distance. Travel cost also includes the value of time spent traveling. The structure of the model is nested logit, with coastal beaches grouped into 76 model sites covering all coastal areas of the continental United States, including the Great Lakes. In response to an environmental change, the model accounts for the change in the value of trips to a given site, the switching of trips between a given site and alternative sites, and changes in the total number of trips to all sites. The recreation data used in the model come from a sample of 41,716 respondents living throughout the continental United States. Additional details about the Deepwater Horizon model and data can be found in English and McConnell (2015), Herriges (2015), and Leggett (2015). While the final Deepwater Horizon model focused on sites in the southeast United States, data were collected for trips to all beaches throughout the country and this more comprehensive data will be used for the marine debris model.

Since the Deepwater Horizon model already estimates the total number of trips to each site, we will not rely on the Regional Pilot Study survey to estimate the total number of trips in the selected coastal locations. Instead, we have defined the coastal locations so that they match sites in the Deepwater Horizon model. A percentage change in trips due to changes in garbage or marine debris, estimated using the contingent behavior questions from the Regional Pilot Study survey will be applied to total trips at the relevant model sites. The resulting change in total trips is the information the model requires to estimate the change in value.

Degree of accuracy needed

The survey for the Regional Pilot Study will be used to calculate average statistics rather than totals. For example, the average percent change in the number of trips is the key result for the analysis of economic value. Statistics for the total change in the number of trips are not required because the Deepwater Horizon model (English et al., In review) that will be used to calculate economic value already includes estimates of the total number of trips. A percent change from the Regional Pilot Study survey will be applied to the Deepwater Horizon totals, which simplifies the weighting procedures. Specifically, we will not calculate weights to extrapolate from trips at the several sites sampled to total beach trips in the region. We also will not calculate weights that extrapolate from the times when onsite sampling is conducted to all times when beach recreation occurs. These weights, which would be required to estimate statistics reflecting totals, are not required for the average statistics to be used in this study.

The largest source of potential inaccuracy is the sampling of a small number of beaches at one point in time during the recreation season. To achieve more accurate representativeness, probability-based sampling would require a random selection of a large number of beaches and a large number of sampling times throughout the recreation season. The cost of such an effort was determined to not be warranted given that we require only average statistics characterizing respondent choices rather than statistics quantifying the total number of trips.

Specialized sampling procedures

We will not employ any specialized sampling procedures, other than the onsite sampling methods described above.

Periodic data collection

The data collection effort will gather information for a full year of recreation activity in one survey effort. There will be no periodic data collection.

3. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.

A number of measures will be implemented to maximize the response rate, including:

• A short beach intercept survey (~ 4 minutes) will identify individuals willing to participate in a mail survey.

  • Onsite interviewers will recruit potential participants from multiple beaches within the study area. Interviewers will have background information to share with respondents about the study to provide potential participants context and credibility for the research.

  • The intercept survey will be administered via computerized tablets to minimize respondent burden and transmit data in real-time.

• The initial mail-survey packet will contain an introductory letter informing respondents about the survey and encouraging their participation by a specific date. All letters will include the NOAA logo and will be signed by the Chief Scientist of the NOAA Marine Debris Program.

• The survey will be sent via first-class mail and will include a self-addressed, postage-paid return envelope to facilitate response.

• One week after sending the initial survey, a thank you/reminder postcard will be mailed to all sampled households thanking them for responding and encouraging them to complete the survey if they have not already.

• One week after sending the first thank you/reminder postcards, another thank you/reminder postcard will be sent to all sampled households who have not yet responded to encourage their survey completion, and provide them with information to request another copy of the survey if it has been lost or misplaced.

• Three weeks after sending the initial survey, a replacement survey will be mailed to all sampled households who have not yet responded via first class mail. The replacement survey will include a letter with a final reminder to complete the survey, a second questionnaire, and a self-addressed, postage-paid return envelope to facilitate response.

• All survey materials were carefully crafted to provide a pleasing appearance that encourages response. Questions are kept short and the total number of questions was minimized, given the research needs. An attractive, color map of local beaches is included with each survey instrument.

A potential alternative to a mixed-mode data collection effort includes a web-based survey and an in-person survey. However, existing probability-based web panels (e.g., GfK Knowledge Networks) would have inadequate sample sizes at the county level, and the cost associated with completing an in-person survey at a study location would be much higher compared to a mixed mode survey. While it would be possible to provide a Web URL that allows mail survey respondents to complete the survey over the internet, recent research has found that providing an internet option in a mail survey does not improve response rates relative to a mail-only approach (Messer and Dillman, 2011; Medway and Fulton, 2012; Dillman et al., 2014). The potential for nonresponse bias will be assessed by comparing the demographic characteristics and responses to attitudinal questions of those who did not agree to take the mail survey during onsite intercepts, those who agreed to but did not return the survey, and those who completed the survey. If substantial differences are observed, sampling weights will be developed through sequential post-stratification (e.g., raking), so that the weighted demographic totals for the survey data align with corresponding totals for the surveyed region (Battaglia et al., 2004).

Table 6 presents information from the Regional Pilot Study pretest that relates to both a nonresponse analysis and the potential for a nonresponse adjustment. The table shows four demographic variables that are available for most people in the target sample, namely, for the 504 people who did not refuse our onsite survey out of a total of 777 people intercepted on the beach. The four variables are the respondent’s age, household size, level of education, and whether the respondent’s household has children. In the final version of the survey, gender will also be available, for a total of five variables that can be used to analyze and control for nonresponse (gender was not recorded for many respondents in the pretest due to an omission in the onsite intercept survey form). Additionally, both age and gender will be available for all but a very small number of people, since even those who refused the onsite survey were asked to give their age, which most did (88.0% of the 273 refusals provided their age).

Table 6. Nonresponse analysis and variables available for nonresponse adjustment
Variable	Mail data N = 49	Onsite with address N = 345	Onsite no address N = 159	Onsite refusals N = 273	Onsite total N = 777	Onsite total without refusals N = 504
Age ≤ 45	36.2%	54.5%	58.4%	60.3%	57.3%	55.7%
Age > 45	63.8%	45.5%	41.6%	39.7%	42.7%	44.3%
Household size	2.7	3	2.8	N/A	N/A	2.9
Household has children	31.9%	39.7%	36.5%	N/A	N/A	38.7%
Less than high school	0.0%	0.9%	1.3%	N/A	N/A	1.0%
High school or General Education Diploma	10.4%	22.9%	17.3%	N/A	N/A	21.1%
Some higher education	33.3%	42.0%	32.7%	N/A	N/A	39.1%
Bachelor degree	31.3%	29.6%	33.3%	N/A	N/A	30.8%
Graduate degree	25.0%	4.6%	15.4%	N/A	N/A	8.0%

Respondents to the mail survey differ from the target sample primarily with respect to age and education (Table 6). Those returning the mail survey were on average older and more educated than those intercepted on the beach. Of those returning the mail survey, 63.8% were older than 45, compared to 42.7% in the target sample. The proportion of mail respondents with a bachelor degree or higher was 56.3%. This compares to 38.8% of the onsite survey respondents. The distributions of the variables analyzed change primarily at the mail stage of the study, indicating that nonresponse at other stages of the study may have less effect on representativeness.

We believe the best approach to adjust for nonresponse in this study – when the first contact is through an intercept survey – is to reweight the mail survey observations so the weighted distributions of key demographic variables match the variable distributions for respondents intercepted onsite. This would potentially be done for all five demographic variables and would allow us to match to about 65% of the target sample (504 / 777 = 0.65). This approach could also be applied using age and gender to match to about 96% of the target sample [(88% x 273 + 504) / 777 = 0.96). As noted earlier, the final onsite survey has been revised to include four attitudinal variables about the importance of certain beach characteristics to respondents, including the presence of garbage or marine debris, and whether they think garbage or marine debris is a problem on beaches in their local area. These variables will allow for a more thorough nonresponse adjustment than just demographic variables alone. We can also perform a nonresponse follow-up study if that approach is preferred.

4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.

Comments on the survey materials were solicited from the following persons outside the agency:

1. Dr. George Parsons, Professor, Department of Economics, University of Delaware.

2. Dr. Eric English, Economist, Bear Peak Economics.

3. Dr. Jason H. Murray, Economist, I. M. Systems Group, Inc.

A pretest was conducted with the approval of OMB from September 2017 through January 2018. During the pretest, we conducted onsite surveys at eight beaches in Orange County on Wednesday, September 27, 2017 and on the following Saturday, September 30, 2017. The onsite surveys asked several brief questions about each respondent’s recreation and also elicited the respondent’s address for completion of a follow-up mail survey. We intercepted 777 recreators and obtained onsite interviews with 504 of them. We obtained addresses for the follow-up mail survey from 345 onsite respondents.

On December 6, 2017, we mailed 345 surveys and sent reminder postcards one week later. For any valid addresses from which we had not obtained a completed survey, we sent a follow-up survey on December 20, 2017, and a second reminder postcard on January 18, 2018. By the end of January we had received 49 completed surveys. The U.S. Postal Service returned a total of 64 surveys as undeliverable due to invalid addresses.

Details of the pretest are described in a memorandum to Amy Uhrin and Carlie Herring of the National Oceanic and Atmospheric Administration, titled “Draft Pretest Results: Economic Impacts of Marine Debris on Tourism-Dependent Communities,” dated March 21, 2018 (included as a supporting document). The main findings and resulting revisions were:

• The two-stage sampling procedure resulted in a low cumulative response rate. We believe this was partly due to the scheduling of the pretest mail survey during the holiday season, which will not be a factor in the final study. The potential effects of nonresponse may be offset by reweighting mail-survey observations so that the distribution of key characteristics match with the onsite survey. To improve the nonresponse adjustment, attitudinal questions from the mail survey were also added to the revised onsite survey.

• There was a high rate of item nonresponse for questions about how many more or fewer trips respondents would take in response to changes in marine debris. We concluded that the format of the original question, requiring respondents to fill out a table, was likely to be confusing to many respondents. The question was revised to take the form of direct questions with a single answer to each question.

• The estimated aggregated responses to changes in debris were statistically significant (e.g., a 14% decrease in trips with a confidence interval of 13.4% to 16.4%). Given that the pretest obtained only 49 completed surveys, and given that the questions about the effects of debris will be revised to reduce item nonresponse, it was determined that the original plan to obtain 200 surveys per region could be revised. The revised study plan reduces the target number of completed surveys per region to 100.

Previous pretesting included one-on-one discussions that were held with beachgoers in the Boston and Los Angeles areas in December 2016 and January 2017. The tests involved seven respondents, in addition to internal testing with Abt employees. The participants filled out a draft version of the survey instrument and discussed the survey and their responses with Dr. Eric English, a research team member. The interviews were designed to evaluate the clarity of the survey questions and the ability of survey respondents to accurately answer the survey questions.

The following summarizes the issues, revisions, and conclusions of the one-on-one discussions:

• The map of the study region was important in helping respondents remember the area of interest when answering questions throughout the survey.

• Respondents were best able to understand the concept of debris density when the idea was described as the respondent picking up all the debris in a specified area and seeing what they find.

• Most respondents said that they were aware of debris levels at the beaches, that they recalled which beaches had more debris and which had less, and were able to make a reasonable estimate of how much debris was present at the beaches.

• For the survey page that explains marine debris and includes any questions, it was important to include text that directed the respondent to the next page for questions about marine debris.

• When estimating any changes in the number of trips because of changes in debris, most respondents described their thought process in ways that indicated that they understood the questions and gave them careful consideration. Examples include respondents thinking about their children playing in the sand, respondents indicating that beaches were already clean enough so that reductions in debris would not matter to them, respondents saying they would choose closer beaches they had previously avoided if there were less debris, and respondents who would change their behaviors consistently by choosing to take more trips if there were less debris and fewer trips if there were more debris.

• The survey took less than 10 minutes for most respondents.

Two additional methodological tests involve comparing survey results to external measures. First, the survey elicits respondents’ estimates of how their recreation choices would change in response to hypothetical changes in marine debris levels. This method is called “stated preference.” It is common in the economics literature to compare stated-preference results to what are called “revealed preference” results. Revealed preference involves inferring changes in behavior from actual choices people have made in the past. We will compare the stated-preference results of the Regional Pilot Study to the revealed-preference results of the Orange County Pilot Study. As described in Part A, this comparison has already been done using data available from the pretest, and the results support the reliability of the stated preference questions. Additional comparisons will be possible using the full dataset from the Regional Pilot Study.

Second, the Regional Pilot Study survey elicits respondents’ estimates of the amount of garbage and debris at beaches in each study area. These estimates are useful in characterizing the baseline level of debris to which changes are compared. For some beaches, information about the level of marine debris has already been collected onsite. The estimates by respondents to the survey will be compared with the onsite measurements for validation or potential adjustments.

5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The following individuals were consulted on the statistical aspects of the design:

1. Dr. George Parsons, Professor, Department of Economics, University of Delaware (phone: 302-831-6891).

2. Dr. Eric English, Bear Peak Economics, Boulder, Colorado (phone: 202-699-6334).

3. Dr. Adam Domanski, ECONorthwest (formerly of NOAA) (phone: 206-387-4364).

Abt will collect and analyze the information for the Program.

References

Ballance, A., P.G. Ryan, and J.K. Turpie. 2000. How much is a clean beach worth? The impact of litter on beach users in the Cape Peninsula, South Africa. South African Journal of Science 96(5):210–230.

Battaglia, M.P., D. Izrael, D. Hoaglin, and M.R. Frankel. 2004. Tips and tricks for raking survey data (aka sample balancing). American Association of Public Opinion Research.

Bess, R. and Z.O. Ambargis. 2011. Input-Output Models for Impact Analysis: Suggestions for Practitioners Using RIMS II Multipliers. Presented at the 50th Southern Regional Science Association Conference, New Orleans, LA. March 23–27. Available: https://www.bea.gov/papers/pdf/WP_IOMIA_RIMSII_020612.pdf. Accessed 11/8/2016.

BLS. 2017. U.S. Department of Labor, Occupational Employment Statistics. Bureau of Labor Statistics. Available: www.bls.gov/oes/.

Brouwer, R., D. Hadzhiyska, C. Ioakeimidis, and H. Ouderdorp. 2017. The social costs of marine litter along European coasts. Ocean & Coastal Management 138:38–49.

Chu, A., D. Eisenhower, M. Hay, D. Morganstein, J. Neter, and J. Waksberg. 1992. Measuring the recall error in self-reported fishing and hunting activities. Journal of Official Statistics 8(1):19–39.

Dillman, D., J.D. Smyth, and L.M. Christian. 2014. Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method. 4th Edition.

English, E. and K. McConnell. 2015. Overview of Recreation Assessment. DWH Lost Recreational Use NRDA Technical Working Group Report. Available: https://www.fws.gov/doiddata/dwh-ar-documents/940/DWH-AR0021412.pdf.

English, E., R. von Haefen, J. Herriges, C. Leggett, F. Lupi, K. McConnell, M. Welsh, A. Domanski, and N. Meade. In review. Estimating the value of lost recreation days from the Deepwater Horizon oil spill.

Hanemann, M., L. Pendelton, C. Mohn, J. Hilger, K. Kurisawa, D. Layton, C. Bush, and F. Vasquez. 2004. Using Revealed Preference Models to Estimate the Effect of Coastal Water Quality on Beach Choice in Southern California. A Report from the Southern California Beach Valuation Project to the National Oceanic and Atmospheric Administration.

Herriges, J. 2015. Model Structure. DWH Lost Recreational Use NRDA Technical Working Group Report. Available: https://www.fws.gov/doiddata/dwh-ar-documents/940/DWH-AR0045972.pdf.

Herring, C. 2018. Supporting Materials for the Study “The Economic Impacts of Marine Debris on Tourism-Dependent Communities”: Marine Debris Monitoring and Assessment Project (MDMAP) Overview and Justification of Marine Debris Types and Concentrations.

IEc. 2014. Assessing the Economic Benefits of Reductions in Marine Debris: A Pilot Study of Beach Recreation in Orange County, California. Final Report. June 15, 2014. Industrial Economics, Incorporated. Available: https://marinedebris.noaa.gov/file/2574/download?token=zIPamF9O. Accessed 11/3/2016.

Krelling, A., A. Williams, and A. Turra. 2017. Differences in perception and reaction of tourist groups to beach marine debris that can influence a loss of tourism revenue in coastal areas. Marine Policy 85:87–99.

Landry, C.E., T. Allen, T. Cherry, and J.C. Whitehead. 2012. Wind turbines and coastal recreation demand. Resource and Energy Economics 34(1):93–111.

Leggett, C. 2015. Travel Cost Computation. DWH Lost Recreational Use NRDA Technical Working Group Report. Available: https://www.fws.gov/doiddata/dwh-ar-documents/940/DWH-AR0056724.pdf.

Leggett, C., N. Scherer, T. Haab, R. Bailey, J. Landrum, and A. Domanski. 2018. Assessing the economic benefits of reductions in marine debris at Southern California beaches: A Random Utility Travel Cost Model. Marine Resource Economics 33(2):133–153.

Lew, D.K. and D.M. Larson. 2005. Valuing recreation and amenities at San Diego County beaches. Coastal Management 33(1):71–86.

Lynn, P. 2013. Alternative sequential mixed-mode designs: Effects on attrition rates, attrition bias, and costs. Journal of Survey Statistics and Methodology 1(2):183–205.

Medway, R.L. and J. Fulton. 2012. When more gets you less: A meta-analysis of the effect of concurrent web options on mail survey response rates. Public Opinion Quarterly 76(4):733–746.

Messer, B.L. and D.A. Dillman. 2011. Surveying the general public over the Internet using addressed-based sampling and mail contact procedures. Public Opinion Quarterly 75(3):429–457.

Millar, M.M. and D.A. Dillman. 2011. Improving response to web and mixed-mode surveys. Public Opinion Quarterly 75(2):249–269.

NOAA 2014. National Oceanic and Atmospheric Administration Information Quality Guidelines. Available: http://www.cio.noaa.gov/services_programs/IQ_Guidelines_103014.html. Accessed 5/3/2018.

Ofiara, D., and B. Brown. 1999. Assessment of economic losses to recreational activities from 1988 marine pollution events and assessment of economic losses from long-term contamination of fish within the New York Bight to New Jersey. Marine Pollution Bulletin 38 (11):990-1004.

Parsons, G. and D. Massey. 2003. A Random Utility Model of beach recreation. In The New Economics of Outdoor Recreation, N. Hanley, W.D. Shaw, and R.E. Wright (eds.). Edward Elgar.

Parsons, G., A. Kang, C. Leggett, and K. Boyle. 2009. Valuing beach closures on the Padre Island National Seashore. Marine Resource Economics 24(3):213–235.

Pendleton, L., P. King, C. Mohn, D.G. Webster, R. Vaughn, and P.N. Adams. 2011. Estimating the potential economic impacts of climate change on Southern California beaches. Climatic Change 109(1):277–298.

Schuhmann, P.W. 2012. Tourist perceptions of beach cleanliness in Barbados: Implications for return visitation. Études Caribéennes (19).

Smith, V.K., X. Zhang, and R.B. Palmquist. 1997. Marine debris, beach quality, and non-market values. Environmental and Resource Economics 10(3):223–247.

Tourangeau, R., E. English, K. McConnell, D. Chapman, I. Flores-Cervantes, E. Horsch, N. Meade, A. Domanski, and M. Welsh. 2017. The Gulf Recreation Study: Assessing lost recreational trips from the 2010 Gulf Oil spill. Journal of Survey Statistics and Methodology 5(3):281–309.

6