Download:
pdf |
pdfSupporting Statement: Visibility Valuation Study Survey (OMB# 1024-0255)
Supporting Statement B
NPS Visibility Valuation Survey
OMB Control Number 1024-0255
Collections of Information Employing Statistical Methods
The agency should be prepared to justify its decision not to use statistical methods in any case where
such methods might reduce burden or improve accuracy of results. When the question “Does this ICR
contain surveys, censuses, or employ statistical methods?” is checked "Yes," the following
documentation should be included in Supporting Statement B to the extent that it applies to the
methods proposed:
1. Describe (including a numerical estimate) the potential respondent universe and any sampling or
other respondent selection method to be used. Data on the number of entities (e.g.,
establishments, State and local government units, households, or persons) in the universe covered
by the collection and in the corresponding sample are to be provided in tabular form for the
universe as a whole and for each of the strata in the proposed sample. Indicate expected
response rates for the collection as a whole. If the collection had been conducted previously,
include the actual response rate achieved during the last collection.
The target population for this collection is individual households in the eight multi-state regions
listed below:
•
•
•
•
•
•
•
•
Northeast- Maine, New Hampshire, Vermont, New York, Massachusetts, Rhode Island,
Connecticut, New Jersey, Pennsylvania, Ohio, and Indiana
Southeast- Delaware, Maryland, Virginia, West Virginia, Kentucky, Tennessee, North
Carolina, South Carolina, Georgia, Alabama, Mississippi and Florida
Upper Midwest- Michigan, Illinois, Wisconsin, Iowa, and Minnesota
Central- Missouri, Arkansas, Louisiana, Texas, Oklahoma, and Kansas
Four Corners- Utah, Arizona, New Mexico and Colorado
Northern Plains/Rockies- North Dakota, South Dakota, Nebraska, Montana, Wyoming, and
Idaho
Sierra Nevada- California and Nevada
Northwest- Oregon and Washington
Sampling Unit: The sampling unit is all residential mailing addresses in the eight regions.
Sample Frame: The respondents for this collection will be drawn from a random sample of 25,600
residential mailing addresses purchased from Survey Sampling International (SSI). Surveys will be mailed
to 3,200 households in each of the eight regions. Based upon the results of the 2012 pilot study, we
expect that 35 percent will return completed surveys (n=1,120/region).
1
Supporting Statement: Visibility Valuation Study Survey (OMB# 1024-0255)
Table 1a. Sample Sizes and Expected Response for the Household Survey
Respondent
Universe
(Households)
Sample Size
Estimated
Response Rate
Estimated
Number of
Completed
Responses
Northeast
~28,000,000
3,200
35%
1,120
Southeast
~30,000,000
3,200
35%
1,120
Upper Midwest
~14,000,000
3,200
35%
1,120
Central
~17,000,000
3,200
35%
1,120
Four Corners
Northern
Plains/Rockies
Sierra Nevada
Northwest
~6,000,000
3,200
35%
1,120
~3,000,000
3,200
35%
~14,000,000
~4,000,000
3,200
3,200
35%
35%
Region
TOTAL
1,120
1,120
1,120
8,960
25,600
Table 1b. Sample Sizes and Expected Response for the Non-response Survey
Non- Respondent
Universe
Sample Size
Estimated
Response Rate
Northeast
2,080
Southeast
2,080
2,080
17.5%
2,080
17.5%
Estimated Number
of Completed
Responses
360
360
Four Corners
2,080
2,080
17.5%
360
Sierra Nevada
2,080
2,080
17.5%
360
Region
TOTAL
8,320
1,440
Based on similar stated-preference studies conducted by the current study team and the results of the
2012 pilot survey, the estimated 1,120 complete responses are expected to be sufficient to estimate
choice parameters and determine the influence of key respondent characteristics on estimated values.
A subsample of non-respondents in four of the regions will be sent a short follow-up survey. To enhance
cooperation the questionnaire will be sent via Fed-Ex and will include a $5 incentive. The non-response
survey will consist of a subset of questions from Section G of the survey.
2
Supporting Statement: Visibility Valuation Study Survey (OMB# 1024-0255)
2. Describe the procedures for the collection of information including:
* Statistical methodology for stratification and sample selection,
* Estimation procedure,
* Degree of accuracy needed for the purpose described in the justification,
* Unusual problems requiring specialized sampling procedures, and
* Any use of periodic (less frequent than annual) data collection cycles to reduce burden.
To estimate values for visibility improvements, we will use the random utility model (Haab and
McConnell, 2002). Under this approach, individual i's utility for a particular visibility program j, which is
defined by a set of K attributes, can be expressed as:
𝑈𝑈𝑖𝑖𝑖𝑖 = 𝛽𝛽𝑦𝑦 ( 𝑦𝑦𝑖𝑖 − 𝐶𝐶𝑗𝑗 ) + ∑𝐾𝐾
𝑘𝑘=1 𝛽𝛽𝑘𝑘 𝑋𝑋𝑗𝑗𝑗𝑗 + εij ,
where y i is individual i's money income, C j is the cost of visibility program j, and X jk is the level of
attribute k that is offered in visibility program j.
The β k 's are the marginal utilities for each of the K visibility attributes and β y is the marginal utility of
money income. Under the RUM (random utility maximization) specification, and given individuals'
stated responses to binary choice questions comparing program j to no program, these parameters can
be estimated using the conditional logit model. Once parameter estimates are available, the marginal
value of any particular attribute k can be estimated as:
𝑊𝑊𝑊𝑊𝑊𝑊𝑘𝑘 = −
�𝑘𝑘
𝛽𝛽
�𝑦𝑦
𝛽𝛽
An important feature of the pilot study, for modeling and estimation purposes, is that the visibility
attributes will be defined two different ways. This will allow for a great deal of flexibility in ultimately
identifying values for different visibility programs. The first approach is to define full visibility programs,
which we will designate as θ's. These θ's are defined by the percentages of days that will occur in a year
at each of the five visibility photos, A, B, C, D, and E. Every unique set of percentages defined in the
survey will be represented by a different program dummy variable θ. This allows for direct estimation of
the marginal values for each of these programs. A key result of this research will be the estimation of
values for specific θ's that are on the projected visibility improvement paths. The paths are defined (in
accordance with the provisions of the Regional Haze Rule) as a linear improvement in the mean of the
20 percent worst visibility days in a year from current to natural conditions by 2064. To demonstrate,
3
Supporting Statement: Visibility Valuation Study Survey (OMB# 1024-0255)
improvement paths for the Southeast Region (Great Smokies photographs) and Four Corners Region
(Canyonlands photographs) are shown in Tables 2 and 3.
Table 2. Southeast Visibility Paths (Great Smokies Photographs) - Percent of Days in Year
Allocated to Each Photograph
Year
Percent
Photo
A
Photo
B
Photo
C
Photo
D
Photo
E
2007
0.05
0.19
0.24
0.21
0.22
0.14
2019
0.25
0.33
0.3
0.19
0.14
0.04
2024
0.33
0.43
0.3
0.16
0.09
0.02
2034
0.5
0.64
0.25
0.08
0.03
0
2044
0.67
0.84
0.14
0.02
0
0
2049
0.75
0.91
0.08
0.01
0
0
2061
0.95
0.99
0.01
0
0
0
2064
1
1
0
0
0
0
Table 3. Four Corners Visibility Paths (Canyonlands Photographs) - Percent of Days in Year
Allocated to Each Photograph
Year
Percent
Photo
A
Photo
B
Photo
C
Photo
D
Photo
E
2007
0.05
0.16
0.19
0.2
0.27
0.17
2019
0.25
0.23
0.23
0.21
0.23
0.1
2024
0.33
0.28
0.24
0.21
0.2
0.07
2034
0.5
0.37
0.26
0.19
0.15
0.04
2044
0.67
0.48
0.26
0.15
0.09
0.02
2049
0.75
0.54
0.25
0.13
0.07
0.01
2061
0.95
0.68
0.21
0.08
0.03
0
2064
1
0.72
0.19
0.07
0.02
0
4
Supporting Statement: Visibility Valuation Study Survey (OMB# 1024-0255)
The following attributes are included in this first model:
θ
health
ecol
time
cost
dummy variable for program, as defined by Photos A, B, C, D, E
dummy variable for health benefits
dummy variable for ecological benefits
time for program to take effect
cost of the program
The second approach to defining visibility attributes is based on the individual photos. We can redefine the θ's as additive functions of the set of five visibility photos, A, B, C, D, and E:
𝜃𝜃𝑗𝑗 = 𝛾𝛾𝐴𝐴 𝑝𝑝ℎ𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑗𝑗 + 𝛾𝛾𝐵𝐵 𝑝𝑝ℎ𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑗𝑗 + 𝛾𝛾𝐶𝐶 𝑝𝑝ℎ𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑗𝑗 + 𝛾𝛾𝐷𝐷 𝑝𝑝ℎ𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑗𝑗 + 𝛾𝛾𝐸𝐸 𝑝𝑝ℎ𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑜𝑗𝑗
where the variables photoA j through photoE j are defined as the percentages of days realized at the
visibility levels defined by those photos under program j.
The following attributes are included in the second model:
photo_A
photo_E
health
ecol
time
cost
the percent of days in a year at the visibility level defined by Photo A
the percent of days in a year at the visibility level defined by Photo E
dummy variable for health benefits
dummy variable for ecological benefits
time for program to take effect
cost of the program
To be able to estimate both of these models, an experimental design must be developed that is
flexible enough to identify all parameters in both models. This requires sufficient variation in each
of the attribute levels defined above; specifically, variation is needed across visibility programs (the
θ's) and across individual photo levels A through E, as well as across the other attributes in the
survey.
The Experimental Design
The experimental design challenge is to define a series of binary choice sets that will allow for the
identification of all sets of parameters defined in the previous section. In the preliminary survey, all
choice sets will be binary choices offering a visibility program that can be provided at a cost
compared to no program at no cost. This means that each binary choice set is fully defined by
5
Supporting Statement: Visibility Valuation Study Survey (OMB# 1024-0255)
specifying the levels of the attributes that are being offered, as well as the cost. Attribute levels vary
both within and across respondents.
To derive these choice sets, a 24-row, orthogonal, main-effects design matrix was drawn from a
well-regarded, on-line catalog of orthogonal matrices by Warren Kuhfeld. The size of this design
matrix allows for orthogonal placement of our three two-level attributes (a health benefit dummy
variable, an ecological benefits dummy variable, and time, which will be 10 or 20 years), one fourlevel attribute (program cost, which will take values of $15, $35, $65, and $115), and one six-level
attribute (the programs, θ, more detail below).
As described above, the goal of this analysis is to estimate the utility model in two separate ways:
one that allows us to estimate marginal values for the visibility improvement programs that are
predicted to occur over time (the θ's), and one that estimates marginal values for the occurrence of
specific levels of visibility improvements, as defined by Photos A through Photo E. This challenge is
addressed by making sure that both approaches to measuring visibility -- the photo percentages and
the definitions of the θ's -- vary sufficiently across and within choice sets. To do this, we first pull
three visibility programs for each region directly from the visibility improvement paths in Tables 2
and 3. The programs pulled are at the 5 percent, 50 percent and 100 percent points along those
paths. Second, to get sufficient variation in the photo percentages, we create four additional
programs by "perturbing" the 50 percent program in the following four ways: we increase and
decrease the percentage occurrence of Photo A, and we increase and decrease the percentage
occurrence of Photo E. In all cases, the amount of increases and/or decreases are added and/or
subtracted from Photo C. This process results in a total of seven visibility programs. 1,2
To demonstrate, the experimental designs for the Southeast and Four Corners regions are presented
in Tables 4 and 5. 3 Following these tables are graphs that show the range of values for Photos A and
Because two programs turned out to be very close for the Southeast region, only six programs are used in the
final experimental design for that region.
2
Since the design matrix only accommodates a six-level attribute, variation over the seven programs is
manufactured by mixing information from two additional two-level columns from the design matrix into the
perturbation routine.
3
A price adjustment was made on a small number of choice sets to decrease the probability of having complete
dominance -- choice sets where all respondents choose the same alternative. When generated choice sets
resulted in a high visibility (100 percent point on visibility path) and low cost ($15) program, or vice versa, low
1
6
Supporting Statement: Visibility Valuation Study Survey (OMB# 1024-0255)
Photos E. 4 The design has 24 choice sets, which are assumed to be randomly assigned to four
different survey versions with six questions per survey.
Table 4. Four Corners Design -- 4 survey versions with 6 questions each
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
+-------------------------------------------------------------------------------------+
| version
health
ecol
time
photoa
photob
photoc
photod
photoe
cost |
|-------------------------------------------------------------------------------------|
|
1
0
1
20
30
26
27
15
3
15 |
|
1
0
0
10
16
19
20
27
17
35 |
|
1
0
0
20
30
26
27
15
3
65 |
|
1
0
0
10
72
19
7
2
0
115 |
|
1
1
0
10
49
26
3
15
8
65 |
|
1
1
0
20
49
26
3
15
8
115 |
|-------------------------------------------------------------------------------------|
|
2
1
1
20
30
26
22
15
8
15 |
|
2
1
1
20
16
19
20
27
17
15 |
|
2
0
1
20
49
26
3
15
8
35 |
|
2
0
1
10
37
26
19
15
4
35 |
|
2
1
1
10
49
26
8
15
3
35 |
|
2
1
1
10
37
26
19
15
4
65 |
|-------------------------------------------------------------------------------------|
|
3
0
1
20
16
19
20
27
17
65 |
|
3
0
0
20
37
26
19
15
4
115 |
|
3
1
0
10
30
26
27
15
3
115 |
|
3
1
0
20
30
26
22
15
8
35 |
|
3
1
1
10
49
26
3
15
8
15 |
|
3
0
0
10
49
26
8
15
3
65 |
|-------------------------------------------------------------------------------------|
|
4
1
0
10
16
19
20
27
17
15 |
|
4
1
0
20
72
19
7
2
0
35 |
|
4
0
0
20
49
26
8
15
3
15 |
|
4
0
1
10
49
26
8
15
3
115 |
|
4
0
1
10
72
19
7
2
0
115 |
|
4
1
1
20
72
19
7
2
0
65 |
+-------------------------------------------------------------------------------------+
visibility (5 percent) at a high cost $115, then the costs were replaced with the more consistent value -- $115 for
the high visibility program and$15 for the low visibility program.
4
Photos A and E are the primary focus of the visibility analysis, so variation in these levels is most important.
7
Supporting Statement: Visibility Valuation Study Survey (OMB# 1024-0255)
Table 5. Southeast Design -- 4 survey versions with 6 questions each
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
+-------------------------------------------------------------------------------------+
| version
health
ecol
time
photoa
photob
photoc
photod
photoe
cost |
|-------------------------------------------------------------------------------------|
|
1
0
0
10
49
25
23
3
0
65 |
|
1
0
0
20
100
0
0
0
0
115 |
|
1
1
0
10
49
25
18
3
5
65 |
|
1
0
0
20
64
25
3
3
5
65 |
|
1
1
0
20
100
0
0
0
0
115 |
|
1
0
0
20
64
25
3
3
5
115 |
|-------------------------------------------------------------------------------------|
|
2
1
0
20
19
24
21
22
14
35 |
|
2
1
1
10
100
0
0
0
0
65 |
|
2
0
0
10
19
24
21
22
14
15 |
|
2
0
1
20
49
25
18
3
5
15 |
|
2
0
1
10
49
25
23
3
0
115 |
|
2
0
1
20
64
25
8
3
0
65 |
|-------------------------------------------------------------------------------------|
|
3
1
1
20
64
25
8
3
0
115 |
|
3
0
1
10
100
0
0
0
0
35 |
|
3
1
0
10
64
25
8
3
0
15 |
|
3
0
0
10
64
25
8
3
0
35 |
|
3
1
1
20
49
25
23
3
0
15 |
|
3
1
1
20
19
24
21
22
14
65 |
|-------------------------------------------------------------------------------------|
|
4
0
1
10
19
24
21
22
14
15 |
|
4
0
1
20
49
25
18
3
5
35 |
|
4
1
1
10
64
25
3
3
5
35 |
|
4
1
1
10
64
25
3
3
5
15 |
|
4
1
0
10
49
25
18
3
5
115 |
|
4
1
0
20
49
25
23
3
0
35 |
+-------------------------------------------------------------------------------------+
Testing the Experimental Design
To verify that the experimental design will identify all parameters, a simulation was run on the Four
Corners experimental design with 1,000 replications. Each replication assumed a sample size of 400:
100 responses to each of the four survey versions. With each survey version having six questions, the
total sample size for each replication was 2,400.
The simulation assumed the following specification for utility:
U = .04*PhotoA - .05*PhotoE + .7*Health + 1.15*Ecol -.03*Time -.025*Cost + ε
Simulation results for both types of models to be estimated are provided in Table 6. 5 All parameters
appear to be well estimated, given the sample size.
These results simply provide a “check” on the design levels. The hypothetical utility parameters were derived
from basic analyses of data from earlier focus groups.
5
8
Supporting Statement: Visibility Valuation Study Survey (OMB# 1024-0255)
Table 6. Four Corners Simulation Results
Mean estimation
Number of obs
=
1000
--------------------------------------------------------------------|
Mean
Std. Err.
[95% Conf. Interval]
--------------------+-----------------------------------------------Health Attribute |
.6526956
.0025738
.6476449
.6577464
Ecological Attribute |
1.151795
.0023456
1.147192
1.156398
Time Attribute | -.0386896
.0001654
-.0390141
-.038365
Cost | -.0258266
.0000354
-.0258961
-.0257572
Program 2 |
1.048048
.0046263
1.038969
1.057126
Program 3 |
1.257548
.0044654
1.248785
1.266311
Program 4 |
1.44215
.0045162
1.433288
1.451012
Program 5 |
1.783491
.0040151
1.775612
1.791369
Program 6 |
1.979975
.0038199
1.972479
1.987471
Program 7 |
3.073499
.0047036
3.064269
3.082729
---------------------------------------------------------------------
Mean estimation
Number of obs
=
1000
------------------------------------------------------------------|
Mean
Std. Err.
[95% Conf. Interval]
------------------+-----------------------------------------------Health Attribute |
.7048772
.0024329
.700103
.7096515
Ecological Attribute |
1.155364
.0022502
1.150949
1.15978
Time Attribute | -.0300786
.0001951
-.0304615
-.0296958
Cost | -.0250515
.0000352
-.0251206
-.0249823
Photo A |
.0400566
.0000779
.0399038
.0402094
Photo E | -.0505181
.0002356
-.0509805
-.0500557
-------------------------------------------------------------------
Analysis of Collected Data
As described above, choice data will be analyzed using standard discrete choice models in the RUM
framework and values for various visibility improvement scenarios will be calculated. In addition,
standard errors and confidence intervals will be calculated using the Krinsky and Robb (1986) simulation
method. We will then perform several tests to evaluate the sensitivity of results to alternative model
specifications within each region.
9
Supporting Statement: Visibility Valuation Study Survey (OMB# 1024-0255)
3. Describe methods to maximize response rates and to deal with issues of non-response. The
accuracy and reliability of information collected must be shown to be adequate for intended uses.
For collections based on sampling, a special justification must be provided for any collection that
will not yield "reliable" data that can be generalized to the universe studied.
A number of methods will be used to maximize survey response rates, as summarized below:
•
Use of USPS Delivery Sequence File as Sample Frame- By drawing the sample from a
comprehensive list of residential mailing addresses, we avoid the potential for incomplete
coverage of the target population potentially associated with other sampling frames. Within
sampled households we ask that the survey be completed by the male or female head of the
household.
•
Careful Survey Design and Focus Group Pre-Testing- The survey was developed and rigorously
tested in 20 two-hour focus group sessions (four groups in each of five different states). The
questions are worded in a manner that is easy to understand and organized in a logical order. In
addition, we have consulted a graphic design expert to assist with survey graphics, layout and
presentation.
•
Administration by a University Survey Research Center- Surveys that are Government/
University sponsored tend to receive higher response (Heberlein and Baumgartner, 1978). Our
survey will be administered by a university survey research center.
•
Best-Practice Implementation Sequence- Following Dillman (2000), households selected to
participate in the survey will receive:
o
A pre-survey notification (initial contact) letter on NPS letterhead and signed by the Director
of the Air Resources Division explaining the purpose and significance of the survey.
o
One week later respondents will be sent a copy of the survey with cover letter (including a
toll-free number for respondents to call with any questions) and an incentive in the form of
a $2 bill. The use of modest monetary incentives has been shown to significantly increase
survey response rates (Rathbun and Baumgartner, 1996 and Warriner et al., 1996).
o
Within five days of the initial survey mailing a reminder postcard will be sent.
10
Supporting Statement: Visibility Valuation Study Survey (OMB# 1024-0255)
o
Within three weeks of the initial survey mailing a second copy of the survey will be sent.
Incoming responses will be tracked and the second mailing may be sent earlier if returns are
tapering significantly.
o
Three weeks after the second survey mailing the data collection period will conclude and
the nonresponse surveys will be implemented.
Identifying Possible Nonresponse Bias
Nonresponse bias refers to the expected difference between an estimate from respondents in the
sample and an estimate from the target population and may arise from both unit (household does
not return survey) and item (returned survey is incomplete) nonresponse. Of particular concern in
this context is whether nonresponse results in biased measures of WTP for visibility improvements.
We propose three specific procedures for investigating potential nonresponse bias in our collected
survey data:
1) Benchmarking- Responses to demographic questions (e.g., age, income, gender, race,
education) from respondents will be compared to data from the 2010 Census. In addition, the
survey includes several questions regarding opinions on environmental issues and government
programs from the National Opinion Research Center General Social Survey (collectively these
are questions 26 to 36 as described in Part A). These responses will also be compared within
survey region.
2) Late Responders- We will compare survey responses, respondent characteristics and estimated
WTP values across individuals who returned their surveys at different times during the data
collection period. For example, we can compare individuals who returned their surveys after the
first mailing versus after the second mailing. Although all of these people are responders, those
who respond later may share important characteristics with non-responders.
3) Non-respondent Follow-Up Survey- In four of the eight regions, nonrespondents will be recontacted via Fed-Ex to complete a short follow-up survey consisting of a subset of five of the
questions from the main survey. Sampled nonrespondents will receive one Fed-Ex package,
which will include a $5 incentive. Up to two reminder postcards will be sent subsequently to
encourage response.
11
Supporting Statement: Visibility Valuation Study Survey (OMB# 1024-0255)
Statistically-significant differences in the means and/or distributions of variables described in (1), (2)
and/or (3) above would provide evidence of likely nonresponse bias.
Adjusting for Nonresponse Bias
In making adjustments for potential nonresponse bias we are concerned with factors that are
related to response rates and individual’s WTP for visibility improvements. The most common
approach for testing and correcting for sample selection is the Heckman two-stage model
(Heckman, 1979). The first stage entails modeling the likelihood of responding as a function of
individual characteristics. We will rely upon the data collected in the nonrespondent phone surveys
regarding demographic characteristics and responses to attitudinal questions.
The estimated
parameters from the first stage are used to calculate the inverse Mills ratio, which is included in the
second stage to correct for selection under certain assumptions. In our case the second stage are
the models explaining responses to the valuation questions.
Finally, we will test for significant differences in WTP estimates from the standard and selection
models.
4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an
effective means of refining collections of information to minimize burden and improve utility.
Tests must be approved if they call for answers to identical questions from 10 or more
respondents. A proposed test or set of tests may be submitted for approval separately or in
combination with the main collection of information.
Survey materials were developed and tested extensively through a series of focus groups and a pilot
survey, and were informed by an exhaustive review of past visibility valuation literature. Focus
groups were conducted in five states in 2008 and 2009. Four groups were held in each state (two
groups per evening on consecutive evenings) at professional focus group facilities. Respondents
were randomly recruited from samples of local telephone numbers.
Atlanta, GA: The first set of groups focused on investigating respondents' understanding of "National
Parks and Wilderness Areas”; evaluating the degree to which respondents focus on visibility
improvements versus any health and/or ecological benefits resulting from reduced haze; evaluating the
12
Supporting Statement: Visibility Valuation Study Survey (OMB# 1024-0255)
degree to which respondents believe that visibility improvements will only occur within a designated
"visibility improvement region;” investigating respondents' reactions to images selected to depict five
levels of visibility due to differing levels of haze; determining the best approach for presenting numerical
and graphical information about the distribution of visibility levels throughout the year; and exploring
respondents’ reactions to different payment vehicles for eliciting willingness to pay for visibility
improvements.
•
Chicago, IL: Key objectives of the second set of groups included evaluating respondents’
understanding of how particles that form haze move to National Parks and National
Wilderness Areas; evaluating whether participants were able to understand how the
Regional Haze Rule will result in improved air quality in National Parks and National
Wilderness Areas; evaluating respondents' reactions to digitally manipulated photographs
that depict visibility at five different levels of haze; evaluating respondents’ reactions to
numerical and graphical presentations of information about the distribution of visibility
levels throughout the year, under baseline (current) conditions and improved (reduced
haze) conditions; and, evaluating respondents’ reactions to the use of electricity bills as the
payment vehicle used for eliciting willingness to pay for visibility improvements.
•
Sacramento, CA: Key objectives of the third set of groups included evaluating respondents’
ability to understand bar charts depicting information about the distribution of visibility
levels throughout the year under baseline conditions (no implementation of haze-reduction
program), natural conditions (all human-caused haze eliminated), and conditions under a
haze-reduction program; evaluating respondents’ reactions to the introduction of visibility
improvement program attributes and levels; and, evaluating respondents’ responses to
draft attribute-based choice questions.
•
Denver, CO: The fourth set of groups focused on further refining the description and
presentation of choice question attributes and levels. In addition, two variants of the survey
were tested- a regional section which only focuses on improvements within the one visibility
improvement region closest to where the participants live, and a national section which
considers visibility improvements within all seven improvement regions across the United
States.
13
Supporting Statement: Visibility Valuation Study Survey (OMB# 1024-0255)
•
Boston, MA: The fifth and final set of groups focused on final revisions to the choice
questions. Specifically, the attribute table was divided into two columns, with-program and
without-program, to explicitly define the status quo conditions; the visibility improvement
scenario represented by bar charts was moved to the top of the table to encourage
respondents to explicitly consider this attribute when answering each choice question; and,
two bar chart formats were investigated, one with current and improved conditions on the
same chart (as in previous groups) and one with separate charts for each state.
Upon completion of the Boston focus groups the study team was confident that the choice
question format with separate charts was superior and that the remainder of the information
and questions in the survey was functioning properly. All survey materials were then provided
to experts in the field of stated-preference and visibility valuation for peer review (Dr. Vic
Adamowicz and Dr. William Schulze). Comments from these experts were incorporated and
final materials were developed. Full reports describing the focus group proceedings and the
peer review reports are submitted as a supplementary document.
Dr. Vic Adamowicz, Distinguished Professor
Department of Rural Economy,
University of Alberta,
Dr. William Schulze, Professor
Applied Economics and Management,
Cornell University,
The purpose of the pilot study was to determine whether survey, valuation scenario and
experimental design parameters functioned properly prior to implementation of the full survey.
A mail survey was administered to a random sample of 4,000 households in the southwestern
and southeastern U.S. in late summer and early fall of 2012. Response rates for the southwest
and southeast surveys were 38.6 and 32.5 percent, respectively. Telephone and mail follow-up
surveys of nonrespondents were also conducted. A comparison of “benchmarking” question
responses to well-established public opinion survey results, as well as respondent characteristics
to Census data, indicated that survey respondents were similar to, but not fully representative
of, the general populations of these regions. Analysis of valuation question responses indicated
that the magnitude of visibility improvement and the occurrence of related ecological and
human health improvements are significant determinants of program choices. Household
14
Supporting Statement: Visibility Valuation Study Survey (OMB# 1024-0255)
willingness-to-pay (WTP) for visibility improvements increased with programs that reduce the
number of lowest visibility days and increase the number of highest visibility days over the
course of a year. Models based on data weighted to reflect general population parameters
resulted in WTP estimates that were generally between +/- 10 percent of unweighted estimates.
Overall, the pilot study results indicated that the survey instrument functioned well and is
appropriate for full implementation. Detailed pilot results are provided in report attached as
Appendix A.
5. Provide the names and telephone numbers of individuals consulted on statistical aspects of the
design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will
actually collect and/or analyze the information for the agency.
•
Dr. Kevin Boyle, Professor and Department Head, Agricultural and Applied Economics, Virginia Tech
University, (540) 231-2907.
•
Dr. Richard Carson, Professor, Department of Economics University of California, San Diego, (858)
534-3384.
•
Mr. Robert Paterson, Principal, Industrial Economics, Incorporated, (617) 354-0074.
REFERENCES
Dillman, D.A. 2000. Mail and Internet surveys: The Tailored Design Method. New York, NY: John Wiley &
Sons.
Groves, R. M. and M. P. Couper. 1998. Nonresponse in Household Interview Surveys. New York, Wiley.
Groves, R. M., M. P. Couper, S. Presser, E. Singer, R. Tourangeau, G. P. Acosta and L. Nelson. 2006.
"Experiments in Producing Nonresponse Bias." Public Opinion Quarterly 70(5): 720-736.
Groves, R. M., S. Presser and S. Dipko. 2004. "The Role of Topic Interest in Survey Participation
Decisions." Public Opinion Quarterly 68(1): 2-31.
Haab, T.C. and K.E. McConnell. 2002. Valuing Environmental and Natural Resources: The Econometrics of
Non-Market Valuation. Edward Elgar, Northampton, MA.
Heckman, J. 1979. "Sample selection bias as a specification error". Econometrica 47 (1): 153–61
Krinsky, I and A. L. Robb. 1986. “On Approximating the Statistical Properties of Elasticities.” Review of
Economic and Statistics68: 715-719.
Kuhfeld, Warren, F., Orthogonal Arrays, Advanced Analytics Division,
SAS, http://support.sas.com/techsup/technote/ts723_Designs.txt
15
Supporting Statement: Visibility Valuation Study Survey (OMB# 1024-0255)
Heberlein, T. A. and R. Baumgartner. 1978. "Factors Affecting Response Rates to Mailed Questionnaires:
A Quantitative Analysis of the Published Literature." American Sociological Review 43(4): 447-462.
Warriner, K., J. Goyder, H. Gjertsen, P. Hohner, and K. McSpurren. 1996. "Charities, No; Lotteries,
No; Cash, Yes: Main Effects and Interactions in a Canadian Incentives Experiment." Public Opinion
Quarterly 60 (4): 542-562.
16
| File Type | application/pdf |
| File Modified | 2015-08-03 |
| File Created | 2015-08-03 |