Supporting_Statement_B

Supporting Statement: Visibility Valuation Survey Pre-Testing and Pilot Study (OMB# 1024-XXXX)

B. Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

Since the data collected during the pre-testing phase of this effort will not be used to generate national benefit estimates, non-probability sampling methods will be employed. Respondents will be recruited via telephone to participate in 12 different focus groups in three different cities: Atlanta, GA, Chicago, IL, and Sacramento, CA. In each city, four groups will be conducted, two each on two consecutive evenings. These cities were chosen to provide information from areas with different visibility conditions and proximity to NPS park units. The following criteria will be applied in recruiting participants from each city:

• Age: Approximately 30% of the participants will be 20 to 34 years old, approximately 30% of the participants will be 35 to 49 years old and approximately 40% of the participants will be 50 or older.

• Education: Approximately 50% of the participants will have a high school education or less, approximately 25% of the participants will have some college or an associate degree, and approximately 25% of the participants will have a bachelor's degree (or greater).

• Income: Approximately 1/3 of the participants will be from households with annual incomes less than $30,000, approximately 1/3 of the participants will be from households with annual incomes between $30,000 and $65,000, and approximately 1/3 of the participants will be from households with annual incomes greater than $65,000.

• Gender: Approximately 50% of the participants will be male and approximately 50% will be female.

• Focus group experience: None of the participants will have participated in a focus group within the last two years.

The first four recruitment criteria (age, education, income, and gender) are designed to ensure that the characteristics of the participants approximately mirror those of the general population of the United States (based on recent Census data). The last criterion (focus group experience) is designed to avoid habitual focus group participants.

For the pilot study, 16 neighborhoods will be selected in two metropolitan areas for sampling. Phoenix, AZ and Syracuse, NY were chosen to provide information from two areas with different visibility conditions and proximity to NPS park units. Within each neighborhood, all owner-occupied households will be enumerated and a simple random sample of 100 households will be selected for the survey. Each neighborhood sample will be split into two groups, with 50 households assigned to a mail survey group and 50 households assigned to an in-person survey group. Assuming an overall response rate of 50 percent, approximately 800 respondents will participate in the pilot study, 50 respondents in each of the 16 neighborhoods.

Sample Frame	Respondent Universe	Sample Size	Response Rate	Estimated Final Responses
Owner-occupied Households in Phoenix and Syracuse Metro Areas	23,699 (Syracuse) 298,839 (Phoenix) (from 2005 ACS)	1,600	50%	800

2. Describe the procedures for the collection of information including:

* Statistical methodology for stratification and sample selection,

* Estimation procedure,

* Degree of accuracy needed for the purpose described in the justification,

* Unusual problems requiring specialized sampling procedures, and

* Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Pre-test (focus groups):

As described in B.1, non-probability sampling methods will be employed to select a sample for the focus groups. Estimation procedures and degree of accuracy for focus groups do not present the same issues as they do in quantitative surveys designed to generalize to a larger universe of study. There are no unusual problems requiring specialized sampling procedures. This is a one-time information collection, so there will be no periodic data collection cycles.

Pilot survey (mail survey and in-person interviews):

100 households within each of 16 neighborhoods in the two metropolitan areas will be selected at random. Assuming an overall response rate of 50 percent, approximately 800 responses will be collected. Discrete choice models will be used to analyze the attribute-based choice question data and estimate per-household values for visibility improvements. These values will serve as the dependent variable in a regression model that posits household willingness-to-pay as a function of response rate and survey mode, among other covariates. The study team estimates that 800 observations will be sufficient to detect any effect of response rate and survey mode on estimated values at conventional levels of significance. There are no unusual problems requiring specialized sampling procedures. This is a one-time information collection, so there will be no periodic data collection cycles.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

Focus groups:

The use of focus groups is intended to gain qualitative insight into the range of public opinions regarding visibility issues and assist in refinement of the questionnaire. As such, response rates and issues of non-response (including non-response bias) do not present the same concern as in quantitative studies.

Pilot survey:

The pilot survey will use a modified Dillman (2000) approach to maximizing response rates. In particular:

• Households selected for the mail survey group will be sent a pre-survey notification letter, followed by a survey instrument.

• Non-respondents will receive a reminder letter and up to two replacement surveys.

• Households selected for the in-person group will receive an advance phone-call followed by a personal visit to the place of residence.

• In-person interviewers will attempt numerous call-backs if the respondent is not home.

With respect to non-response bias, the purpose of the pilot study is to undertake a split-sample comparison of the data obtained from self-administered questionnaires collected through mail and in-person survey modes. The study will be designed to develop a calibration equation that would be used to adjust responses to the nationwide mail survey data for potential non-response bias.

Dillman, D.A. (2000). Mail and Internet surveys: The Tailored Design Method. New York, NY: John Wiley & Sons.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

The intention of the pilot survey is to compare information collected through alternative modes, and with varying response rates. The study will be designed to develop a calibration equation that would be used to adjust responses to the nationwide mail survey data for potential non-response bias.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

• Susan Johnson, Air Resources Division, National Park Service, (303) 987-6694

Ms. Johnson is responsible for project oversight on behalf of the NPS.

• Dr. Kevin Boyle, Virginia Polytechnic Institute, (540) 231-2907

Dr. Boyle is the Principal Investigator; he will direct survey development, design, implementation and analysis of collected information.

• Dr. Richard Carson, University of California, San Diego, (858) 534-3384

Dr. Carson will advise on survey development, design, implementation and analysis of collected information

• Pam Rathbun, PA Consulting, (608) 827-7820

Ms. Rathbun will oversee implementation of the pilot study.

• Robert Paterson, Industrial Economics, Incorporated, (617) 354-0074

Mr. Paterson will provide technical and administrative support for the focus groups and pilot study.