Supplemental_Statement_B_02022007

Supporting Statement

Public Support for Fuel Reduction Policies: Multimedia versus Printed Materials

The Supporting Statement for OMB 0596-NEW

PUBLIC SUPPORT FOR FUEL REDUCTION POLICIES: MULTIMEDIA VERSUS PRINTED MATERIALS

February 2007

Note: All changes are in blue.

B. Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sam­pling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corre­sponding sample are to be provided in tabular form for the uni­verse as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

According to the 2000 census, there are 2,152,343 households in the six California counties under study, with a total population of almost 6.3 million people. Of the total households, 1.434 million are white households and .444 millions are Latino households, with a total population of 3.5 and 1.9 million people respectively. The statistics for the five counties in Montana are 157,250 households for a total population of 396,411. Approximately 1,400 households will be sampled in California and Montana to obtain a set of 1,000 completed interviews.

A stratified random sampling procedure will be used. The sample will be divided in equal numbers of Spanish and English (333 each) speaking respondents in California and English (334) speaking respondents in Montana. The sample will target the same counties in California and Montana used in the original phone-mail-phone survey conducted in California and Montana. The counties are Alameda, Contra Costa, Kern, Riverside, San Bernardino, and Imperial in California; and Flathead, Missoula, Ravalli, Lewis and Clark, and Yellowstone in Montana.

There are no circumstances foreseen that would cause the sample to be unreliable or invalid. However, if the response rate were below 50 percent, results could not be generalized to the populations of the counties studied. The sample size is large enough to achieve reliable results for the intended populations and will be within a 5.7 percent error margin for all three populations in the dichotomous choice CVM question (Babbie 1991).

Sample frame is as shown in the following table (Bates et al. 2000)

Sample Frame		California		Montana	Total
		English	Spanish	English
1.	Total Contacted Less: Non-working-Changed-wrong numbers No answer-busy-answering machine No appropriate respondent	467 25 31 23	467 21 34 27	466 35 29 26	1400 81 94 76
2.	= Net Sample Less: Refusals Callback	388 84	385 97	376 93	1149 274
3.	= Completed Screener Less: Non-response	304 49	288 63	283 59	875 171
4.	= Completed Self-administered Questionnaire	255	225	224	704
5.	Overall Response Rate (4/1) %	54.6	48.1	47.9	50.3

2. Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection,

• Estimation procedure,

• Degree of accuracy needed for the pur­pose described in the justification,

• Unusual problems requiring specialized sampling procedures, and

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

A stratified random digit dialing along a fire-risk gradient across several counties in California and Montana of 1,400 heads of households to reach 1,000 completed surveys.

The counties included in the survey are Alameda, Contra Costa, Kern, Riverside, San Bernardino, and Imperial in California; and Flathead, Missoula, Ravalli, Lewis and Clark, and Yellowstone in Montana.

The sample is divided as follows: 467 each for Spanish and English speaking households in California, and 466 English-speaking households in Montana.

The risk gradient varies from counties with a large number of wildland fires to counties with very few wildland fires. The counties with few to zero fires will serve as control.

Proponents will sample three times as many people in the high number of fires counties as in the control (low to zero) counties. There are more counties in the high number of fires category, and two times as many households in the medium number of fires counties, than in the control counties. The approximate distribution is 168 households in the high fire counties, 110 in the medium fire counties and 55 in the control counties.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sam­pling, a special justification must be provid­ed for any collection that will not yield "reli­able" data that can be generalized to the universe studied.

Using stratified random digit dialing (RDD) allows proponents to contact households with listed and unlisted telephones, increasing the probability of including all households in the calling universe. As a rule, commercially available household listings are based on telephone directories and do not include all existing households in the county of interest.

The initial RDD procedure will locate study participants willing to respond to the initial short phone survey. These individuals will receive the video and questionnaire package. After one week, participants still not responding will receive a reminder postcard encouraging them to send in their answer sheets. After another week, participants who have not returned their answer sheet will receive another video survey and answer sheet, and again asked to complete the self-administered, in-depth questionnaire. Two weeks after sending the second letter requesting their participation without a response, we will contact non-respondents by phone to ask if they will complete the answer sheet, and if not to obtain some demographic characteristics of the respondent while on the phone for a non-response check.

To deal with item non-response issues, all respondents will be asked why they chose not to respond to the questions or why they answered the way they did. These questions are included in the video survey. Responses help determine if the zero responses are valid responses or protests to the scenarios presented in the survey.

A tally of all non-responses will be analyzed to determine if those individuals are different from the ones that responded. As proposed by Cameron et al. (1999)¹ zip code is the key to generating variables that capture the salience and unobserved tendency of respondents to answer or not answer the survey. The zip code is used to obtain demographics and socioeconomic characteristics of the potential respondents’ neighborhood.

We can obtain zip code for those in the first contact that did not respond, and zip codes for those that did, and do a 1st stage Heckman on that decision to respond or not as a function of demographics and socioeconomic characteristics. The resulting Inverse Mills Ratio from this Heckman can be included as a variable in our subsequent WTP analyses (e.g., logit or probit models) to correct for this first stage non-response bias, if it exists.

A Tabular comparison of sociodemographics of respondents and non-respondents can also be made to provide more of an intuitive comparison of differences between respondents and non-respondents.

Respondents will receive a $5 incentive for participation and completion of the survey interview to help achieve a higher response rate.²

Token financial incentives of a few dollars included with the request are effective at raising response rates. Response rates increased by 7 percent to 11 percent in studies cited in Mail and Internet Surveys: the Tailored Design Method.³

The study, Contingent Valuation of Hazardous Waste Risk Reductions, determined that a $1 incentive increased response rate from 45 percent to 64 percent. In this study, a two mailing treatment without a $1 incentive was compared to an identical survey with a $1 incentive included in the package. The package with the monetary incentive had a 19 percent greater response rate in California.⁴

The results will be generalized only to the counties in their respective states. However, it would help to clarify that these counties were selected to represent three strata of counties that cover the entire state: (a) little or no wildfires; (b) occasional wildfires; (c) frequent wildfires. All counties in each state fall into one of these three strata.

To deal with the issue of non-response, once the participant has agreed to but does not complete the self-administered survey, the proponents will use the bivariate probit model with sample selection. This model incorporates Heckman’s (1979) thoughts on sample selection bias into the standard bivariate probit, a model with two simultaneously estimated equations that allows for correlation between the error terms in each equation.

The premise of Heckman’s sample selection model is that “using non-randomly selected samples to estimate behavioral relationships” results in biased and inconsistent parameter estimates (1979, p. 153). Because self-selected respondents may differ in some significant way from non-respondents, it is important to correct for this bias. Ignoring this issue could lead to inconsistent parameter and WTP⁵ estimates, making them unfit for generalization to the desired population.

To reduce the possibility of a different person completing the answer sheet, the proponents will check for gender and age during the pre-screening, and then crosscheck with the answer sheet to make sure there is a match for gender and age. However, because the unit of analysis is the household, proponents think it will not make much difference. Furthermore, this is unlikely to occur because the transmittal letter will state that the person the letter is addressed to is the person who must complete the survey.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separate­ly or in combination with the main collection of information.

The survey instrument was refined based on a peer review process, as well as employing statistical review. A small focus group of nine persons reviewed the survey instrument for clarity and understanding of content, to ensure the reality of the fuels reduction alternatives presented. To ensure the accuracy of the information presented, Forest Service fire managers and planners reviewed the survey instrument. Adjustments and refinements were made based on these reviews.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Statistical consultation was provided by:

• Dr. Jim Baldwin, Pacific Southwest Research Station, USDA Forest Service; 510.559.6332;

• José Sánchez, Mathematical Statistician, Pacific Southwest Research Station, USDA Forest Service, 951.680.1589; and

• Dr. Alexandra Riley, Statistical Division, Statistical Methods Branch, NASS.

Data will be collected jointly by:

• Dr. John B. Loomis, Colorado State University;

• Dr. Hayley Hesseln, University of Saskatchewan,

• Dr. Armando González-Cabán, Pacific Southwest Research Station, USDA Forest Service; and

• Dr. Richard T. Serpe, Director, Social and Behavioral Research Institute, California State University, San Marcos

Data will be analyzed by Drs. González-Cabán, Loomis, and Hesseln

Reports and manuscripts will be jointly prepared by Drs. González-Cabán, Loomis, and Hesseln.

References:

Babbie, Earl. 1991. The practice of research, 6^th ed.; Belmont CA: Wadsworth Publishing Co. 493 p.)

Bates, Nancy; Doyle, Pat; Winters, Franklin. 2000. Survey nonresponse: new definitions and measurement methods. A paper presented at the Federal Committee on Statistical Methodology Statistical Policy Seminar hosted by the Council of Professional Association on Federal Statistics (COPAFS); Bethesda, MD; November 9-0, 2000.

Dillman, Don. 2000. Mail and Internet Surveys: The Tailored Design Method, pp 167-16

duVair, Pierre. 1994. Contingent Valuation of Hazardous Waste Risk Reductions, dissertation, Graduate Group in Ecology, Office of Graduate Studies, University of California, Davis

Loomis, J., González-Cabán, A., Gregory, R. 1994. Do reminders of substitutes and budget constraints influence contingent valuation estimates? Land Economics 70(4): 499-506.

Heckman, J. 1979. Sample selection bias as a specification error; Econometrica 47(1): 153-161.

Vera, Robert M. 2003. Video education for wildfire mitigation in the wildland urban interface of Colorado; Fort Collins, CO: Colorado State University; Master Thesis 85 p.

1 Cameron, Trudy Ann; Shaw, W. Douglass; Rangland, Shannon R.; 1999. Nonresponse bias in mail survey data: Salience vs. endogenous survey complexity. In Valuing Recreation and the Environment: Revealed Preference Methods in Theory and Practice. Pp. 217-251.

2 The survey research centers approached as potential contractors to conduct the survey shared difficulties they are experiencing in recruiting participants for survey research studies. One main reason is large telemarketing campaigns. The research centers suggested providing a cash incentive of $5 per respondent to help increase the response rate.

3 Dillman, Don. 2000. Mail and Internet Surveys: The Tailored Design Method, pp 167-169

4 duVair, Pierre. 1994. Contingent Valuation of Hazardous Waste Risk Reductions, dissertation, Graduate Group in Ecology, Office of Graduate Studies, University of California, Davis.

5 Willingness-to-pay

6