NASS Comments

Comments from the National Agricultural Statistical Service regarding the survey entitled: Homeowner Risk Reduction Behaviors Concerning Wildfire Risks

July 2006

The respondent universe is not clearly defined; consequently, it is uncertain what population the study will ultimately draw conclusions about. The report [supporting statement] mentions that the survey will focus on targeted Home Owner Associations (HOAs) in three cities in the Western US that are high-risk areas for forest fires. The report did not specifically mention:

1. How were the three cities that the study will take place in chosen? How will the choice of these three cities affect the Forest Service's ability to generalize the results of the study to other cities in the Western US that did not have a chance to participate in the survey?

2. If an HOA chooses not to participate in the study, what methodology will be used to adjust for their non-response?

3. If a household within an HOA does not respond, what methodology will be used to adjust for their non-response?

4. Will homeowners that are not part of a community that has an HOA be given the opportunity to take part in the survey? If so, how will they be selected to participate? If not, how will that affect the interpretation of the results?

If the purpose of the survey is to study the opinions and behaviors of a specific community so that Forest Service personnel can be better prepared to help that particular community, then the design of this survey is statistically defensible. However, the aggregated data from all the households that were surveyed should not be generalized beyond the communities that actually had a chance to participate in the study. Communities in other cities and other states may have different factors that would affect their responses to the survey.

It is also unclear who within a selected household will actually complete the survey itself. Will it be all adults or the head of the household?

The two survey instruments do not have any information identifying the survey, its purpose, who is conducting it, and any contact information, or basic instructions.

Each questionnaire needs a unique identifier for survey management purposes. There is also no space for the respondent to identify himself/herself to verify that the correct household responded. If the two survey instruments are independent and there is no identification on either survey, the management of the completed questionnaires will be difficult (e.g.: the two surveys could become separated and the data may no longer be useable. Or the respondent may only return one of the questionnaires).

Alexandra Riley

Statistics Division

Statistical Methods Branch

NASS