CFLRP Survey Supt Stmt Part B

Supporting Statement B

Southwestern Crown Collaborative Forest Management Social Monitoring

OMB Control Number 0596-NEW

Collections of Information Employing Statistical Methods

The agency should be prepared to justify its decision not to use statistical methods in any case where such methods might reduce burden or improve accuracy of results. When the question “Does this ICR contain surveys, censuses, or employ statistical methods?” is checked "Yes," the following documentation should be included in Supporting Statement B to the extent that it applies to the methods proposed:

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample.

The potential respondent universe for the Southwestern Crown Collaborative survey will include:

1. Residents within and adjacent to collaborative forest landscapes: these are people that live in communities directly affected by activities conducted under the CFLRP. Based on 2010 US Census Bureau data, within the South Western Crown landscape there are 4,429 occupied dwellings. We plan to randomly sample 2,200 households from a list of addresses from this population. We expect 10% undeliverable addresses reducing the sample to 1,980 addresses. We expect an initial 30% response rate (594 completions) and will conduct a non-response check by mail (200 additional mailings) to reach the 600 completed responses.

2. Collaborative participants: these are stakeholders who have actively participated in Collaborative Forest Landscape Restoration (CFLR) Program collaborative decision making processes. For the universe of collaborative participants, we will send an email with a link to the survey to all 150 individuals on our collaborative email list. We will treat this as a population not a sample. We expect at least a 75% response rate.

Strata	Population Number	Sample or population estimate	Expected responses
Number of residences	4,429	2,200 (Sample)	600
Collaborative participants	150	150 (Population)	113

2. Describe the procedures for the collection of information including:

* Statistical methodology for stratification and sample selection,

* Estimation procedure,

* Degree of accuracy needed for the purpose described in the justification,

* Unusual problems requiring specialized sampling procedures, and

* Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

We plan to conduct a random sample of 2,200 households from a population of 4,429 occupied dwellings in our landscape. We expect 10% of these will be bad addresses leaving a sample of 1,980. We expect a 30% response rate giving us 600 completed surveys. We estimate that with 600 responses we will obtain approximately a +/- 3% confidence interval on our population of approximately 4,429 households. We believe this to be an adequate error margin for our purposes. We plan to sample both year-round and seasonal households equally. We do not plan to use any specialized sampling procedures. We plan to survey only twice: once now midway through the implementation of the CFLR program, and again after implementation of the program has been completed (~ 5 years from now).

The Southwestern Crown Collaborative’s participant email list consists of 150 addresses of individuals that have directly participated in collaborative activities or have asked to be informed about collaborative activities. This list includes individuals that reside outside the Southwestern Crown landscape and are actively engaged, and well-versed, in forest management topics. We plan to email a link to the online survey to all members of this list and expect a 75% response rate.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

We will give all respondents an initial opportunity to complete the survey online. Past research suggests approximately 15-25 percent of local residents will respond to online surveys and that approximately 50 percent of other stakeholders will respond to online surveys. Those residents and other stakeholders not responding to the online survey will be given an opportunity to complete the monitoring instrument via mail. The full monitoring process will begin with a pre-survey notice mailing which will include a unique URL where respondents may complete the survey online, powered by Qualtrics. After responses to this online option diminish, we will mail respondents a hardcopy of the monitoring instrument with a postage-paid return envelope. A reminder postcard will then be sent two weeks after this mailing to non-respondents. Two weeks following the reminder postcard we will mail a replacement monitoring instrument to any remaining non-respondents. We expect an overall completion rate of 30% for residents and 75% for other stakeholders. In all cases, appropriate non-response bias strategies will be used to ensure that responses are representative of the contact universe. This will include telephone non-response bias checks where non-respondents are contacted via telephone and asked a subset of the ten most important survey questions; this will allow us to detect and correct for significant differences between respondents and non-respondents.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

The Bureau of Business and Economic Research (BBER) at University of Montana tested our survey instrument by conducting cognitive interviews of eight individuals. The interviews were conducted between June 28th and July 5th in Seeley Lake, Montana, which is in the middle of our survey landscape. Seven of the eight interviews were recorded and transcribed. Notes were taken during all eight interviews. Three of the respondents were ages 20-29, four were ages 60+, and one was age 50. All interviews were conducted with residents of Seeley Lake and followed an interview script developed by University of Montana BBER. The length of the interviews ranged from 45 to 75 minutes.

We learned many things from the testing and edited the survey accordingly. We shortened the survey based on respondents’ frustration with the length. We learned that respondents quickly stopped reading any introductory statements and skipped straight to the questions, and we removed many of those statements. We also changed the language in many questions to make them more comprehensible to individuals that are not as familiar with forest management. We also removed a series of questions directly related to recent management projects because most respondents were not familiar with the projects.

5. Provide the names and telephone numbers of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The following individuals were consulted on statistical and other design aspects of this program and will conduct the analysis of the data. Data collection will be conducted by the Bureau of Business and Economic Research under the supervision of Dr. Baldridge.

Individuals consulted on statistical aspects of the design of the Programmatic Approval Process

Dr. John Baldridge, Director of Survey Research

Bureau of Business and Economic Research

University of Montana

Phone (406) 243-5113

Dr. Alex Metcalf, Research Assistant Professor

Department of Forest Management

University of Montana

Phone: (406) 243-6673

3