Supplemental_Statement_Part_B

Request for Approval of a Survey of Public Attitudes, Beliefs, and Values about National Forest Systems Land Management in the Southwestern Region

The Supporting Statement for OMB 0596-NEW

Public Attitudes, Beliefs, and Values About National Forest Systems Land Management

February 2007

Addendum (3/27/2007): Based on OMB review and changes to the survey instruments, only one survey instrument will be used instead of two as originally proposed. Other than this change, the survey methodology will remain the same. Modifications made to item A-12 reflect this change, as well as a reduction in burden hours due to modifications of the survey instrument per OMB instructions.

B. Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sam­pling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corre­sponding sample are to be provided in tabular form for the uni­verse as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

The respondent universe for this survey is the public in Forest Service Region 3 (Southwestern Region). That region is comprised of the states of Arizona and New Mexico, and a few counties each in Texas and Oklahoma. Total number of households in that region is approximately 2.6 million. The sampling unit is the household—we will sample a maximum of one person (an adult) per household. The sampling plan calls for stratified random sampling. Stratification will be used to ensure statistically valid and representative samples at: (1) the regional level, (2) state levels for Arizona and New Mexico, (3) the sub-state areas adjacent to each of the national forests and grasslands in the region (of which there are 13).

Maps of the region have been overlaid with national forest and grassland boundaries, county boundaries, and zip code boundaries. The maps will be used to specify groups of counties and zip code areas that need to be randomly sampled according to a formula based on locality population of households, county populations of households, state and region populations of households, subject to the constraint of achieving minimum sample size needed to produce statistical validity. Within the specified groups of counties, data on numbers of households and distribution by census tract will be used to spread the sampled households throughout the county grouping to avoid “clumping” of observations in areas of highest population. To adequately sample members of minority populations, demographic data at each area level will be considered. The result will be a sample that will be representative and statistically valid at each of the multiple levels mentioned above.

We have calculated minimum sample size needed in each of our sampling cells using assumptions of maximum variance to ensure statistical validity at the smallest aggregation level. Having the population and household data for the entire respondent universe will allow us to weight and aggregate to a representative sample at any of the levels described above.

The specifications determined using the formulas mentioned above would be given to a commercial sampling firm. Such firms use lists compiled from telephone listings, supplemented by other data sources, as the universe from which they randomly sample based on the specifications provided. That firm will supply contact information and some demographic characteristics of the sample to allow basic level verification that the sample meets the specifications provided. The primary sampling unit will be one respondent per household.

One large sample will be drawn and then people will be randomly assigned to the four treatment groups for this project (region-wide and test surveys by mail back response and web-based response option).

2. Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection,

• Estimation procedure,

• Degree of accuracy needed for the pur­pose described in the justification,

• Unusual problems requiring specialized sampling procedures, and

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

For the mail back survey mode: Mail contacts will be sent to (specific) selected individuals (determined in the stratified random process above). An initial letter will be sent to potential participants telling them about the survey and its purpose, and uses to which the information will be put, and asking for their participation. That initial letter will be followed in about 1 week by Survey Packet 1. Approximately 10 days after Survey Packet 1 a postcard will be sent to all potential participants thanking them if they have already responded and reminding them of the importance of responding for those who have not responded. Approximately 10 days after the postcard, Survey packet 2 will be sent to non-respondents. Approximately 3 weeks after Survey Packet 2, Survey Packet 3 will be sent to non-respondents. All survey packets will consist of a cover letter, the questionnaire, a Questions & Answers page, a map of the region showing locations and names of forests and grasslands, and a postcard with which people can request a Spanish language version of all materials.

For the mail back plus web-based option survey mode: Potential respondents will be sampled from the same population in the same way that sampling is done for the mail back survey—they will be a random sub-sample of the same population. An initial letter will be sent (using the regular postal system) to potential respondents telling them about the survey and its purpose, and uses to which the information will be put. That letter will also invite them to respond to the survey online and provide a web site address and a unique User ID. The letter will contain the Questions & Answers page and the Spanish language version request postcard. Approximately 2 weeks after the initial letter, Survey Packet 1 will be sent to non-respondents to the web-based survey. Approximately 3 weeks after Survey Packet 1, Survey Packet 2 will be sent to non-respondents (by either mode of response). Survey packets will consist of a cover letter, the questionnaire, a Questions & Answers page, a map, and a postcard with which they can request a Spanish language version of all materials. The packets will also contain the web address and User ID and will again invite the person to respond via the web, either in English or in Spanish. This procedure provides a test of recent suggestion in the survey research literature that providing multiple response mode options could increase response rates.

At no time will respondents be pressured to respond. Response will always be voluntary, and respondents will be so informed.

Survey responses from the mail-back surveys will be entered into a database using a data entry program written specifically for this survey. All data will be entered twice and differences reconciled to minimize data entry errors. Bilingual data entry technicians are available for entering responses received in Spanish, and translating as needed. Responses from the web-based survey will be recorded, saved, and entered directly into the database when the respondent indicates he/she is finished.

Standard methods of analysis will be used as are common in the social science and economic literature. Such statistical methods as frequencies, cross tabulation and contingency tables, regression analysis, factor analysis, cluster analysis, and discriminant analysis, among others, will be used as appropriate to individual data items and groups of items. Analyses will be compiled and reported, then subjected to peer-review before release, as is required by the Rocky Mountain Research Station.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sam­pling, a special justification must be provid­ed for any collection that will not yield "reli­able" data that can be generalized to the universe studied.

There are several keys to maximizing response rates. Foremost among them is good survey design and question construction. Survey instruments, and questions within the instrument, must be well written, clear and concise, and flow well from question to question. Instructions must be clear and unambiguous. Respondents must know what is being asked of them and in what manner they are being asked to respond. Several scientists involved in this effort are experienced and recognized experts in survey-based data collection. They are known for their attention to detail with respect to survey design and question construction. Focus groups and pretests are helpful tools in survey design and question construction. Focus groups will be used in this effort to ensure a high quality survey instrument.

A second key to maximizing response rate is motivating respondents. For mail surveys, cover letters and a page of questions and answers will be used to convey to respondents the importance of the information and their participation in providing the information. Participants will receive a personalized letter from the Southwestern Regional Forester, emphasizing the importance of their participation. Information will be given to respondents outlining the purpose of the data collection; and how survey results will be used to modify public policy regarding natural resources and public lands. In addition, every effort will be made to include native Spanish speakers. The cover letter will be in both English and Spanish and participants can easily request a Spanish translation survey by sending in a postcard. The web-response option on the “test survey” will be available in both English and Spanish.

A third key is follow up. We will use multiple contacts. As described above, up to 5 contacts per respondent will be used in the mail back survey and up to 3 contacts per respondent will be used in the web-based survey. In each case, that will allow for three points of response over time—survey 1, 2, 3 in the mail back case, and initial invitation, followed by two follow-up invitations in the web-based survey. Responses will be tracked by when they are received so a relationship can be estimated between response and time of response. By providing for three “response points,” i.e., three waves of survey response, one can statistically estimate response curves for individual questions as one test for non-response bias. Those response curves can also be used to model the expected responses one might have received from non-respondents had they responded, and provide the basis for weights to adjust for any biases that might be detected. In every follow-up contact, and for both survey modes (mail only or mail plus web response option), respondents will be presented with the opportunity to take the survey in Spanish. While, unfortunately, this remains uncommon in large surveys, it is critical for maximizing response rate in this case because of the high proportion of Spanish-speaking households in the Southwest.

We cannot accurately predict the final response rate that might be achieved by this proposed survey, though a response rate of 50 percent is anticipated for this survey.

Survey response rates, in general, have been declining over the last several years. That is a common topic of discussion among survey researchers. Whereas response rates in the 70 percent to 80 percent range, and higher, were once relatively common, response rates now commonly hover in the 40 percent to 50 percent range for a general population survey; and those are considered good response rates. Many surveys now are achieving response rates in the 30 percent to 35 percent range. As noted above, there are steps that we can and will take to maximize response rate; these include: initial endorsement letter, good survey design, salient topic, reasonable length, multiple response mode options, multiple contacts (3-5), and multiple primary language options (English and Spanish).

While these efforts will help to maximize response rate, they also help in achieving our primary goal of obtaining a statistically valid and representative sample. It is important to recognize that response rate is not the only issue to consider. Beyond the response rate, one must be concerned about respondent characteristics. In particular, are survey respondents different in some systematic way then those who did not respond? If they are not, then response rate does not matter. The responses received represent the population without bias. If respondents are different then non-respondents, the possibility of non-response bias must be considered during data analysis. It is important to test for such non-response bias so any differences can be accounted for in interpreting the results. At a minimum, one needs to acknowledge and describe non-response bias if it exists. In some cases, it is appropriate to weight the data to adjust for non-response bias. In some cases, it may be appropriate to collect data in a different manner to enable evaluation and interpretation of the effects of non-response bias, or otherwise account and adjust for its presence.

Tracking when responses are received will allow one test of potential non-response bias; i.e., are there systematic differences in responses received earlier in time and those received later in time? As noted above, use of three response points will allow for one potential method of anticipating how non-respondents might respond, and therefore allow analysts to adjust for non-response in presentation and interpretation of results. The survey proposed here will utilize three response points and we will use those data on when responses were received to test for and adjust for any non-response bias.

As a further check on non-response, we will conduct a very brief telephone interview of a random sample of non-respondents (approximately 200 people). People will be asked some socio-demographic questions, a few key questions from the original survey, and reasons they did not respond to the original survey. The purpose of that brief interview is to provide data with which to analyze for non-response bias. Data from respondents (to the mail and to the web-based surveys) will be compared with that of non-respondents using this telephone-based information as well as a variety of geographic and census data. If non-response bias exists, the supplemental data from non-respondents, along with census information, can be the basis for constructing sample weights to ensure representative results.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separate­ly or in combination with the main collection of information.

The intent is to use focus groups for additional testing of particular questions and sections of the survey instruments.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Analytical/Statistical and cooperating personnel include:

Forest Service Personnel

• Dr. Daniel W. McCollum, Economist, Rocky Mountain Research Station, 970-295-5962

• Dr. Patricia A. Champ, Economist, Rocky Mountain Research Station, 970-295-5967

• Dr. Daniel R. Williams, Research Social Scientist, Rocky Mountain Research Station, 970-295-5970

• Dr. Thomas C. Brown, Economist, Rocky Mountain Research Station, 970-295-5968

• Dr. Deborah J. Shields, Economist, Rocky Mountain Research Station, 970-295-5975

• Dr. Brian M. Kent, Research Forester, Rocky Mountain Research Station, 970-295-5955

• Rudy M. King, Station Statistician, Rocky Mountain Research Station, 970-498-1240

• Dr. Richard D. Periman, Social Science Coordinator, Ecosystem Analysis and Planning, Southwestern Region, 505-842-3225

• Dr. Patrick Reed, Regional Social Scientist, Alaska Region, 907-743-9571

• Denise Wickwar, Quality Assurance Specialist and Survey Statistician, Inventory and Monitoring Institute, 970-295-5717

• Fred P. Clark, Regional Social Scientist, Eastern Region, 414-297-1181

Non-Forest Service

• Dr. Gregory S. Alward, Branch Chief, Planning and Analysis Group, Inventory and Monitoring Institute, 970- 295- 5714

• Dr. Robert Berrens, Professor, University of New Mexico, 505-277-9004

• Dr. Jennifer Thacher, Asst. Professor, University of New Mexico, 505-277-1965

Dr. Robert Berrens and Dr. Jennifer Thacher of the University of New Mexico will accomplish data collection. Graduate students will provide assistance, in particular Michael Hand and Jeff Bjarke.

Page 7