Supporting Statement Part B 0596-0201

2009 Supporting Statement – Part B OMB 0596-0201

Role of Communities in Stewardship Contracting Projects.

B. Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sam­pling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corre­sponding sample are to be provided in tabular form for the uni­verse as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

Since the number of stewardship contracting projects changes each year the potential respondent universe will vary slightly, however the manner of selecting respondents to be surveyed will not. The Forest Service (FS) and Bureau of Land Management (BLM) originally estimated that over the first three years there would be no more than 550 stewardship contracting projects in a given year. However, the actual number of stewardship contracts has generally been increasing each year (except for the BLM which decreased from 113 in 2008 to 67 in 2009). Thus, the estimate has been revised to no more than 560 stewardship contracting projects in a given year. (See B.1. Table – Summary of FS Stewardship Project and Interview Response Data, which includes a footnote (2) regarding BLM’s number of projects.)

Strata used in stratified random sampling process¹

Total active projects	Northeast /Lake states	Southeast	Northern Rockies	Rocky Mountain /Southwest	Pacific Northwest
FS	random sample	random sample	random sample	random sample	random sample
BLM	no projects expected	no projects expected	random sample	random sample	random sample

¹ Each year the number of projects in each stratum will be determined by the Forest Service and BLM. The Pinchot Institute and subcontractors will then interview a random sample of enough projects in each stratum to provide an effective and efficient statistical sample.

This project will utilize a process of stratified random sampling to determine which projects to study as part of the national programmatic monitoring efforts, based on the following criteria. Each year, the Forest Service and Bureau of Land Management will separately provide a list of stewardship contracting projects which includes project name, the State in which each project is located, and the federal project manager’s email and phone number. From this list of projects, the Pinchot Institute and their subcontractors will utilize a process of stratified random sampling. Projects will be stratified by the managing agency (FS or BLM) and by geographic region (five regions will be used – Northeast/Lake States (CT, DE, IA, IL, IN, MA, ME, MD, MI, MN, MO, NJ, NH, NY, OH, PA, RI, VT, WI, WV), Southeast (AL, AR, FL, GA, KY, LA, MS, NC, SC, TN, VA), Northern Rockies (ID, MT, ND, SD, WY), Southwest (AZ, CO, KS, NE, NM, NV, OK, TX, UT) , Pacific Northwest (AK, CA, HI, OR, WA)). Projects will be randomly selected from each stratum. Maureen McDonough from Michigan State University, who designed the sampling process, estimates that no more than 25 percent of the projects in each strata will be selected for the survey each year. If there are less than 12 projects in a given strata the sampling rate may exceed 25 percent. For example, because of low numbers, 45 percent of the Bureau of Land Management projects were sampled in each strata during the 2008 sample year. If the number of Bureau of Land Management projects were to increase, then potentially the sampling rate would decrease to 25 percent of the projects in a given strata.

For each project entered into the sampling pool, three people shall be interviewed, including the agency project manager and two external (to the agencies) participants in the project (ideally one community participant and one contractor involved in implementation of the project). When conducting the phone survey with the agency project manager for a randomly selected project, the Pinchot Institute and its subcontractors will ask for a complete list of all community members and contractors involved in the project. The Pinchot Institute and its subcontractors will contact three individuals for each project, the project manager and then randomly select two external participants to survey from the federal project manager’s list, by using stratified random sampling based on type (i.e. community participant versus contractor)

Analysis of the data is on a project basis and not on a respondent basis. To explain further, some of the questions are respondent based, providing individual perceptions, such as how an individual would define stewardship contracting. The responses to respondent-based questions are reported as total individual responses. Other questions are project based, such as the interviewees’ answers to questions regarding the role of communities in a particular project, or the benefits of a particular project. All responses to project-based questions are aggregated into a project level analysis.

The sampling pool consists entirely of individuals who have voluntarily participated in a stewardship contracting project. The sampling pool consists of three people for each project, the project manager, and two external participants. Non-response of selected participants is expected to be low. In fact, 100 percent of those individuals contacted did agree to be interviewed. However, the Pinchot Institute and their Partners were not always able to contact three people per project due to various circumstances (such as a potential interviewee being out-of-town and unavailable during the sampling time period). Non-response of selected participants is expected to be low and in fact, 100 percent of those individuals contacted have agreed to be interviewed. However, if there were a selected individual that does not wish to participate in the survey, or an individual could not be contacted for various reasons, the interviewer would randomly select (using stratified random sampling based on type) another participant from the project’s list.

As an example, the following table summarizes FS stewardship project and interviewee response data during 2006 through 2012.

B.1. Table – Summary of Forest Service Stewardship Project and Interview Response Data.)

Project Year	Number of Forest Service projects available to sample	Number of Forest Service projects actually sampled	Percent of Forest Service projects actually sampled	Number of potential interviewees (3 per project actually sampled)	Number of actual interviewees	Percent interviewed
2006	206	51	24.7	153	121	79
2007	255	58	22.7%	174	125	72
2008	285	71	24.9%	213	144	67
2009	349	87	25% (estimated)	261	To be determined (TBD)	TBD
2010	396 estimated (1)	99	25% (estimated)	297	TBD	TBD
2011	443 estimated	111	25% (estimated)	333	TBD	TBD
2012	490 Estimated (2)	123 estimated)	25% (estimated)	369	TBD	TBD

⁽¹⁾ (Based on an estimated average increase of 48 projects per year calculated from the 2006-2009 data.)

⁽²⁾ The total estimated FS projects are 490. The # of BLM projects available to sample were as follows: 2006 – 71; 2007 – 89; 2008 – 113; 2008 – 67. This averages to an approximately 1% increase…or 68, 69, 70 during 2010 – 2012. The estimated total number of potential projects that might be sampled, during 2010, is 560 (490 FS plus 70 BLM).

2. Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection,

• Estimation procedure,

• Degree of accuracy needed for the pur­pose described in the justification,

• Unusual problems requiring specialized sampling procedures, and

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

The stratification is two–fold. Projects will be stratified first by agency, then by geographic region (five regions will be used –Northeast/Lake States (CT, DE, IA, IL, IN, MA, ME, MD, MI, MN, MO, NJ, NH, NY, OH, PA, RI, VT, WI, WV), Southeast (AL, AR, FL, GA, KY, LA, MS, NC, SC, TN, VA), Northern Rockies (ID, MT, ND, SD, WY), Southwest (AZ, CO, KS, NE, NM, NV, OK, TX, UT) , Pacific Northwest (AK, CA, HI, OR, WA)). The Pinchot Institute and subcontractors will then randomly sample 25 percent of the projects in each strata each year. If there are less than 12 projects in a given strata the sampling rate may exceed 25 percent. For example, because of low numbers, 45 percent of the Bureau of Land Management projects were sampled in each strata during the 2008 sample year. If the number of Bureau of Land Management projects were to increase, then potentially the sampling rate would decrease to 25 percent of the projects in a given strata. Collapsing of the strata, due to lack of insufficient project numbers has not been necessary to-date, as the number of projects continues to steadily increase for both the Forest Service and the Bureau of Land Management. However, if necessary, we could collapse the strata and consider a 100 percent sample of projects. As information is collected during the interview process, it will be entered into a uniform report format and sent to Michigan State University for analysis. Following receipt of the data, MSU researchers will code questions and responses for entry into SPSS and NVivo software programs used for quantitative and qualitative analyses, respectively. Since the information is needed to write an annual report to Congress, data collection cycles can not be less frequent than annual.

The original survey questions are more open-ended and allow for more qualitative and less quantitative analyses such as frequencies of responses. The proposed revisions to the survey are more closed-ended questions, based on several years of actual and quite consistent responses from interviewees. However, there is always a chance for the interviewee to explain a response, in a more qualitative manner, per the “other – please explain” category. The revision to more closed-ended questions will actually enrich the data analysis. The more closed-ended questions provide the ability to go beyond frequency analysis, to also include means, Chi-squared analyses, confidence intervals and other statistical analyses. All of these will definitely strengthen the data analysis and potential use of and inferences developed from the data.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sam­pling, a special justification must be provid­ed for any collection that will not yield "reli­able" data that can be generalized to the universe studied.

The phone survey method will be used in order to secure a high response rate. The phone interview can be conducted at a time convenient for the participant. In addition the potential respondent universe includes only individuals who have chosen to be involved in some manner in a stewardship contracting project; therefore they will be familiar with the information in the survey. Currently, we have had 100 percent of the individuals actually contacted responding. However, if there were a selected individual that does not wish to participate in the survey the interviewer would randomly select (using stratified random sampling based on type) another participant from the project’s list.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separate­ly or in combination with the main collection of information.

No tests or procedures will be undertaken. An OMB survey number was awarded, and the survey has been conducted for three years. Minor format and wording changes are being submitted for approval (as noted in under Section A(2)(g)).

The original survey questions are more open-ended and allow for more qualitative and less quantitative analyses such as frequencies of responses. The proposed revisions to the survey are more closed-ended questions, based on several years of actual and quite consistent responses from interviewees. However, there is always a chance for the interviewee to explain a response, in a more qualitative manner, per the “other – please explain” category. The revision to more closed-ended questions will actually enrich the data analysis. The ability to go beyond frequency analysis, to also include means, Chi-squared analyses, confidence intervals and other statistical analyses will definitely strengthen the data analysis and potential use of and inferences developed from the data.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Maureen McDonough, Michigan State University, (517) 432-2293, [email protected], designed the statistical aspects of the survey. The Pinchot Institute for Conservation and their subcontractors will be collecting and analyzing the data. The project manager at Pinchot Institute is Brian Kittler, mailto:[email protected], (202) 797-6585.

1

Page 4