0596_0201SuptStmtPartB_10_06_2012

2013 Supporting Statement – Part B OMB 0596-0201

Role of Communities in Stewardship Contracting Projects

B. Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sam­pling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corre­sponding sample are to be provided in tabular form for the uni­verse as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

Since the number of stewardship contracting projects changes each year, the potential respondent universe varies slightly. However, the manner of selecting respondents to be surveyed does not. The Forest Service (FS) and Bureau of Land Management (BLM) originally estimated that, over the first three years, there would be no more than 550 stewardship contracting projects in a given year. However, the actual number of stewardship contracts has generally been increasing each year (except for the BLM which decreased from 113 in 2008 to 69 in 2009 and dropped to 63 in 2010, and then increased to 100 in 2011). Thus, the estimate is revised to no more than 675 stewardship contracting projects in a given year. (See B.1. Table – Summary of FS Stewardship Project and Interview Response Data, which includes a footnote (2) regarding BLM’s number of projects.)

Strata used in stratified random sampling process¹

Total active projects	Northeast /Lake states	Southeast	Northern Rockies	Rocky Mountain /Southwest	Pacific Northwest
FS	random sample	random sample	random sample	random sample	random sample
BLM	no projects expected	no projects expected	random sample	random sample	random sample

¹ Each year the total number of stewardship projects in each stratum is determined by the Forest Service and BLM. The Pinchot Institute and subcontractors interview a random sample of enough projects in each stratum to provide an effective and efficient statistical sample.

This project utilizes a process of stratified random sampling to determine which stewardship projects to study as part of the national programmatic monitoring efforts. Each year, the Forest Service and Bureau of Land Management separately provide a list of stewardship contracting projects which includes project name, the State in which each project is located, and the federal project manager’s email and phone number. From this list of projects, the Pinchot Institute and their subcontractors utilize a process of stratified random sampling. Projects are stratified by the managing agency (FS or BLM) and by geographic region (five regions will be used – Northeast/Lake States (CT, DE, IA, IL, IN, MA, ME, MD, MI, MN, MO, NJ, NH, NY, OH, PA, RI, VT, WI, WV), Southeast (AL, AR, FL, GA, KY, LA, MS, NC, SC, TN, VA), Northern Rockies (ID, MT, ND, SD, WY), Southwest (AZ, CO, KS, NE, NM, NV, OK, TX, UT) , Pacific Northwest (AK, CA, HI, OR, WA)). Projects are randomly selected from each stratum. Dr. Maureen McDonough, from Michigan State University, who designed the sampling process, estimates that no more than 25 percent of the projects in each strata are selected for the survey each year. If there are less than 12 projects in a given strata, the sampling rate may exceed 25 percent. For example, because of low numbers, 45 percent of the Bureau of Land Management projects were sampled in each strata, during the 2008 and 2009 sample years. If the number of Bureau of Land Management projects were to increase, then potentially the sampling rate would decrease to 25 percent of the projects in a given strata (which did occur in the 2010 and 2011 sample years, per the Pinchot’s reports to the agency).

For each project entered into the sampling pool, three people are interviewed, including the agency project manager and two external (to the agencies) participants in the project (ideally one community participant and one contractor involved in implementation of the project). When conducting the phone survey with the agency project manager for a randomly selected project, the Pinchot Institute and its subcontractors ask for a complete list of all community members and contractors involved in the project. The Pinchot Institute and its partners/subcontractors contact three individuals for each project, the project manager and then randomly select two external participants to survey from the federal project manager’s list, by using stratified random sampling based on type (i.e. community participant versus contractor)

Analysis of the data is on a project basis and not on a respondent basis. To explain further, some of the questions are respondent based, providing individual perceptions, such as how an individual would define stewardship contracting. The responses to respondent-based questions are reported as total individual responses. Other questions are project based, such as the interviewees’ answers to questions regarding the role of communities in a particular project, or the benefits of a particular project. All responses to project-based questions are aggregated into a project level analysis.

The sampling pool consists entirely of individuals who have voluntarily participated in a stewardship contracting project. The sampling pool consists of three people for each project, the project manager, and two external participants. Non-response of selected participants is expected to be very low. Per personal conversation with Dr. McDonough, if they did have any non-respondents, it was VERY low, as she did not hear about it from the partners. The Pinchot Institute and their Partners are not always able to contact three people per project due to various circumstances (such as a potential interviewee being out-of-town and unavailable during the sampling time period). If a selected individual does not wish to participate in the survey, or an individual cannot be contacted for various reasons, the interviewer randomly selects (using stratified random sampling based on type) another participant from the project’s list. However, Dr. McDonough stated she is unaware of anyone who, when contacted, has refused to participate.

As an example, the following table summarizes actual and estimated FS stewardship project and interviewee response data during 2006 through 2015.

B.1. Table – Summary of Forest Service Stewardship Project and Interview Response Data.

Project Year	Number of Forest Service projects available to sample	Number of Forest Service projects actually sampled	Percent of Forest Service projects actually sampled	Number of potential interviewees (3 per project actually sampled)	Number of actual interviewees	Percent interviewed
2006	206	51	24.7	153	121	79
2007	255	58	22.7%	174	125	72
2008	285	71	24.9%	213	144	67
2009	349	88	25.2%	264	226	86
2010	319	83	26%	229 (3)	206	90 (3)
2011	399	100	25.1%	300	251	84
2012	438 (estimated) (1)	110 (estimated)	25% (estimated)	330	TBD	TBD
2013	477 (estimated) (1)	119 (estimated)	25% (estimated)	357	TBD	TBD
2014	516 (estimated) (1)	129 (estimated)	25% (estimated)	387	TBD	TBD
2015	555 (estimated) (1), (2)	139 (estimated)	25% (estimated)	417	TBD	TBD

⁽¹⁾ (Based on an estimated average increase of 39 projects per year calculated from the 2006-2011 data.)

⁽²⁾ The total estimated FS projects are 555. The # of BLM projects available to sample were as follows: 2006 – 71; 2007 – 89; 2008 – 113; 2009 – 69; 2010 - 63; 2011 – 100. This equates to an estimated average increase of 4.4 projects per year or, rounding to 5 projects per year, 105, 110, 115, 120 projects during 2012 – 2015, respectively. Therefore, the estimated total number of potential projects that might be sampled, during 2015, is 675 (555 FS plus 120 BLM).

⁽³⁾ In their FY 2010 report to the Forest Service, the Pinchot reported “In some instances, less than three interviews were undertaken due to difficulties in contacting project participants or because the project manager was unable to provide the names of two or more involved non-agency individuals. This resulted in a total of 229 interviewees being available. A total of 206 individuals (82 agency personnel, 34 community members, 36 contractors, and 53 others) participated in the survey, resulting in a 90 percent response rate.”

2. Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection,

• Estimation procedure,

• Degree of accuracy needed for the pur­pose described in the justification,

• Unusual problems requiring specialized sampling procedures, and

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

The stratification is two–fold. Projects are stratified first by agency, then by geographic region (five regions will be used – Northeast/Lake States (CT, DE, IA, IL, IN, MA, ME, MD, MI, MN, MO, NJ, NH, NY, OH, PA, RI, VT, WI, WV), Southeast (AL, AR, FL, GA, KY, LA, MS, NC, SC, TN, VA), Northern Rockies (ID, MT, ND, SD, WY), Southwest (AZ, CO, KS, NE, NM, NV, OK, TX, UT), and Pacific Northwest (AK, CA, HI, OR, WA)). The Pinchot Institute and subcontractors randomly sample 25 percent of the projects in each strata each year. If there are less than 12 projects in a given strata the sampling rate may exceed 25 percent. For example, because of low numbers, 45 percent of the Bureau of Land Management projects were sampled in each strata during the 2008 and 2009 sample years. If the number of Bureau of Land Management projects were to increase, then potentially the sampling rate would decrease to 25 percent of the projects in a given strata (which did occur in the 2010 and 2011 sample years, per the Pinchot’s reports to the Bureau of Land Management). Collapsing of the strata, due to lack of insufficient project numbers has not been necessary to-date, as the number of projects continues to steadily increase for both the Forest Service and the Bureau of Land Management. However, if necessary, we could collapse the strata and consider a 100 percent sample of projects. As information is collected during the interview process, it is entered into a uniform report format and sent to Michigan State University for analysis. Following receipt of the data, MSU researchers code questions and responses for entry into SPSS (originally, Statistical Package for the Social Sciences) and NVivo (a qualitative data analysis computer software package) software programs used for quantitative and qualitative analyses, respectively. Since the information is needed to write an annual report to Congress, data collection cycles can not be less frequent than annual.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sam­pling, a special justification must be provid­ed for any collection that will not yield "reli­able" data that can be generalized to the universe studied.

The phone survey method is used in order to secure a high response rate. The phone interview is conducted at a time convenient for the participant. In addition the potential respondent universe includes only individuals who have chosen to be involved in some manner in a stewardship contracting project; therefore they will be familiar with the information in the survey. Currently, we have had 100 percent of the individuals contacted responding. However, if there were a selected individual who did not wish to participate in the survey, the interviewer would randomly select (using stratified random sampling based on type) another participant from the project’s list.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separate­ly or in combination with the main collection of information.

No tests or procedures are undertaken. An OMB survey number was awarded, the information collection has been renewed, and the survey has been conducted for six years.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Maureen McDonough, Michigan State University, (517) 432-2293, [email protected], designed the statistical aspects of the survey. The Pinchot Institute for Conservation and their subcontractors will be collecting and analyzing the data. The project manager at Pinchot Institute is Brian Kittler, mailto:[email protected], (503) 836-7880.

1

Page 3 of 4