OMB 0596-0010 Forest Industries Renewal 2022-2024 Supporting Statement B_rev

Supporting Statement B for OMB 0596-0010

Forest Industries and Logging Operations Data Collection Systems

Note: This request is for the renewal of the approved information collection OMB 0596-0010, Forest Industries and Logging Operations Data Collection Systems, which will expire February 28, 2021. The USDA Forest Service requests approval from OMB to continue the collection of information related to forest industries and logging operations and to implement two tests (varying the mill survey length and of providing financial incentives) to assess effects on response rates.

B. Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sam­pling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corre­sponding sample are to be provided in tabular form for the uni­verse as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

Mill Survey: Data is collected from a stratified simple random sample of all known primary wood-using mills using harvested wood in log or chip form, such as sawlogs, veneer logs, pulpwood, and pulp chips, to manufacture products such as lumber, paper, and biomass energy. All known primary wood products industries in the United States, and Canadian mills that process wood material from the United States, are sampled annually. Sample intensity will be dependent on the primary product and the state. For states not participating in the annual sample, a canvas of all other known wood products industries will be conducted every 2-5 years, depending on the state.

A total of 10,485 primary wood-using mills will be sampled during the 2022-2024 time-period covered by this Information Collection Request (Table 1). All primary wood-using mills will be contacted and/or accounted for during this survey for which the sample will be drawn. For respondents that are still in operation but cannot be contacted or decline to participate, historical information from previous mill responses, or other ancillary information is used. If a respondent is deemed to be out of business, it is removed from future canvasses.

Table 1.—Number of primary wood-using mills by type of mill and data collection year (including proposed tests and cognitive surveys for 1 year)

		Data collection year				Response Rate
Survey type	Total	2022	2023	2024		2018
Pulpwood Producers	363	121	121	121		73%
Other Primary Wood Products Industries	7,004	2,379	2,375	2,250		68%
Form length and incentive tests	2,598	2,598	--	--		--
Cognitive Interviews	520	520	--	--		--
Total	10,485	5,618	2,496	2,371

Tests for form length, incentives, and cognitive interviews are proposed for one year only. However, the postcard option will be implemented each year unless the first year shows low mill response.

Loggers Survey: The loggers’ survey will be conducted on active logging sites visited as part of ongoing Harvest Utilization Studies, which are carried out annually across the nation to collect information on fell tree utilization.

Table 2. Number of logging establishments (2018) and expected annual number of sites to visit

Logging Establishments	Operations per year	Average Response Rate
8,019	125-175	+85%

2. Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection,

• Estimation procedure,

• Degree of accuracy needed for the pur­pose described in the justification,

• Unusual problems requiring specialized sampling procedures, and

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Mill Survey: The annual sample of all primary processors provides a barometer of timber industry activity as well as information specific to current product demand for states that elect to participate is describe in the sample design below.

Sample frame:

The USDA Forest Service Forest Inventory and Analysis (FIA) program maintains a current list of all facilities that accept roundwood in the United States. This list will include, at a minimum, the facility’s name, location, type, primary roundwood product, number of employees, and previous total mill receipts (or estimated total receipts if the facility is new). This list serves as the sampling frame and is updated annually.

Sample selection:

Several sampling designs were tested by Coulston et al. (2018) including simple random sampling, probability proportional to size sampling, and two types of stratified simple random sampling. The two types of stratified simple random sampling were (1) creating equal size cumulative measure of size strata by primary roundwood product and sampling 2 elements per strata, and (2) using Neyman allocation and cluster analysis to determine the number of strata per primary roundwood product and the sample size for each stratum. Recommendations from Coulston et al. (2018) were to use a stratified simple random sample where strata are determined by equal sizes of cumulative total facility receipts. This approach was shown to provide estimates significantly more precise that recommended in Forest Service Handbook 4809.11.Chapter 10 (5% sampling error for annual estimates of timber cut in the eastern United States, and 10% sampling error in the western United States).

FIA uses a three-level stratified simple random sample design to sample wood using facilities. State is the first level. Primary roundwood product is the second level. The third level is developed by (1) identifying sampled with certainty strata for facilities with > 10 million cuft of receipts and then by (2) creating equally sized strata of cumulative previous facility total receipts (or estimated total receipts) for those facilities with <= 10 million cuft of receipts. Two facilities will be selected at random within each third level stratum (except for the sampled with certainty strata). The target sampling fraction within each second level strata (State by primary product) is 0.4. However, because of the sampled with certainty strata, all pulp & paper mills that receive pulpwood (primary product) will be sampled.

Estimation:

1. We use the standard direct estimators for stratified simple random sampling from Cochran (1977). Under the stratified simple random sample design each N facility (i) belongs to a single strata h. The within strata selection probability is πh=nh/Nh. The estimated population total for Y is then:

(1)

with estimated variance

(2)

where

(3)

and

(4)

is the mean for stratum h.

Equations 1-4 are used when the desired estimate follows strata boundaries. These equations are modified to construct estimates for domains or subpopulations j that cross strata boundaries.

The estimated total for Yj is then:

(5)

with estimated variance

(6).

Here we introduce the domain indicator Ij which takes the value of 1 when facility i is in the domain and zero otherwise. The indicator is then used in the construction of strata (h) and domains (j) means and variances:

(7)

and

(8).

References:

Cochran, W.G. 1977. Sampling Techniques. John Wiley & Sons. New York. 428p.

Coulston, J.W., Westfall, J.A., Wear, D.N., Edgar, C.B., Prisley, S.P., Treiman, T.,

Abt, R.C., Smith, W.B. 2018. Annual monitoring of US timber production: rationale and design. Forest Science 64 (5), 533–543. https://doi.org/10.1093/forsci/fxy010.

USDA Forest Service. 2008. Operational procedures. First report of the Blue Ribbon Panel on forest inventory and analysis. Forest Service Handb. 4809.11. Washington, DC: 11: 1-1.

For states not participating in the annual sample a canvas of all other known wood products industries will be conducted every 2-5 years, depending on the State. The information, when combined with the annual survey of pulpwood processors, will provide a complete set of industrial harvest information of the State, as well its impact on the forest resources.

Loggers Survey: Logging sites are selected assuming an infinite or uncountable population, as the population of sites depends on a state’s logger capacity but also on weather and market conditions at a given time. This makes the size of the population varying and unknown. A state’s sample size is determined using the utilization ratios’ sample error tables developed by Zarnoch et al. (2004).

Sites are selected independently from a stratified population to allocate sites to all significant primary products including saw logs, veneer logs, pulpwood, composite panels, poles and pilings, posts, firewood, and other miscellaneous as applicable. Stratification is accomplished using information from a state’s most current primary mill survey, which provides the species group and product type breakdown.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sam­pling, a special justification must be provid­ed for any collection that will not yield "reli­able" data that can be generalized to the universe studied.

Mill Survey: Respondents are provided several formats to provided information, using mail, phone, e-mail, or personal visits. These same methods are used for multiple follow up attempts to contact respondents. For respondents that are still in operation but cannot be contacted or decline to participate, historical information from previous mill response is used. If a respondent is deemed to be out of business, it is removed from the sample frame.

Loggers Survey: Loggers that allow entry to the site for fell tree data collection will be asked to participate on the survey. If a logger doesn’t allow access to the site, then a replacement site is selected if available. If the logger allows entry to the site but declines to participate on the survey, then the record will be noted as non-response. To maximize response, survey questions will be asked and recorded on-site. No follow-up is needed.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separate­ly or in combination with the main collection of information.

Two modifications to FIA’s mill collection approach will be tested to assess improvements to response rates and reduce burden in future. This is the first time these two modifications will be tested:

Test 1- Providing various survey lengths

Survey length and complexity have long been identified as factors that impact survey response rates. We propose to test the impacts of the mill survey length/complexity on response rates. The following experiment will be run in up to three states across the Northern U.S. The states will be selected based on a diversity of mill types and support from state partners. States selected for the incentives test (test 2) will be excluded. Across each selected state, the annual sample will be randomly divided into three treatments. One group will be the control (standard implementation). One treatment will consist of first sending all mills a short postcard and sending a subset of mills (the same as would be contacted in the base scenario) the full questionnaire. The third group will receive a version of the questionnaire that is roughly half the length of the standard survey and all complex questions (e.g., matrices) are minimized.

Test 2- Offering a range of financial incentives

To test the efficacy of providing respondents with financial incentives, we propose to run the following experiment in up to three states across the Northern U.S. The states will be selected based on a diversity of mill types and support from state partners. States selected for the survey length test (test 1) will be excluded. Across each state, the annual sample will be randomly divided into five groups, one control and four treatments:

• Control: no financial incentive received.

• Treatment 1: $25 debit card included with first mailing and a note added to the cover letter stating: “The enclosed debit card for $25 is to help compensate you for the time it takes you to complete the survey.”

• Treatment 2: $50 debit card included with first mailing and a note added to the cover letter stating: “The enclosed debit card for $50 is to help compensate you for the time it takes you to complete the survey.”

• Treatment 3: $100 debit card included with first mailing and a note added to the cover letter stating: “The enclosed debit card for $100 is to help compensate you for the time it takes you to complete the survey.”

• Treatment 4: $25 debit card included with first mailing, a $75 debit card will be provided after completion of the survey, and a note added to the cover letter stating: “The enclosed debit card for $25 is to help compensate you for the time it takes you to complete the survey. An additional debit card for $75 will be sent to you after we receive your completed survey.”

Although there are costs associated with the financial incentive, there will likely be a net savings to the government because of the reduced costs in conducting nonresponse follow-up and the benefit of higher response rates leading to more accurate data summaries.

Additionally, the Forest service is requesting permission to conduct cognitive interviews to test survey questions and get a deeper understanding of responses. We propose to do this by contacting 500 respondents to assess interest in participation in the cognitive interview. From these 500 initial contacts 20 respondents will be interviewed to solicit feedback on understanding of survey questions.

Sample sizes for the questionnaire length and incentive studies were calculated based on power analyses. Using the standard equation for a power analysis for differences between two samples:

And assuming:

• A 90% confidence level (z-score = 1.645);

• A pooled sample standard deviation ( ) of 0.25; and

• A desired margin of error (E) of 0.05.

The target sample size is 135 per treatment. Sensitivity analyses with varying levels of these values were also tested, but the values presented above represent our best approximations at the values. Treatment groups will be compared using a 5% margin of error.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Mill Survey: Sampling is based on methods developed by Coulston et al. (2018). Consult John Coulston (540-231-4674) for additional information on statistical aspects of the design. Data will be gathered by FIA personnel or FIA representatives, University of Montana-Missoula Bureau of Business and Economic Research personnel, and collaborators from State agencies. All analysis will be done by FIA personnel.

References:

Cochran, W.G. 1977. Sampling Techniques. John Wiley & Sons. New York. 428p.

Coulston, J.W., Westfall, J.A., Wear, D.N., Edgar, C.B., Prisley, S.P., Treiman, T.,

Abt, R.C., Smith, W.B. 2018. Annual monitoring of US timber production: rationale and design. Forest Science 64 (5), 533–543. https://doi.org/10.1093/forsci/fxy010.

USDA Forest Service. 2008. Operational procedures. First report of the Blue Ribbon Panel on forest inventory and analysis. Forest Service Handb. 4809.11. Washington, DC: 11: 1-1.

Loggers Survey: Sampling is based on method developed by Zarnoch et al. (2004). Data will be gathered by FIA personnel or FIA representatives, University of Montana-Missoula Bureau of Business and Economic Research personnel, and collaborators from State agencies. All analysis will be done by FIA personnel.

References:

Zarnoch, Stanley J.; Bentley, James W.; Johnson, Tony G. 2004. Determining Sample Size for Tree Utilization Surveys. Res. Paper SRS-34. Ashville, NC: U.S. Department of Agriculture, Forest Service. Southern Research Station. 11p.

Page 1 of 7