NASS Peer Review

NASS Review of OMB Docket for the 0596-NEW Environmental Justice in Atlanta GA Docket

This document summarizes my review of the OMB clearance request for the evaluation of OMB 0596 - NEW. The study evaluates environmental justness by examining three key elements: 1) cumulative hazard risk, 2) estimates of environmental services provided by the urban forest and 3) resident perception of, engagement and advocacy for trees in the city. These elements are evaluated across communities including affluent and poor/ minority and non-minority. The study is unique in that it evaluates: 1) the multiplicity of environmental burdens, 2) environmental services, as well as burdens, and 3) civic engagement in terms of attitude and actions concerning urban trees. The request for information collection only involves the evaluation of resident perception of, engagement, and advocacy for trees.

Justification

1. Data collected are necessary to comply with executive order 12898, the National Environmental Policy Act of 1969 and the Civil Rights Act of 1964.

2. Data are procured from residents of owned property and rental property regarding their support of tree planting by the city or by private residents. Information will be used by multiple data users. First, the U.S. Forest Service will use the data in participation with the Green Infrastructure Community of Practice. Second, the information collection will engage urban residents in the both public and private green space in cities as called for by President Obama’s America’s Great Outdoors Initiative. Third, data on resident engagement with Atlanta’s urban forest will be delivered to the Vibrant Cities & Urban Forests Task Force. Fourth, both basic and applied research will be generated to: 1) assess human influence on ecosystems that affect human derived benefits from ecosystems and 2) evaluate the complex relationships between different social groups and natural resource use and engagement along the urban to rural continuum. Data will be collected via personal interview, using electronic devices, and one time per address/respondent. Data will be shared to permit work of the various users listed above.

3. Data will collected by personal interview and responses will be recorded using electronic tablets.

4. The US Forest Service has researched and concluded that duplicate information has not been collected by: 1) searching the OMB website to determine if any projects related to urban residents’ perceptions of and interactions with urban trees had been submitted, 2) requested information from lead contacts of Research Stations of the Forest Service.

5. The consequences of not collecting the information would lead to an incomplete assessment of environmental justice, i.e. not providing an examination of resident perception of, engagement and advocacy of trees.

6. There are no special circumstances for data collection. The agency intends on soliciting public comment and has consulted with persons outside the agency and in multiple disciplines. The information is confidential and will not be reported accept in aggregate. There are no sensitive questions asked

7. Public burden of data collection is estimated. Two letters will be mailed. Contact sheet left with respondents or person at the door, but not correct respondent. An estimate of the time burden of the survey was given.

8. Annualized cost to the Federal Government was reported.

Comments on Justification:

It appears that there were changes to the justification section that has left some of the outline incorrectly numbered or not numbered as a result. Before submitting to OMB, cleaning up the outlined information save the time of explaining where each component is addressed.

The primary importance of the collecting the information is to more accurately reflect environmental justness. The goal is clearly stated at the beginning of the document and re-iterated in the section of the consequences of not collecting the data. In the section 16, how the data will be integrated to assess environmental justness is not well addressed. Section 14 also does not address the expenditure of time to integrate the information to assess environmental justness. In section 2, a number of uses of the data are listed, but these are not clearly addressed in time estimates in section 14 and in section 16 of what will be done with the data.

In section 2, the data are to be collected by personal interview of the household resident who last had a birthday over a two month period. At any given date that can change, especially for households in which multiple families are living. It may be better to choose a stationary measure such as oldest adult in the household or owner/renter.

In section 4, duplication of information is addressed. It is stated that there are three principal means of identifying duplicate information, but only two are stated: a review of the OMB website and contacting Research Stations.

Just a friendly reminder, some sections need updates and additional information added to them. Section 8 needs to be updated to reflect dates in which public comment is sought. In addition, a note regarding the summary of comments and response would be appropriate. Section 12 references prior mailings, but none are attached in the packet. The estimate for time is incomplete as well.

Collections of Information Employing Statistical Methods:

The target population is city of Atlanta residents. Households are selected based on a stratified random sample. Strata are defined as Neighborhood Planning Units (NPU). The percent of the sample from each stratum represents the percent of the number of improved residential units in the target population. The target sample size is 700, with an expected response rate of 85%. It is stated that there are 38 questions. The sample size was based on a simple random sample, when stratification may reduce the variability. The choice was made to estimate the response of a question with a precision of .15 with 95% confidence.

The methods to maximize response rates include: 1) providing advance notice of interview, 2) using a personal interview and 3) selecting for additional samples to be drawn when no contact may be made. The additional sample for each stratum will allow for achieving the necessary 700 samples.

The survey will be pre-tested in January of 2014. The pre-test will allow assessment of the clarity of questions, redundancy and any other issues.

Comments on Collections of Information Employing Statistical Methods:

From the statistical analysis outlined above it would appear that the primary objective is to estimate the population response to each individual Likert scale question within the survey. From the information in the introduction, I was more under the impression that the objective was to understand civic engagement as a part of determining environmental justness. This appears to be a perfect study to evaluate the odds of being more engaged from one NPU to another. Maybe I missed the point, but I would have based my sample size calculation on being able to detect a meaningful difference in the odds of being more engaged, where the odds is the probability of an event happening relative to the probability of it not happening. Given that this is the first time a study of this nature has happened, it may be well worthwhile to evaluate the internal consistency in responses using Cronbach’s Alpha. This is a unique and quite thoughtful addition to the evaluation of environmental justness, but if not accounted for in a meaningful manner it may fall short of providing the needed data.

It can be a difficult to address odds ratios in the case of Likert scale questions, because they are not binary. Most sample size calculations are based on binary assessments, but you will get the cumulative odds from common statistical procedures, which is by default binary. Many sample size calculators will allow for calculating samples sizes to assess relative risk.

In response to ensuring a measureable response, the sample size is calculated with an 85% response rate. In additional supplemental sample is used when no contact can be made. If a supplemental sample is used because a respondent cannot be reached, the response rate could not be reported as 85%. The true sample size is over 1000. I can appreciate the issue that is being avoided – in order to account for strata where samples may be harder to collect, there could be oversampling in other places. There may be other ways to account for this than doing supplemental samples.