Support State B and Refs 0923-05AT

SUPPORTING STATEMENT B

OF THE REQUEST FOR

OMB REVIEW AND APPROVAL OF

A Site Specific Modular Evaluation Instrument for

Behavior Outcome Measurement

June 22, 2007

Project Officer: Susan Robinson, MS

Agency for Toxic Substances and Disease Registry (ATSDR)

Century Center, Building 1825, Room 3100

Atlanta, GA 30333

[email protected]

404-498-0312 (phone)

404-498-0092 (fax)

Table of Contents

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

B1. Respondent Universe and Sampling Methods

B2. Procedures for the Collection of Information

B3. Methods to Maximize Response Rates and Deal with Nonresponse

B4. Test of Procedures or Methods to be Undertaken

B5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

References

B. STATISTICAL METHODS

Communities around hazardous waste sites are extremely variable with respect to factors such as social-demographic composition (e.g., race/ethnicity, education, income, family status) and population density (rural, urban, town, suburban). This variation occurs across the universe of sites as well as may be characteristic of the population living near a site. In addition, these communities often interact with a wide range of federal, state, and local agencies, including ATSDR, over considerable periods of time – often years. Community reaction to any one of these agencies at any point in time may or may not be favorable and can influence the degree of community cooperation in responding to a behavior survey.

In implementing a behavior survey in communities near hazardous waste sites, flexibility in defining and accessing the population is critical to successfully collecting standardized data that provides sufficient information for measuring behavior. For maximum community participation, multiple options must be available to identify the respondent universe, sampling method and data collection method. The use of non-probability sampling procedures and local community networks will likely be the most feasible way for collecting standardized data on a series of questions around behavior change. These data will provide information to form conclusions about the impact of our health promotion efforts at a site.

B1. Respondent Universe and Sampling Methods

To estimate the respondent universe for this supporting statement, we reviewed the estimates of the number of persons in potential and completed exposure pathways for sites assigned a hazard category of indeterminate, public health hazard, or urgent public health hazard for the period October 1, 1993 to October 5, 2004. Because a site may have more than one potential or completed exposure pathway, the data as found in table B.1 represent a total of 747 sites and 957 pathways.

The population estimates as found in table B.1.are provided by Agency and State Partner environmental and health scientists who are responsible for completing the public health assessments at an uncontrolled hazardous waste site. Since 1986, ATSDR has been required by law to conduct a public health assessment at each of the sites on the EPA National Priorities List of uncontrolled hazardous waste sites. The aim of these assessments is to find out if people are being exposed to hazardous substances and, if so, whether that exposure is harmful and should be stopped or reduced. If appropriate, ATSDR also conducts public health assessments when petitioned by concerned individuals.

Table B.1-1 Exposure Pathway Population Estimates*

Estimated Population in Completed and Potential Exposure Pathways	Number of pathways	%	Cumulative number of pathways	Cumulative %	Number of unique sites included in this population estimate**	%	Cumulative %
10,001 – 464,000	103	11	957	101%***	88	11%	100
1,001 – 10,000	199	21	854	90%	172	21%	89
501 – 1,000	73	8	655	69%	72	9%	68
151 – 500	145	15	582	61%	124	15%	59
0 – 150	437	46	437	46%	365	44%	44

*Based on data from 1993-2004

**Note: Sites can have more than one pathway, therefore, this number will be greater than the total number of sites (N=747). For example,

Site A with two pathways – Pathway 1 is estimated to have 525 people

Pathway 2 is estimated to have 15 people

Site B with one pathway – Pathway 1 is estimated to have 100 people

Therefore, there is one site in the 501-1,000 population estimate category but two sites in the 0-150 population estimate category. In addition, if a site had two pathways with the same estimated number of persons, the site was only counted once in that column.

***Rounding error.

As is shown in Table B.1, the estimate of exposed population varies widely (0 to 464,000). However, the largest percentage of pathways is estimated to have an exposed population of 150 people or less (46%) and 90% of the pathways have an estimated exposed population of 10,000 or less. This is also about 90% of the sites.

At sites where the estimated exposed population in any pathway is less than 10,000 persons, we will attempt to survey no more than 150 households (with one person 18 years of age or older per household). For some of the smaller, less densely populated communities near a site, this may result in a census of all households in the area. Pathways with extremely large numbers of potentially exposed populations will require additional respondents. However, these are only approximately 10% of our sites (and pathways), and realistically we estimate that we would contact no more than 500 persons (one person per household).

Because of the variety of settings in which ATSDR conducts health education and promotion activities, we are proposing two options to be available for defining the respondent universe and sampling methods. These options, which are not mutually exclusive for obtaining data, are described below.

Option 1: Respondent Universe: All households in a predefined area in the site vicinity

Sampling Method: Convenience sample of households in the predefined area

All housing units in a defined area will represent the potential respondent universe. This area will be defined by the site team using maps generated using GIS hardware and software. Estimates of the number of households and population in the area will be obtained through the use of census information (e.g., census geographic boundaries and housing/population estimates). Where possible, we will access parcel maps, aerial photos, and other data in a GIS to have a more accurate assessment of the count and location of housing units.

For sites where the exposed population in a potential or completed exposure pathway is estimated to be less than 10,000 persons, we will attempt to survey no more than 150 households (with one person 18 years of age or older per household). For some of the smaller, less densely populated communities near a site, this may result in a census of all households in the area. For sites where the exposed population in a pathway is estimated to be greater than 10,000, we will attempt to survey up to but no more than 500 households.

Option 2: Response Universe: All persons attending a public availability session or community meeting

Sampling Method: Convenience sample

In this option, people who attend a public availability session or community meeting will be asked to respond to the survey. There will be no attempt to first identify if the respondent is in the area of exposure (exposure pathway) but rather we will take a convenience sample of those attending the meeting. For those respondents who provide address information, we will be able to use GIS to ascertain if the respondent is residing in the area of exposure. Without address data, we would not be able to stratify the respondents by location (i.e., residence). If agency interventions focus on persons in the exposure pathway, we would then analyze the data either stratified by location or possibly excluding those whose residence is not located in the exposure pathway.

Table B.1-2: Summary of Sites by Type, Evaluation Plan, and Number of Participants per Site

Site Type	Evaluation Plan	Number of sites annually	Number of respondents per site	Number of Responses per respondent
Existing Sites with Exposed Populations of 10,000 or Less	Post-Intervention	55	150	1
Existing Sites with New Interventions or New Sites with Exposed Populations of 10,000 or Less	Pre- and Post-Intervention	170	150	2
Existing Sites with Exposed Populations of 10,000 or More	Post-Intervention	5	500	1
Existing Sites with New Interventions or New Sites with Exposed Populations of 10,000 or More	Pre- and Post-Intervention	20	500	2

B2. Procedures for the Collection of Information

As noted in the project time line, guidelines will be developed for agency staff and cooperative agreement partners for using the survey module tool. On-site training will be provided to these staff. In addition, meeting times will be scheduled for ATSDR staff and its partners to share experience in the use of the tool. This can be done through numerous mechanisms including face-to-face meetings, conference calls, and list serves.

The specific procedures for the collection of information will vary depending on the community characteristics and resources available. In general, one or more of the following procedures is to be used for data collection. The decision on which procedure to be used will be made by the site team following the guidance to be developed. The decision will be based on several parameters that will include the estimated size of the exposed population, the geographic area encompassed, resources available, the socio-economic/socio-cultural characteristics of the community, and the anticipated level of health education and health promotion efforts. Random sampling of respondents is not expected and power tests are not performed. However, in all cases, we will attempt to select the most appropriate method to make the collection as easy on the respondents as possible, maximize response rate, and use available technology where possible.

Option 1: Respondent Universe: All households in a predefined area in the site vicinity

Sampling Method: Convenience sample of households in the predefined area

A. Single Data Collection/Pre- and Post-Intervention Data Collection

We anticipate that for approximately 75% to 80% of the sites where the survey will be used under this option, data from respondents will be collected via 1) door-to-door personal interview, 2) door-to-door drop off for pick up or mail back, or 3) telephone. However, in certain circumstances such as where resources are limited, the population is very large and widely dispersed, and obtaining telephone numbers is cost or technologically prohibitive, we may rely on mail surveys. In addition, in certain situations where it has been determined that the population to be surveyed has internet access, and/or prefers a web or email based communication strategy, the surveys will be converted to web format to allow for easier completion by those respondents. We anticipate using a mail, internet, or web format infrequently. In all cases, we will select the most appropriate method to make the collection as easy on the respondents as possible while using available technology where possible.

Option 2: Response Universe: All persons attending a public availability session or community meeting

Sampling Method: Convenience sample

1. Single Data Collection

Depending on available resources, persons attending a public availability session or community meeting will be asked at the end of the meeting to participate in either 1) a personal interview survey, 2) fill out a self-administered survey, or 3) fill out a survey on a computer-station.

B. Pre- and Post-Intervention Data Collection

Depending on available resources, persons attending a public availability session or community meeting prior to the intervention will be asked at the meeting to participate in either 1) a personal interview survey, 2) fill out a self-administered survey, or 3) fill out a survey on a computer-station. At ATSDR meetings with the community, typically a sign-in sheet is available for people to provide their name and address if they would like to be put on a mailing list for additional information as it becomes available. The survey also asks respondents to provide their name and address. This information will be used to help define the target population for health education activities and conduct a post-intervention survey to ascertain behavior change. In the event that very few people provide us with their name and address, we may conduct a post-intervention survey at another public meeting that may or may not have the same attendees present as before. Although we are not assured identical groups, generally we are assuming some commonalities between the two groups that are site related.

Attachment L has samples of a pre-intervention and post-intervention/single data collection cover letter. Attachment M has the interviewer script templates for a telephone interview and an in-person interview.

Under all options, notices about the survey may be sent to individual households, local newspapers, handed out at ATSDR/State Public Health Department public availability sessions or community meetings, and posted on the ATSDR and State Public Health Department websites informing the public of this data collection effort.

B3. Methods to Maximize Response Rates and Deal with Non-response

The collection of data primarily through household interviews and at ATSDR meetings with the community should help maximize response rates. However, a definitive figure is difficult to calculate because the agency has had limited experience in implementing household surveys (summarized below). In reviewing available agency information on survey response rates and anticipating the use of various methods to maximize response rates (resources permitting), we anticipate a response rate to the modular survey format of 50% to 65%.

ATSDR’s Division of Health Studies has calculated draft participation rates for a number of the registries for which it is responsible (e.g., for the World Trade Center Health Registry, the response rate was 60.3% calculated as the completed interviews divided by the number of interviews plus the number of non-interviews plus cases of unknown eligibility). However, registries are not directly comparable to household surveys.

ATSDR’s Division of Health Assessment and Consultation has attempted to gather information using surveys that are attached to agency prepared documents or handed out at community meetings. Response rates for FY04 are as follows:

1. 2.5% return on the customer satisfaction surveys which measure the timeliness, usefulness, and understandability of the agency prepared public health assessments. These surveys are attached to the public comment versions of the public health assessments that are sent to stakeholders (e.g., EPA, local government), libraries and other document repositories, and other interested individuals. In FY04, a total of 82 customer surveys were returned from an estimated 3360 attached to the public comment versions of the public health assessments.

2. 7% return on customer satisfaction surveys which measure the timeliness, usefulness, and understandability of the agency prepared community fact sheets. These surveys are attached to select fact sheets when disseminated by mail or handed out at a community meeting. In FY04, for 15 hazardous waste sites, a total of 9,363 fact sheets with a customer satisfaction survey attached were distributed at a meeting or via a mailout. One site had two distributions on two different dates. While the total response rate varied by site, the average response rate was 7% (sum of the response rate for each distribution/total number of distributions). The range of the response rates was 0% to 25%.

3. 11% return on targeted community health concern surveys. These surveys are handed out at select community meetings when attendees cannot stay to meet with ATSDR staff individually. These surveys may also be mailed out when ATSDR cannot hold a public availability session or when attendance at a community meeting is low in order to gather additional community concerns. In FY04, for eight sites, a total of 3,612 Community Health Concern Surveys were distributed. While the response rate varied by site, the average response rate was 11% (sum of the response rate for each distribution/total number of distributions). The range of response rates was 2% to 50%.

4. 46% return on surveys distributed at community meetings. These surveys are handed out at some community meetings to gather information on how the residents heard about the meeting, if the meeting was convenient, if the information presented was helpful and informative, and how the residents would like to be kept informed about ATSDR’s work in their community. In FY04, for 10 sites, a total of 298 Community Meeting Surveys were distributed at a meeting or via a mailout. While the response rate varied by site, the average response rate was 46% (sum of the response rate for each distribution/total number of distributions). The range of response rates was 0% to 100%.

From 1996 to 2001, ATSDR, working through the National Association of City and County Health Officials, provided grants to local public health agencies to work with communities affected by hazardous waste through a project entitled Environmental Health Education Needs Assessment Project. Grantees, with their community partners and other agencies, assessed the community’s health needs and concerns pertaining to the hazardous waste site. The projects are described in the Grant Sites Database section at the following web site – http://www.naccho.org/topics/environmental/superfund_ehena.cfm#PeerNetwork .

A number of grantees used various survey methods to help assess local community residents’ health needs and concerns. A review of the project descriptions revealed the following data about response rates:

Mail Survey

1. 1,247 surveys mailed, 50% response rate (Intermountain Waste Oil and Bountiful Woods Cross Fifth South PCE Plume – Bountiful, Utah)

2. 1,830 surveys mailed, 27% response rate (Community Involvement Surveys: Coeur d’ Alene River Basin Community, Idaho)

3. 488 surveys mailed, 35% response rate (Remington Rand Superfund Site, Connecticut)

4. 2,500 surveys mailed, 25% response rate (Torch Lake Superfund Site, Michigan)

5. 353 surveys mailed, 16% response rate (Forest Glen Site, Niagara Falls, New York)

Public Meeting

1. 150 people attended including children, 94 people filled out surveys, 63% response rate (The Turtle River Contamination Project – Georgia-Pacific Mill Site, Georgia)

Mixed Methods

1. 175 homes, 127 responses, 73% response rate (30 door to door, 79 telephone, 66 mail but only 18 completed surveys received by mail) (Spectron Superfund Site, Maryland)

2. 321 mailed surveys and those collected during the picnic, there were only 26 respondents, 8% response rate. (Greenwood Chemical Company Site, Virginia)

3. Of the 514 community members that answered the survey, 227 responded through a home visit and 287 responded via a mailing. (Oronogo-Duenweg Mining Belt Superfund Site, Missouri)

Door-to-Door

1. 642 occupied housing units, 332 surveys completed, 52% response rate. (New Jersey Zinc/Mobil Chemical Superfund Site, Illinois)

The modular format should eliminate unnecessary questions thus reducing the likelihood that respondents would become frustrated by the survey and fail to respond. Depending on resources, additional methods to maximize response rates and deal with nonresponse could include:

• Advance notification about the survey will be sent to individual households, local newspapers, handed out at ATSDR/State Public Health Department public availability sessions or community meetings, and posted on the ATSDR and State Public Health Department websites informing the public of this data collection effort

• For mail surveys, a stamped self addressed return mail envelope

• For mail surveys, one mail follow-up will be sent reminding the respondent about the survey

• For telephone surveys, the interviewer will try the phone number at different times of the day. A new number will be substituted after three attempts.

• Respondents may have the option of going to the internet to participate in the survey.

• For door-to-door surveys, if a selected respondent refuses to participate, the interviewer will proceed to the next household and continue until the pre-specified number of respondents is obtained.

B4. Tests of Procedures or Methods to Be Undertaken

Many ATSDR staff and State Cooperative Agreement Partners have weighed in on the readability of the survey and the suggested method(s) of delivery. Modifications to the survey instrument were made in response to many of the comments. These modifications were primarily in the areas of the wording of existing items, adding new items (e.g., “Limit tuna consumption”) and eliminating others (items regarding wearing a filter face mask during specific activities).

The following modules were adapted from existing sources:

Stress Module (Cohen, Kamarck, & Mermelstein, 1983; Chen & Williamson, 1988)

Health Related Quality of Life Module (Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, http://www.cdc.gov/hrqol/

Demographic Module (Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Behavioral Risk Factor Surveillance System (BRFSS), http://www.cdc.gov/brfss/)

B5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Instruments and methods were reviewed by ATSDR staff and ATSDR State Cooperative Agreement Partners. In addition, outside consultation was provided by the following individuals:

Indu Ahluwalia, PhD

Epidemiologist

Behavioral Surveillance Branch

Division of Adult and Community Health

National Center for Chronic Disease Prevention and Health Promotion/CDC

Michelle Kegler, DrPH

Department of Behavioral Sciences and Health Education

Rollins School of Public Health

Emory University

A number of individuals at ATSDR and with the ATSDR cooperative agreement states will be collecting and/or analyzing the data. The ATSDR staff contact for this survey is:

Susan Robinson, MS

ATSDR

1600 Clifton Road, NE MS E-29

Atlanta, GA 30333

404-498-1343

References

Baum, A. 1987. “Toxins, technology, and natural disasters.” In: Van den Bos GR, Bryant BK, editors. Cataclysms, crises, and catastrophes. Washington (DC): American Psychological Association: 5-54. 

Bland, A.D., Kegler, M.C., Escoffery, C, and Malcoe, L.H. 2005. “Understanding childhood lead poisoning preventive behaviors: the roles of self-efficacy, subjective norms, and perceived benefits.” Preventive Medicine 41:70-78.

Bowler, R.M. and Schwarzer, R. 1991. “Environmental Anxiety: Assessing emotional distress and concerns after toxin exposure.” Anxiety Research 4(2): 167-180.

Edelstein, MR. 1988. Contaminated communities: the social and psychological impact of residential toxic exposure. Boulder (Colorado): Westview Press.

Gatchel RJ, Newberry B. 1991. “Psychophysiological effects of toxic chemical contamination exposure: a community field study.” Journal of Aplied Social Psychology 21(24):1961-76. 

Gibbs M. 1989. Psychopathology in victims of toxic exposure. In: The proceedings of the Fourth National Environmental Health Conference, Environmental issues: today's challenge for the future, 1989 June 20-23; San Antonio, Texas. p. 25764. 

Prochaska, J.O., and DiClemente, C.C. 1982. “Transtheoretical therapy toward a more integrative model of change.” Psychotherapy: Theory, Research and Practice, 19(3), 276-287.

Seguin, C., Pelletier, L.G., and Hunsley, J. 1999. “Predicting Environmental Behaviors: The Influence of Self-Determined Motivation and Information About Perceived Environmental Health Risks.” Journal of Applied Social Psychology 29(8): 1582-1604.

Vyner HM. 1988. Invisible trauma: psychosocial effects of invisible environmental contaminants. Lexington, Massachusetts: D.C. Heath.

- 12 -