ATSDR Communication Activities Survey (ACAS)
OMB Control No. 0923-0055
Reinstatement with Change
Information Collection Request
Supporting Statement – Part B
Collections of Information Employing Statistical Methods
Project Officer
Matt Sones, PhD, MPH
Communication/Program Evaluation Team Lead
Division of Community Health Investigations
Agency for Toxic Substances and Disease Registry
4770 Buford Highway, Atlanta, GA 30341
Office: 770.488.0731
Email: [email protected]
Date: November 30, 2020
B.1. Respondent Universe and Sampling Methods 3
B.2. Procedures for the Collection of Information 4
B.3. Methods to Maximize Response Rates and Deal with Nonresponse 8
B.4. Test of Procedures or Methods to be Undertaken 8
B.5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 9
Part B. Collections of Information Employing Statistical Methods
Respondent Universe and Sampling Methods
Recruitment will take place at public community meetings. Eligible respondents include a convenience sample of community members and state/local environmental public health (EPH) employees or officials who attend ATSDR meetings at each of the six to seven annually selected sites. The goal is to recruit 52-61 participants at each site per year including community members (e.g., general public, community leaders, faith-based leaders, business leaders, etc.) and stakeholders (e.g., state and local environmental health department employees, environmental health assessors, toxicologists, and officials). Obtaining this information will allow ATSDR to understand how to improve the impact of the agency’s activities in addressing environmental issues in the communities they serve. The mix of respondents will be approximately n=275 community members per year (75%) and n=92 stakeholders per year (25%).
These sites must have the potential for a harmful exposure to a population of at least 500 people. The Agency for Toxic Substances and Disease Registry (ATSDR) will select the sites as they near the end of agency site activities. Based on these criteria of population size, chemical exposure, and timing of activities, these sites may not be geographically distributed across the nation at any given time; however, ATSDR will endeavor to select at least two sites per region if possible.
Sites will be selected based upon the following criteria: a community meeting is held where the ATSDR released a site document within the past 6 months; there is an environmental hazard present at the site; and there are a large number of people potentially exposed to the hazard at the site. Table B.1.1 shows the list of sites selected for the requested three-year Paperwork Reduction Act (PRA) clearance for the “ATSDR Communication Activities Survey (ACAS).”
Table B.1.1: List of Example ATSDR Sites where the ACAS may be conducted
Site Name/Location |
Number of people with harmful exposures at each site (n) |
Region |
|
5,000 |
Central |
|
1,180 |
Western |
|
345 |
Eastern |
|
250 |
Central |
|
500 |
Central |
|
250 |
Eastern |
|
100 |
Eastern |
|
1,000,000 |
Central |
|
100 |
Eastern |
|
500 |
Central |
|
125 |
Central |
|
33,000 |
Eastern |
|
500 |
Central |
|
200 |
Western |
|
40,300 |
Eastern |
|
60 |
Central |
|
40 |
Eastern |
|
22 |
Central |
|
17 |
Eastern |
|
16 |
Table B.1.2: Potential Annual Respondent Universe
Entity |
Potential Respondent |
N |
Community Members |
General public, community leaders, faith-based leaders, business leaders |
275 |
Agency Stakeholders |
State and local environmental health department employees (Environmental Health Assessors, Toxicologists, and Agency officials) |
92 |
Total Number of Potential Respondents |
367 |
|
ATSDR typically conducts community meetings at sites where agencies implement activities to address environmental issues. ATSDR will choose sites that will have community meetings to administer the ACAS or the SMS Text Survey only one time at the end of the community meeting. This is a post-only design where data is collected after the community meeting is held to limit burden.
Administration of the ACAS –
For each engaged community, the ACAS will be used to assess a set of effectiveness indicators for ATSDR site-specific activities including respondent involvement, knowledge, satisfaction, observations, opinions, and ATSDR community engagement, interactions, and educational outreach efforts to inform communities. These questions will help ATSDR understand how its activities are making an impact in addressing environmental issues in the communities it serves.
After each of approximately six to seven such meetings per year, ATSDR will have a table set up at the entrance where they will hand out a fact sheet to the community meeting attendees (Attachment D). The fact sheet explains what ATSDR does, and the purpose of ATSDR’s site activities and the survey options.
At the end of the meeting there will be an announcement to ask interested community meeting attendees to sign in with trained ATSDR staff (Attachment E). The ACAS Sign-in Sheet Aid (Attachment E1) will be eliminated because the respondent type will be removed from the ACAS Sign-in Sheet (Attachment E). To protect the individual privacy of respondents, the survey administrators will only record and securely store whether respondents are a community member or stakeholder, and whether they prefer to take the ACAS in-person or online (Attachments F1-F2). If online mode is preferred, they will be asked to record their email addresses on the sign-in sheet (Attachment E).
Written consent will not be administered. The fact sheet and the sign in sheet will explain how respondents are asked to give voluntary consent. They will indicate they are 18 years of age or older and consent to participate in the survey by proceeding to answer questions on the survey. The first page of the ACAS will reiterate how respondents are asked to give voluntary consent.
The ACAS will preferably be self-administered by hardcopy (Attachment F1), right after the public community meeting. If this not a convenient time for the participant, the ACAS may be completed online via SurveyMonkey® link (Attachment F2). An email containing the link to take the survey (Attachment G) will be sent to those that would like to take the survey online.
Proposed changes to the ACAS are detailed in Section A.15. The ACAS will be administered by ATSDR staff after a community meeting at the site. The ACAS indicators below will measure ATSDR’s effectiveness in the following areas, which are mapped to the following questions:
respondent affiliation (Question 1)
how respondent received information from ATSDR and how they prefer to receive ATSDR information (Questions 2)
knowledge and understanding of site activities (Questions 3-6)
knowledge of how to reduce or prevent hazardous exposures at the site (Questions 7-9)
ATSDR contact information (Question 10)
observations and opinions of ATSDR site activities, including ATSDR’s understanding of respondent health concerns and whether respondent thinks ATSDR gave the information they need to address their environmental-related health concerns (Questions 11-14)
observations and opinions of the community, and the community’s ability to address environmental issues (Question 15)
questions on how the respondent feels at the time of taking the survey about his or her risk of exposure to possible environmental hazards at the site (Question 16)
demographic questions to understand the make-up of the community as an aid in tailoring messages and activities to the audience (Questions 17-23)
environmental concerns (Question 24)
These indicators will help ATSDR ascertain the effectiveness and evaluate the differences and consistency of ATSDR activities delivery and respondent perceptions across sites and over time; determine how effectively ATSDR’s site team engages community members; and discover how well ATSDR provides effective, clear and consistent communication and information on how to promote healthy and safe environments and prevent harmful exposures through the activities they implement in communities. This will help ATSDR understand whether their activities are helping the communities address environmental issues and how they may improve ATSDR’s activities to make a greater impact within the communities they serve.
The data collection tool is a survey consisting of 25 Likert-scale or open-ended questions divided into 7 sections.
Section One (three questions) measures respondent involvement in site activities. The questions help ATSDR measure activities implemented in the communities, when and how the respondent first learned about potential hazards in their community, if they live or work near the site, how they prefer to receive communication about potential hazards in their community, and whether or not respondents have shared ATSDR information with others.
Section Two (seven questions) measures respondent knowledge and understanding of ATSDR site activities. The focus of these questions is to gain insight into participant awareness of the information ATSDR has communicated to the community including understanding of the environmental hazard(s) at the site, what ATSDR does in communities, what respondents should do to prevent contact with potential hazards, what they should do if they come in contact with potential hazards, and ATSDR contact information.
Section Three (four questions) attempts to learn about respondent observations and opinions about ATSDR site activities and whether respondents feel like ATSDR understands their health concerns and has given all the information needed to address health concerns.
Section Four (one question) learns about respondent observations and opinions about their community and their community’s ability to address environmental issues.
Section Five (one question) asks the respondent to answer how they feel about their risk of exposure at the time of taking the survey.
Section Six (seven questions) asks about respondent demographics including age, gender, race/ethnicity, employment, education, and number of children.
Section Seven (two open-ended questions) asks respondents if they have any other concerns related to environmental hazards that the survey didn’t ask or if they have any further feedback.
Administration of the Abbreviated SMS Text Survey –
At select sites per year, where it is deemed that text messaging surveys will be more effective due to meetings with large numbers of attendees, an abbreviated SMS Text Survey will be offered in lieu of the paper or online ACAS. There will be no sign-in sheet for the SMS Text Survey. Respondents in the SMS Text Survey will give voluntary consent via SMS response only. Respondents will not have to provide their phone number, or any contact information.
ATSDR will administer the SMS Text Survey in the following manner. Respondents will be notified by ATSDR that at the end of the meeting that a voluntary four-question survey will be administered via text messaging. At the beginning of the survey, a PowerPoint slide with the burden statement and consent will be shown (Attachment H). Respondents who consent will be asked to text a code to enable them to participate in the SMS survey, after which each of four questions taken from the ACAS survey (Questions 6, 9, and 14) will be shown. Then they will respond by texting their answers to the questions to a number generated by Poll Everywhere™ software. Results of the survey will be displayed in real time on the screen.
Sites that usually have high participation rates at community meetings, or with medium to high involvement are part of the criteria for document selection. Sites with medium involvement have anywhere from 10-15 individuals of interest whereas a high involvement site would have a minimum of 16 individuals. This will improve response rates so that each site has 51-62 participants.
An email containing the link to take the survey (Attachment G) will be sent to those that would like to take the survey online. No follow-up for non-response is planned due to ATSDR constraints on resources.
In order to maximize response rates, around 367 community members and stakeholders will be approached each year at 6 to 7 sites per year. Higher response rates will yield more reliable information; however, no scientific inferences will be made.
The preliminary ACAS time burden estimate was obtained in 2017 as follows. A community meeting was held in July 2015, in Lockhart, SC, related to the Bennett Landfill fire in Chester, SC. Eight months after this community meeting was held, trained ATSDR contractors pilot tested the ACAS data collection tool among 5 community meeting attendees. The 2017 estimate for burden hours was based upon this pilot test. In the pilot test, the average time to complete the survey data collection tool including time for reviewing instructions, gathering needed information and completing the survey, was approximately 30 minutes. Based on the pilot test results, ATSDR reduced the number of questions and the ACAS time burden to 15 minutes per response, regardless of data collection mode. This estimate remains unchanged in the 2020 ACAS, as replacing one question and adding two more are not expected to appreciably change the 15-minute time burden per response (Attachment F1-F2).
The survey data collection tool was prepared by ATSDR staff. The data will be collected and maintained by ATSDR. Trained staff will recruit, obtain consent from respondents, and administer the survey once per respondent after each community meeting in-person, or send an email link for an online survey, or administer the survey over the phone. Data analysis of the survey findings and reporting including a draft document for internal and external dissemination will be conducted by ATSDR as well. Stata® statistical package will be used for managing, analyzing, and graphing data.
Table B.5.1. Personnel Consulted on Statistical Design
Name |
Title |
Affiliation |
Phone |
|
|||
FEDERAL AGENCY |
|||||||
Matt Sones, PhD |
Lead Public Health Analyst |
ATSDR |
(770) 488-0731 |
||||
J. Smith-George, MSW |
Senior Program Management Officer |
ATSDR |
(770) 488-0719 |
||||
Sima M. Rama, MPH |
Public Health Analyst |
Northrop Grumman Corporation |
(404) 307-3320 |
||||
Jill G. Hensley |
Program Manager |
Northrop Grumman Corporation |
(240) 750-7202 |
||||
Table B.5.2. Personnel Responsible for Collection and Analysis of Information
Name |
Title |
Affiliation |
Phone |
|
Richard A. Krop, PhD |
Principal |
Cadmus |
(424) 291-7678 |
|
Rebecca Wright |
Senior Health Writer |
CommunicateHealth |
(413) 582-0425 |
|
Sarah Pomerantz, MPH, CPH |
Project Director |
CommunicateHealth |
(413) 387-1482 |
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Smith-George, Januett P. (ATSDR/DCHI/OD) |
| File Modified | 0000-00-00 |
| File Created | 2021-01-12 |