Part B of the Supporting Statement
1. Survey Objectives, Key Variables, and Other Preliminaries
1(a) Survey Objectives
The objective of the filter adoption survey is for the EPA to: review the percentage of Denver Water and integrated systems customers (those who have been distributed a pitcher filter through the Denver Water variance filter program) who are using their filters for making infant formula, drinking, or cooking; and to confirm whether customers are using and maintaining the filters correctly -- per manufacturer’s instructions. The survey will also collect demographic information to inform filter adoption rate by neighborhood or demographic group so EPA can evaluate Denver Water’s health equity and environmental justice principles set forth in their variance.
1(b) Key Variables
The key questions in the survey will ask eligible Denver Water respondents how they are using the pitcher filters that Denver Water had distributed: whether for making infant formula, drinking, or cooking. It also asks whether customers are using unfiltered tap water, bottled water, or some other certified filtration method for the uses of making infant formula, drinking and cooking. The survey also asks whether customers are cleaning and replacing the filter cartridges in accordance with the manufacturer’s instructions. The survey asks what would increase the likelihood that the customer would use the filter more often (e.g., larger or lighter pitcher filter, etc.). Lastly, the survey asks whether the customer has a formula-fed infant in their household, in addition to several optional, demographic questions such as the customer’s race and ethnicity, household income, education, gender, primary language in the household, and the age range of the youngest person in the household.
1(c) Statistical Approach
A statistical survey approach in which a randomly drawn sample of households is asked to complete the survey is appropriate for estimating the values associated with filter use, maintenance, and to assess demographic information. A census approach is impractical because of the extraordinary cost of contacting all households. Therefore, the statistical survey is the most reasonable approach. Specifically, the target population consists of customers to whom Denver Water has distributed filters through the filter distribution program. The survey pool and the anticipated results will serve survey objectives and the EPA’s information needs. The EPA will review the survey results analysis that Denver Water submits.
1(d) Feasibility
The EPA anticipates few obstacles regarding collection of the survey results for several reasons. First, respondents will be answering questions on their daily practices regarding the use of the filter and demographic information about themselves, so the responses should be readily available. Second, Denver Water will engage in a robust communication, outreach, and education plan to promote the survey amongst respondents and to encourage participation. Third, cost should not be an obstacle for a response because respondents will able to mail the surveys back at no cost to them and they also have an online option to complete the survey. Fourth, Denver water has completed a pilot program to test the survey’s success and has created the database to support analysis (please see the sections below). Finally, the largest factor to feasibility is respondents’ time and desire to complete the survey and this has been addressed by sending 20,000 surveys to ensure at least 1,059 responses. This number is based upon historical return rates.
2. Survey Design
2(a) Target Population and Coverage
The target population will consist of all individual homes and multiple household homes that have a known, suspected, or possible lead service line (as defined in the EPA’s December 2019 variance) and who have been distributed a pitcher filter through the filter distribution program, and who are served by the community water system that Denver Water owns and operates, as well as certain integrated systems (as referenced in the Denver 2019 variance). This target population is estimated to be approximately 119,250 potential respondents. The coverage is a subset of the target population and constitutes those who receive and complete the randomly sent surveys to ultimately assess the filter adoption rate, filter use, and other demographic information.
2(b) Sample Design
For the covered population Denver Water will utilize the sample design as defined in the sections on sampling frame, sample size, stratification variables below and other sections discussed below.
2(b)(i) Sampling Frame
The set of potential respondents from which the sample will be drawn (sampling frame) are customers enrolled in the Denver Water Filter Program, meaning they have a known, suspected, or possible lead service line (LSL), as the terms are defined in the December 2019 variance. Using available data, it is estimated that Filter Program participants consist of an approximate total of 119,250 Denver Water household units, with each household unit receiving a filter. Denver Water will maintain the sampling frame and report this information to EPA as part of its reporting and recordkeeping requirements of the variance.
2(b)(ii) Sample Size
The actual number of units to be included in the survey (sample size) each year is based upon historical data from Denver Water that shows a 10% return rate. The variance requires Denver Water to collect at least 1,059 completed surveys annually to assess the filter adoption rate. To ensure compliance with the 1,059 surveys Denver will be sending out 20,000 surveys. When the 10% historical return rate is applied to the 20,000 surveys that are distributed, it provides a 941 survey reply buffer. To assess whether customers are using the filters correctly and per manufacturer instructions, Denver Water will also survey 50 homes in-person from the sampling frame.
2(b)(iii) Stratification Variables
The method used to segment the population into homogeneous groups (stratification) to reduce sampling error is filtered by those customers that are enrolled in the Filter Program, and thus have known, suspected, or potential LSLs, in contrast to customers with no LSLs. Only the population with known, suspected, and potential LSLs will receive a survey. The suspected and potential LSLs will continuously be verified through a series increasingly rigorous investigations to characterize them as either known or no LSL further refining the group. This stratification variable will increase the interest by the groups most affected to accurately complete the survey and reduce sampling error.
2(b)(iv) Sampling Method
A random sampling from the sampling frame will be conducted. Denver water has created a database that contains households with known, suspected, and potential LSLs. A random generator function will be applied to this list of 119,250 Denver Water household units each year to select where the 20,000 surveys will go each year. This will allow Denver Water to generalize the sample findings to the population of interest.
2(b)(v) Multi-Stage Sampling
A multi-stage sampling effort of respondents with known, suspected, and potential LSLs will not be conducted for selecting respondents.
2(c) Precision Requirements
2(c)(i) Precision Targets
Based on a pilot filter adoption survey that Denver Water conducted in June of 2019, the EPA believes that precision targets will be met. Through this pilot study, the minimum completed survey responses for filter adoption rates and filter performance (1,059 minimum responses and 50 homes were met).
2(c)(ii) Non-sampling error
The most serious bias is expected from non-response bias. The bias from non-response may be that they are doing something different like not using the filter but do not want to inform Denver Water. The steps Denver Water plans to take to minimize the impact on sample estimates is their communication, outreach and education plan. Denver Water is committing in this plan to:
Educate and engage with residents, customers, distributors, local public health agencies, and government stakeholders about lead awareness and reduction efforts.
Educate the public on measures they can take to reduce their exposure to lead in water used for drinking, cooking, and infant formula preparation.
Tailor and support a communications, outreach, and education program focused on expecting and existing families with formula-fed infants/children up to age 2, at homes with copper piping with lead solder, with special emphasis on homes built from 1983-1987.
Seek feedback from residents and other stakeholders to learn best practices and effective ways to implement program activities.
Strive for 100% participation in the Filter Program
2(d) Questionnaire Design
The filter adoption survey, although not formally broken down into categories, can be separated in several categories of questions.
First section: Questions about filter use. This section includes two questions, question 1 & 2.
Question 1. This question on filter use asks if the homeowner uses the filter all the time or most of the time for the purpose of drinking water. An answer of yes allows Denver Water to identify the respondents using the pitcher filter Denver Water provided. An answer of no allows Denver Water to determine if they are using an equivalent type of certified filter that can then be counted toward the number of respondents using an equivalent filter but not the one Denver Water offered. It also allows Denver to know where that equivalent filter is located in the home. The no answer also allows Denver Water to ask the number of respondents using bottled water.
Question 2. This question on filter use asks if the homeowner uses the filter all the time or most of the time for the purpose of cooking especially where water is a base ingredient. A yes or no answer allows Denver Water to identify respondents who use a filter for cooking and allows Denver Water to target educational materials to improve understanding that the filter must be used for cooking also. When the respondent answers no, several options are provided to understand where they are getting their water for cooking.
Second section: Questions about the water used for making infant formula. This section includes two questions, question 3 & 3A. These questions allow Denver Water to identify whether the household has a formula-fed infant, a vulnerable part of the population, and asks what type of water the household uses to make the formula. This information will allow Denver Water to target communication, outreach, and education to these households, if necessary.
Third Section: Questions about filter maintenance and use of the filter. This section includes three questions, questions 4, 5 & 6. Question 4 allows Denver Water to identify how many respondents are replacing their cartridges for their pitcher filters. Because replacing the cartridges according to manufacturers’ recommended replacement time ensures the continued effectiveness of the filters, if respondents are not replacing filters Denver Water asks why respondents are not replacing the filters. This is essential to targeting education materials regarding replacement of filters. Question 5: This question allows Denver Water to determine whether households are engaging in routine cleaning of the pitcher. Pitchers that are not cleaned with mild soap according to manufacturer’s recommendations may lead to higher levels of bacteria and potential pathogens. This question is also essential for targeting education materials in the proper maintenance of filters. Question 6: This question anticipates that some respondents may not be using their filters 100% of the time for drinking and cooking and asks what criteria would make them likely to use the filter more often. This gives Denver Water useful information on what the limitations are for more frequent pitcher use. If they know this then they may be able to look for products that would encourage use.
Fourth Section: Questions in this section include foundational questions to address health equity and environmental justice. This section includes six questions, questions 7, 8, 9, 10, 11 & 12. Questions on race, age, language, income, education, and gender will assist Denver Water to determine whether there are health equity and environmental justice issues. This foundational data can be overlaid with the other questions in this survey to determine whether health equity and environmental justice concerns are related to decreased filter use. Denver Water can use this information to target education materials to these areas and may increase the priority of lead service line removal.
3. Pretests and Pilot Tests
Beginning on July 8, 2019 Denver Water voluntarily conducted a pretest. As Denver Water describes in its Lead Reduction Program Plan, lessons learned from the Filter Lead Out of Water pilot outreach program (FLOW Program) that will be implemented into the full-scale FLOW Program were the following:
Provide advance targeted communications, outreach and education prior to filter
distribution to introduce the program and explain the importance of filter use.
Reinforce the importance of using the filter for cooking and infant formula preparation (in addition to drinking water).
Inform participants the filters and replacement cartridges are provided at no cost to the customer for the duration of the program.
Provide alternative filters such as refrigerator, larger pitchers, and faucet mount.
Provide additional Spanish-speaking staff for field crews for initial distribution and follow-up visits.
Have one adoption survey after the participants have been contacted, are aware of the program, and have been using the filter for period of time.
Send filters addressed to tenants, not owners of the homes, if renters reside in the
household.
Print individual participant’s survey access codes directly on their survey in order to easily track the participant’s responses.
Make survey questions clear, so that each answer doesn’t have more than one meaning.
Have more outreach materials educating customers about how the service line is
owned by the homeowner and how they can request a lead test kit.
Simplify outreach materials.
Update phone numbers in the database as project progresses.
Provide alternative filters and additional filters as filling the pitcher is cumbersome and slow.
Younger generation prefer online survey responses and electronic communications.
Not all residents have email addresses and internet access and hard copy surveys should continue to be provided.
Follow-up calls should be made from a Denver Water phone number.
Outreach staff should fill out and request a water quality sampling kit for concerned residents.
Include lead service line replacement information and talking points with filter
program.
Follow-up visits and door-to-door outreach is not preferred for all participants. Some have requested communication via email only.
Simplify survey questions to prevent confusion.
The initial communication to encourage participation included a monetary reward. These lessons learned were incorporated into the filter adoption and performance survey in this ICR.
4. Collection Methods and Follow-up
4(a) Collection Methods
The selection of collection methods will be by mail and on-line options. For the 50 homes that are also part of the LCR compliance tap sampling, Denver Water will conduct the surveys in-person. The characteristics of the survey are simple questions on filter use and do not involve any research on the respondents part. Denver Water is encouraging participation by offering a drawing for a monetary reward. Denver Water is also training their staff on these surveys. Denver Water plans on increasing the success of these collection methods by employing what they learned during the FLOW Pilot (as described above).
4(b) Survey Response and Follow-up
For the survey with the online and paper responses, Denver Water’s survey response rate is to obtain at least 1,059 survey responses. To meet this minimum response rate, as described above, Denver Water will distribute approximately 20,000 surveys to customers that are part of the Filter Program. For the in-person surveys that include customers that are part of the LCR tap sampling compliance program, Denver Water will survey at least 50 homes.
5. Analyzing and Reporting Survey Results
5(a) Data Preparation
Denver Water anticipates using the following programs to manage data related to the FLOW Pilot: Microsoft SharePoint, Microsoft Excel, and GIS. The use of SharePoint and Excel and the accelerated schedule of the FLOW Pilot, which favors ease of access and customizability compared to more structured data management platforms, like databases, which may have limited access or require advanced knowledge to manipulate. Use of Excel in the SharePoint environment avoids many of the versioning issues resulting from stand-alone Excel files while also taking advantage of distributed access and concurrent editing capabilities. Standards were set for editing information to ensure consistency and facilitate data analysis and reporting, such as applying data validation and locking columns which should not be edited. Quality Assurance/Quality Control (QA/QC) techniques were used to check the accuracy of the data. Filter recipients during the FLOW pilot were asked to respond to several online surveys related to filter use. The surveys have been administered using the survey tool, Snap Survey, to seamlessly collect data directly from FLOW Pilot participants. The collected survey data was then exported to Microsoft Excel on SharePoint for further analysis/aggregation of results. Microsoft Excel or Power BI may be used to produce a dashboard tracking metrics or interest for the FLOW Pilot.
5(b) Analysis
As Denver Water describes in the Lead Reduction Program Plan (LRPP) that was submitted as part of its Variance request, Denver Water will conduct the following evaluations to analyze the filter surveys:
1. The comparison of the adoption rate noted on completed surveys to the diverse population for each surveyed area.
2. The comparison of areas where surveys where sent but not returned or the adoption rate is low.
3. The identification of specific diverse populations where surveys were sent but not returned or the adoption rate is low.
4. The comparison of the survey results between different areas and diverse groups to establish trends by ethnic population.
5. The populations where filters were sent but surveys not returned.
6. The comparison of adoption rate to the baseline approved adoption metric.
Also, health equity and environmental justice (HE&EJ) principles set forth in the LRPP will be evaluated in the surveys.
The survey results will be geolocated based on the address, tap number, or other available spatial information. The geolocation allows the data collected from the surveys will be associated to a location and a block group for further analysis of the adoption rates and other survey results compared to the available diverse population statistics for each area.
The survey results will be compared to the neighborhood demographics. The COE program can be adjusted if there is a lack of survey results or negative feedback from a certain area is identified. If an area is systemically not participating, then the areas’ criticality (and thus risk) will be ranked as higher priority in the ALSLR Program.
5(c) Reporting Results
No later than 30 days following the end of each calendar year, Denver Water must submit an annual program year report to CDPHE and the EPA.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Denawa, Mai |
| File Modified | 0000-00-00 |
| File Created | 2021-01-13 |