Download:
pdf |
pdf TO:
Mr. Lee Sears
Materials Engineering Research Laboratory
86-68180
Bureau of Reclamation
P.O. Box 25007
Denver, Colorado 80225
DATE:
April 28, 2014
SUBJECT:
Collection and Compilation of Water Pipeline Field Performance Data
(79 FR 10842)
The following provides my assessment and recommendations for modifications to the “Buried Metallic
Water Pipe” survey which is a component of the above named project. Overall, the lack of definitions
for types of breaks, lack of collection of causal factors for the breaks and lack of a defined sample plan
are the most concerning areas.
BACKGROUND & CREDENTIALS
I have been involved in survey research in both consumer and business-to-business applications since
1981. My education includes a BS in Chemical Engineering from Clarkson University (1981) and MBA
in Marketing from Xavier University (1988). My career includes over ten years in product development
at Procter & Gamble, three years in marketing research and brand management at Church & Dwight
(Arm & Hammer brands), and six years in marketing research at Coors Brewing Company. I have been
in private practice since 2001 (Mann Consulting Inc.) providing clients with a wide variety of marketing
research and consulting. This experience includes primary responsibility for the American Water Works
Association’s annual State of the Industry issue assessment and analysis (2003-2012) and various other
water industry surveys for AWWA and the AWWA Research Foundation (now the Water Research
Foundation).
INITIAL SCREENING
Invitation Letter. “However, there is a possibility that a request for the data could be made
under the Freedom of Information Act.” This type of declaration will lead to response bias due
to respondent concerns about the possibility of data they provide being publically disseminated.
A6. To allow for better segmentation and balancing of the eventual utility sample after
collection, states should be listed individually in the drop down menu in alphabetical order rather
than in predetermined regions.
B1i. It will be critical to specify in advance the soil corrosivity data requested in the survey will
be for the specific soils around the breakage, and not a general soil corrosivity profile throughout
a given utility’s service area. Generalized regional soil information may not provide adequate
understanding of the causal factors in pipe breakage if a utility has a wide variety of soils present
in its service area.
MANN CONSULTING, INC. P.O. BOX 270887 LITTLETON, CO 80127-0015
COLLECTION INSTRUMENT
Language. The language used in the collection instrument should be tailored to a high school
graduate level at a maximum given the typical credentials of those at utilities, in particular
smaller utilities, who are most likely to provide this information.
Invitation Letter. The stated expected completion time of “up to 60 minutes” seems
insufficient, especially for large utilities that may have numerous breaks to report and/or may
require significant manipulation of their internal datasets to report the information as requested.
Data Collection:
Unless all of the data is collected only from drinking water, it is critical to provide a
column to specify the liquid(s) being transported within the pipe (e.g., raw water, treated
water, storm water, sewage, etc.) to understand the internal reactions that might be
occurring between the liquid and the interior of the pipe.
“Break Type” definitions should be provided so that respondents across different utilities
are reporting the same types of breaks in the same manner. This may require subcategories including location of break (mid-pipe, at joint, etc.). As the debate over
allowable break frequency or pipe service life ensues, understanding what types of breaks
will likely be critical to assessing performance standards. Additionally, the types of
breaks occurring may help point to installation issues or other causal factors that are not
inherent to the type of pipe as well as help assess the adequacy of various protection and
maintenance methods (such as corrosion control).
Causal information regarding breaks is critical, and should be added to the data required
for participation and requested from eventual utility participants. Forensic understanding
such as the type(s) of causal factors likely involved in the break is important to
understanding the role of material in the failure. If causal factor data are not available in
a utility’s database, they should be excluded from the sample due to this insufficiency.
It would be beneficial to better understanding causal factors in breakage to also be able to
cross-reference other site conditions that can significantly contribute to breakage such as
the presence of stray current (nearby light rail operations or other stray current sources),
bury depth and/or exposure, roadway or other surface traffic conditions that would lead to
cyclic stress, presence of fixture restraint to compensate for hammering and surges, and
pipe installation (such as if a water transmission line is installed within or crossing
through a larger sewer or storm water pipe).
SAMPLE CONSIDERATIONS
While the supporting documents I reviewed outline specifics of the survey instrument in detail, I was not
able to find similar clarity in the specifics of the sampling plan for the study. The selection of utilities to
include in the database can introduce significant response bias if important factors such as installation,
maintenance and soil conditions are not adequately understood and balanced in the database.
MANN CONSULTING, INC.
P.O. BOX 270887 LITTLETON, CO 80127-0015
Utility Installation and Maintenance Capabilities and Practices. Installation and
maintenance capabilities and practices are likely key variables in the relative pipe breakage
experience between utilities. It is easy to imagine significant sample bias if, for instance, utilities
that predominantly use one type of pipe have poorer installation skills or maintenance programs
than utilities that predominantly use a different type of pipe. Great care in balancing the utility
sample base will be necessary, as well as perhaps standardizing and normalization of the
resulting data base post collection.
Soil Conditions. Related to the screening question for soil conditions, it will be critical to select
utilities for the study to balance or allow for eventual standardization of the dataset by important
soil conditions such as corrosivity.
Thank you for your consideration of these comments.
Sincerely,
John Mann, BSChE, MBA
President
Mann Consulting, Inc.
P.O. Box 270887
Littleton, CO 80127-0015
MANN CONSULTING, INC.
P.O. BOX 270887 LITTLETON, CO 80127-0015
TO:
Mr. Lee Sears
Materials Engineering Research Laboratory
86-68180
Bureau of Reclamation
P.O. Box 25007
Denver, Colorado 80225
DATE:
October 30, 2014
SUBJECT:
Collection and Compilation of Water Pipeline Field Performance Data
(79 FR 10842)
The following provides my further assessment and recommendations for modifications to the “Buried
Metallic Water Pipe” survey which is a component of the above named project. I appreciate that some
of my feedback from the prior review (including adding soil corrosivity documentation) was addressed
in the latest review.
However, I remain very concerned limitations in clearly defining key terms and documenting causal
factors will negate the utility of this study as currently proposed and defined.
BACKGROUND & CREDENTIALS
To review my credentials, I have been involved in survey research in both consumer and business-tobusiness applications since 1981. My education includes a BS in Chemical Engineering from Clarkson
University (1981) and MBA in Marketing from Xavier University (1988). My career includes over ten
years in product development at Procter & Gamble, three years in marketing research and brand
management at Church & Dwight (Arm & Hammer brands), and six years in marketing research at
Coors Brewing Company. I have been in private practice since 2001 (Mann Consulting Inc.) providing
clients with a wide variety of marketing research and consulting. This experience includes primary
responsibility for the American Water Works Association’s annual State of the Industry issue
assessment and analysis (2003-2012) and various other water industry surveys for AWWA and the
AWWA Research Foundation (now the Water Research Foundation).
COLLECTION INSTRUMENT
The importance of understanding the type of failure to the eventual utility of the analysis cannot
be overstated.
B1m is currently labeled “Break/Leak Type.” My understanding from the field is those close to
operations see breaks and leaks as very different events, and these need to be tabulated and
classified separately.
MANN CONSULTING, INC. P.O. BOX 270887 LITTLETON, CO 80127-0015
Terminology is absolutely critical to the analysis of the subject pipe technologies. Terms in both
the types of breaks and leaks (separate and specific to either breaks and leaks) and causal factors
(separate and specific to either breaks and leaks) need to be specified more completely in the
review process so those reviewing this study can make adequate input. At this juncture, the level
of specificity of terms available for comment is insufficient to draw firm conclusions as to their
appropriateness.
I would suggest two remedies:
1. If it has not already been done, develop terminology with field operations personnel
and/or those persons from utilities who are expected to be tasked with completing the
survey. If terms used in the survey are ambiguous, confusing or unknown to those
completing the survey, or if the terminology is not consistent or translatable to utility’s
internal terminology, the data collected will be substantially useless.
2. During data collection, survey takers should be provided with an option to link to more
explicit descriptions of the examples in both these questions to provide for consistency in
reporting critical to the underlying analysis. And upon completion of the data collection,
those analyzing the database should perform an audit to confirm or refute that the
classifications were used consistently across responding utilities, and that the
classifications used represent the actual failure event as accurately as possible given the
utilities’ available records.
Further regarding B1m, the examples currently given in parentheses following the title “Break
Type” (specifically, “construction defect, corrosion, settlement, frost heave, etc.”) seem to be
causal factors and not descriptors of failure types. Ideally, these improper examples of breaks
and leaks would be replaced by terms developed in the process described above.
Following the split Breaks and Leaks B1m, there should be an additional question titled “Cause
of failure” which would then use terms similarly developed in the process described above.
In the current comments (Federal Register/Vol. 79, No. 190/Wednesday, October 1,
2014/Notices) there is a response to the comment regarding the importance of understanding
causal factors stating, “This question is included in the survey. While we agree this piece of
information is important, we expect many utilities may not document the causes. Because this
column will be in our database, we will be able to compare datasets with and without this data.
We are not planning to exclude utilities that do not have this data.”
Categorically, information regarding pipe failure without notation of causal factors is of very
limited value to drawing sound conclusions about pipe performance, and instead of providing
clarity and sound direction will likely contribute to misleading or spurious conclusions.
MANN CONSULTING, INC.
P.O. BOX 270887 LITTLETON, CO 80127-0015
For example, as industry practices improve with experience, failures due to improper or
insufficient installation practices might be eliminated or substantially reduced as these failures of
practice are identified and better understood. The intrinsic value of a particular pipe technology
would be unnecessarily clouded by past insufficient and improper installation and maintenance
practices that may have long been remedied. Failures due to these remedied practices should be
factored out of the analysis of the underlying voracity of the pipe technology.
In the comment that immediately follows (also regarding causal factors), the response is, “Some
of these factors will be difficult to collect for many break events. While these data could be
important, we do not want to require all of them for fear it would create an undue burden on the
respondent. Burial depth has been added to the survey.”
With respect to undue burden, the critical nature of this analysis and the potential for the findings
to affect billions of dollars in infrastructure investments in the broad water utility sector for
generations to come seems to warrant imposition on utilities for this causal data of acknowledged
importance.
Perhaps the undue burden is actually having utilities without causal factors noted in their
database participate in the first place. Their efforts will not contribute importantly to a sound
scientific understanding of the technology. Worse, the presence of data without well-defined and
well-understood definitions and causal factors will undermine the scientific foundation of any
conclusions derived from this inadequate data. The presence of incomplete information is not
necessarily better than the absence of incomplete data in the analysis.
Thank you for your consideration of these comments.
Sincerely,
John Mann, BSChE, MBA
President
Mann Consulting, Inc.
P.O. Box 270887
Littleton, CO 80127-0015
MANN CONSULTING, INC.
P.O. BOX 270887 LITTLETON, CO 80127-0015
TO:
Mr. Lee Sears
Materials Engineering Research Laboratory
86-68180
Bureau of Reclamation
P.O. Box 25007
Denver, Colorado 80225
DATE:
December 1, 2014
SUBJECT:
Collection and Compilation of Water Pipeline Field Performance Data
(79 FR 10842)
Now that revised “Supporting Statements A and B” have been released, I wish to offer further comment
on the “Collection and Compilation of Water Pipeline Field Performance Data (79 FR 10842).”
BACKGROUND & CREDENTIALS
To review my credentials, I have been involved in survey research in both consumer and business-tobusiness applications since 1981. My education includes a BS in Chemical Engineering from Clarkson
University (1981) and MBA in Marketing from Xavier University (1988). My career includes over ten
years in product development at Procter & Gamble, three years in marketing research and brand
management at Church & Dwight (Arm & Hammer brands), and six years in marketing research at
Coors Brewing Company. I have been in private practice since 2001 (Mann Consulting Inc.) providing
clients with a wide variety of marketing research and consulting. This experience includes primary
responsibility for the American Water Works Association’s annual State of the Industry issue
assessment and analysis (2003-2012) and various other water industry surveys for AWWA and the
AWWA Research Foundation (now the Water Research Foundation).
SUPPORTING STATEMENT A:
2.B4 (page 5): Minimum standards for utility participation are set forth in this statement to be
the following (referencing 2.B1):
a. Pipe Material
c. Pipe Installation date
j. Pipe Condition Monitoring System and Year Installed
These three criteria seem quite inadequate to allow for even crude analysis of the complex
science of pipe technology performance in question. This list of minimum criteria for
participation should be expanded to include not only more of the forensic information included
in other data items listed in 2.B, but also include additional forensic information that is pertinent
to understanding true causal factors not currently addressed in 2.B1 as I stated in my most recent
letter to you dated October 30, 2014. Without more robust minimum criteria, the validity and
utility of this survey are highly questionable.
MANN CONSULTING, INC. P.O. BOX 270887 LITTLETON, CO 80127-0015
7.f. (page 7):
It is imperative the collection instrument has clearly defined and consistent terminology
for breaks, leaks and causal factors that are understood by those persons at utilities tasked
with completing the data transfer. Further, this terminology must be clear and consistent
with the terminology used in the subject utilities’ databases such that a person completing
the collection instrument is transferring breakage, leakage and causal factor data that
accurately represent events recorded in their databases. The person providing the data for
this study may or may not be a subject matter expert, and may therefore not be able to
reliably translate definitions and terminology used in their database to potentially
different definitions and terminology used in the survey instrument.
Given this potential for incorrect translation, the program to analyze the utility data
should include an audit of a subsample of utilities to understand the accuracy with which
the true life experiences are being portrayed in the data collected by the survey
instrument. Without clear and consistent terminology, the opportunity to misreport
events of one nature as something substantially different is too significant to ignore. Best
practices suggest that Battelle should clear up any differences or ambiguity in
terminology, especially for breaks, leaks and causal factors prior to deploying the survey
instrument for data collection. And then upon completion of the data gathering, Battelle
should audit a sample of data gathered to be sure what is transferred to the database for
analysis accurately represents real world experience.
16. (page 17): To be in compliance with the stated requirement “Address any complex analytical
techniques that will be used” Battelle must come forward with a plan that addresses how they
will deal with incomplete data. As currently constructed, the project will allow for data
collection from utilities that do not have complete data regarding failure types and causal factors.
The manner in which Battelle proposes to use incomplete data in their analysis is critical to
understanding the validity of their work. As I have stated in the past, the lack of clear and
understood causal factor data should disqualify utilities from having the rest of their incomplete
data included in the analysis. It is a waste of the utilities’ time and, if misused, the incomplete
data can result in misleading findings and conclusions.
SUPPORTING STATEMENT B:
1. (page 1) and 2b (page 2): Again, the advance review of Battelle’s proposed analysis and
statistical treatment of the non-representative sample base proposed in this study is critical to
understanding the value and validity of Battelle’s work.
The sample universe includes Federal water facilities and larger/very large water utilities.
While this sampling strategy seems convenient, this sample pool is likely not
representative of the universe of utilities and water facilities as a whole.
Page 2 of 3
MANN CONSULTING, INC.
P.O. BOX 270887 LITTLETON, CO 80127-0015
Given the estimates cited in Table 2-1 “Number of Utilities and Facilities” there will be a
substantial overrepresentation of Federal water facilities. With 92 of 250 Federal water
facilities expect to report versus 68 of 418 water utilities expected to report, Federal
water facilities will represent 57.5% of the reporting entities while they represent only
37.4% of the universe of entities invited to participate. That alone significantly skews the
response in a substantial manner.
QUESTIONS STILL UNADDRESSED FROM THE FEDERAL REGISTER NOTICE
On page 59293 of the Federal Register/Vol. 79, No. 190/Wednesday, October 1, 2014/Notices there is a
comment beginning on the bottom leftmost column beginning with “The Bureau of Reclamation has
obtained the services of an outside….” etc. that goes on to state “4. Internal corrosion, external
corrosion, or both? How to you define and quantify a corrosion related failure? By percentage cause or
other method?” While the Response states, “Supporting Statements A and B have been revised and
clarified to address these questions,” I see no detail regarding corrosion adequately addressing the
questions stated under #4 above. Given the importance of corrosion in the understanding of pipe failure,
this lack of clarification is concerning.
TOTAL LIFE CYCLE ANALYSIS
The current study as proposed is limited to understanding the failure rates of various pipe technologies.
But failure rates are only a portion of the consideration for the best overall choice of pipe technology.
To better serve the public and the water industry, it would seem prudent to expand the scope of the study
under consideration to assess the total life cycle economics of the subject pipe technologies. Other
important factors include initial costs of purchase and installation, maintenance, repair costs and losses
due to downtime, and ultimate life expectancy.
Thank you again for your consideration of these comments.
Sincerely,
John Mann, BSChE, MBA
President
Mann Consulting, Inc.
P.O. Box 270887
Littleton, CO 80127-0015
Page 3 of 3
MANN CONSULTING, INC.
P.O. BOX 270887 LITTLETON, CO 80127-0015
| File Type | application/pdf |
| File Modified | 2016-12-09 |
| File Created | 2016-12-09 |