The following is an amalgam of several similar event investigations. Details that would otherwise allow a specific company or person to be identified have been generalized, combined, or removed. The purpose of this brief study is to illustrate specific principles and work patterns that led to an otherwise-avoidable failure. The steps in this failure investigation, along with the findings, are applicable to a variety of operational and maintenance processes.
EVENT SYNOPSIS
The night watchman was making a routine round inside the shopping mall just after midnight when he heard what sounded like the report of a rifle. It was followed immediately by a shrill high-pitched hiss, loud crashes, breaking glass, and metallic ringing sounds.
The watchman cautiously turned the corner of the L-shaped shopping mall and observed what he thought might be a small missile or submarine torpedo randomly spinning, bouncing, and flying about the mall’s food court. The missile was crashing into store fronts, mall displays, and food-court equipment. Fearing some type of peculiar criminal activity, the watchman summoned the police.
SCENE EXAMINATION
When the commotion stopped and it seemed safe to proceed, the watchman found a pressurized carbon dioxide steel cylinder partially embedded near the floor in a storefront Haydite block wall. The top of the cylinder had penetrated into the wall several inches. The bottom two-thirds or so of the cylinder was sticking out of the wall.
Examination of the damaged area of the shopping mall found that the cylinder had come from a soft-drink stand in the middle of the food court. It was one of several such cylinders used to add fizz to fruit-flavored drinks. This particular cylinder was a fully-pressurized spare that had not yet been connected to any of the soft-drink dispensers.
CYLINDER EXAMINATION
An initial examination of the “missile” noted that, besides various fresh impact marks on the exterior, the top where the valve, regulator, and gages would have been coupled to the cylinder were impact damaged. There was also a small rupture at the bottom of cylinder. This approximately 2-cm long rupture roughly followed the curvature of the cylinder wall where the previously flat bottom intersected the side of the cylinder. The bottom area immediately adjacent to the rupture was slightly distended, such that the bottom of the cylinder was no longer flat enough to allow the cylinder to stand by itself.
The internals of the cylinder were non-destructively examined. First, a fiber optic camera probe was inserted through the rupture to visually examine the internal area around the rupture. The interior bottom of the tank had pitting, and there was obvious material loss in the adjacent areas to the rupture. Examination of areas within the cylinder that are normally higher than the flat bottom when the cylinder is standing upright found no loss of material or pitting.
After the fiber-optic-camera examination, the cylinder’s thickness was checked using ultrasonic testing equipment. Except for the bottom area of the cylinder, the other areas of the cylinder had appropriate thicknesses. There were no physically damaged areas in the cylinder, such as drilled holes, dents, or damaged threads at the top that pre-date the missile event.
FIELD-VISIT FINDINGS
Investigators visited the supplier of the cylinder and examined the equipment used to re-fill cylinders with pressurized carbon dioxide. No specific deficiencies in the equipment were noted that, for example, might have caused the cylinder to be over-pressurized or otherwise incorrectly re-filled.
The maintenance records of the cylinder itself were reviewed. They indicated that all the pressurized carbon dioxide cylinders provided by the company to customers were inspected approximately every five years, in accordance with Department of Transportation (DOT) safety requirements. The inspections included an internal and external visual examination, hydrostatic testing, and ultra-sonic checking of a cylinder’s critical thicknesses.
Granular-desiccant filters are used in the re-fill equipment to strip out any carry-over water vapor to ensure that moisture is not introduced into a cylinder along with the pressurized carbon dioxide. The particular desiccant used in the equipment turns color from blue to pink if the desiccant has absorbed moisture and requires replacement. All the desiccant filters in service at the time of the visit were blue.
Inventory records for desiccant replacement were also reviewed. In the past, desiccant filters were replaced more or less in regular intervals, in accordance with cylinder re-fill orders. Per company procedure, the re-fill equipment operator is to check the desiccant color before the equipment is used. If the operator notes a color change from blue to pink, he or she is to remove the old filter, request a fresh replacement from the tool crib, and then install the fresh filter. Old desiccant is collected by a vendor, regenerated by heating, and reused.
In reviewing the tool crib’s inventory records, it was discovered that no desiccant filters had been checked out for well over a year, despite an ample number of cylinder re-fill orders. Several fresh desiccant filters, however, had recently been dispensed from the tool crib right after a new operator for the re-fill equipment was hired. The tool-crib attendant, though, was not required to monitor the frequency of requests for desiccant-filter replacements. Furthermore, no second check of the color status of desiccant filters on the re-fill equipment had been conducted by a supervisor.
ADDITIONAL LABORATORY WORK
Swabs of surface film from the interior area around the rupture were collected to determine the chemical composition of any products of corrosion. The only significant compounds identified in the swabs were variants of iron carbonate.
PHYSICAL MECHANISM ASSESSMENT
The immediate cause of the rupture, which allowed pressurized carbon dioxide gas at ~860 psi to escape and propel the cylinder around the mall like a released balloon, was corrosion within the cylinder.
Moisture that had been entrained in the pressurized carbon-dioxide stream when the cylinder was refilled collected on the bottom of the unit’s surface. The pressurized gas reacted with the wetted bottom surface to form carbonic acid. Unfortunately, carbonic acid readily reacts with carbon steel. For various reasons, the reaction tends to proceed preferentially in corners and crevices. It also causes surface pitting.
The cylinder re-fill process was designed to prevent moisture from entering the pressurized carbon-dioxide stream. This design, however, depended upon consistent use of desiccant filters to remove any residual or carry-over moisture in the gas. As long as the filters were blue, any moisture was being removed. Again, the re-fill-equipment inventory records indicated that, for a period of time, the filters were not replaced, as would have been expected from prior work patterns. During that time, moisture would have been able to enter the cylinder in the pressurized gas stream and subsequently collect at the bottom of the cylinder.
ADMINISTRATIVE ANALYSIS
In the above case, there is only one line of defense present to safeguard the pressurized carbon-dioxide cylinders from carbonic-acid attack and degradation: the re-fill equipment operator replacing desiccant filters as required. Failure of that operator to do this task led to failure of the cylinder.
Moisture entering pressurized-carbon-dioxide steel cylinders and causing corrosion is not a new or unknown problem in the compressed-gas industry. The issue has been documented and discussed in various publications for many years. Thus, the argument that this is an unusual failure mode, or a one-in-a-million failure, isn’t a viable defense,
Since the failure could have had more serious consequences than a wrecked food court, a “defense in depth” strategy could have been employed to significantly reduce the risk of such incidents. There were at least three administrative points in the chain of events leading to the failure where the failure sequence could have been arrested.
1. The tool-crib attendant could have noted that fresh desiccant filters were not being requested as before, and that old ones were not similarly being turned in for reprocessing. Since most inventory records were computerized, having a periodic automatic reminder would have provided a second check on this important work task.
2. A supervisor could have periodically checked the filters on the re-fill equipment for indications of pink color. All this requires is a walk-by and visual check now and then. The double-check activity could be readily incorporated into a supervisor’s required weekly or monthly work tasks.
3. When the newest operator was hired and then replaced several filters on the re-fill equipment at the same time, no one questioned why. If the equipment had been well maintained up to that point, all the filters would have been blue when the new-hire started work.
By formally putting into a procedure at least one redundant check to ensure the desiccant filters were blue and being regularly replaced, the risk of a similar failure occurring would be significantly reduced.
It’s worth keeping in mind that humans are not 100% reliable when it comes to completing a required task. If one human is 95% reliable, however, then two doing the same task sequentially provide an overall theoretic reliability of 99.75%.
EXTENT OF CONDITION
During the period when the desiccant filters were not replaced as often as they had in the past, it’s likely that moisture was similarly allowed to pass into other pressurized carbon-dioxide cylinders. Consequently, the failure that occurred at the shopping mall could simply have been the first of more such failures to come.
Having discovered the above facts in the course of investigating the failure, the cylinder company is obligated to protect the general public, as well as its own clients. Due diligence requires the company to expeditiously check its records, locate cylinders that were re-filled during the period when desiccant filters were not regularly replaced, warn whomever currently possesses such a cylinder, and take measures to ensure that no items from this problematic group of cylinders pose a similar risk of failure.
THE NEXT STEPS
Upon discovering that a previous re-fill equipment operator had apparently failed to replace desiccant filters as required, some companies would simply blame the prior operator for the failure. In doing this, the level of risk that the company assumes with respect to a similar failure recurring has more or less stayed the same. In short, merely understanding the failure does not improve things.
In assessing the failure from an organizational point of view, however, it is seen that the path to the failure could have been arrested at several points by requiring a few, relatively simple administrative checks. Thus, the latter type of investigation provides a better roadmap to reducing the risk of a similar recurring failure as compared to conveniently placing all the blame on the prior operator.
In short, people make mistakes that can lead to failures. Organizations, however, can be designed to prevent human mistakes from becoming failures.TRR
FOR ADDITIONAL READING
NACE International Corrosion Conference 2015, Paper No. 5671, “Corrosion of Mild Steel in an Aqueous CO2 Environment,” by Tran, Brown, Nesic, Institute for Corrosion and Multiphase Technology, Ohio University, Athens Ohio, 2015.
Asia Industrial Gases Association, AIGA 062/09, “Methods to Avoid and Detect Internal Gas Cylinder Corrosion,” Barthelemy, Domer, Gabrieli, Birch, Kriese, Lleonsi, Vandereven, and Webb, available online at asiaiga.org.
American Society of Civil Engineers (ASCE), Guidelines for Failure Investigation, Greenspan, O’Kon, Beasley and Ward, ASCE 1989.
Human Error, James Reason, Cambridge University Press 1990, especially Chapter 8: “Assessing and Reducing the Human Error Risk.”
ABOUT THE AUTHOR
Randall Noon is a registered professional engineer and author of several books and articles about failure analysis. He has conducted root-cause investigations for four decades, in both nuclear and non-nuclear power facilities. Contact him at noonrandy@icloud.com.
Tags: reliability availability, maintenance, RAM, root-cause analysis, failure investigation, failure analysis
