C-130 Fireball Due to Modification Error
A botched modification program unnecessarily changed the original design for a hydraulic system modification when a drawing was misread. A longer hydraulic hose was necessary after that change but an inappropriate design process short-cut meant the shorter hose remained listed as an alternative. The shorter hose was fitted to the modified fleet and resulted in chaffed electrical wiring and a fire that damaged an aircraft beyond economic repair over ten years later.
Introduction and the Accident Flight
During a touch and go at NAS Key West, on 21 Feb 2012, a fireball erupted in the back of Royal Canadian Air Force (RCAF) Lockheed Martin CC-130 / C-130H(T) Hercules Air to Air Refuelling tanker CC130342, just in front of the 25,000 lbs, 3,600 US gallon cabin mounted AAR tank.
According to the recently issued investigation report:
Concurrent with the fire alert, the aircraft became airborne and reached 10 feet in altitude above the runway. With sufficient runway remaining, the Flying Pilot landed straight ahead and aggressively stopped the aircraft…all nine crewmembers quickly egressed and moved upwind of the aircraft.
Crash Fire and Rescue services responded and expeditiously extinguished the fire.
The aircraft was extensively damaged [beyond economic repair] and one crewmember received a minor injury.
The aircraft had a hydraulic system modification (CF-378) to install ground test connections to the auxiliary hydraulic system, just below an electrically driven pump.
This safety investigation identified routing and clamping deficiencies in the modification that resulted in chafing between the hydraulic pump electrical wiring and a hydraulic flexible hose. Electrical arcing resulted in a pin-hole breach of the hose, release of high pressure hydraulic fluid and ignition of the fire.
A re-design followed, along with more education on the hazards associated with chafing. Other observations focused on the dual layer clothing principle for aircrew fire protection and improving communication between airworthiness authorities when imposing and lifting operational restrictions.
The Modification and the Fireball
Modification CF-378 was originally designed in 1976 for the first five Canadian C-130Hs. CC130342 was one of the third batch of RCAF C-130sHs, ordered in 1990 as tanker aircraft. The RCAF Weapon System Manager (WSM) subsequently requested that CF-378 be embodied on this batch and two further C-130H-30s during contracted base maintenance.
However, the contractor noted that they could not embody modification CF-378….as the auxiliary hydraulic system on the CC130 H(T) and H-30 aircraft were different than depicted in the modification instruction… To facilitate the work, the third line contractor produced a Maintenance Production Permit (MPP) in which a “standard repair” was developed to change the CF-378 modification to accommodate the aircraft configuration. The WSM approved the MPP and the first aircraft to embody the MPP version of CF-378 was CC130341 in 2001.
The modification was embodied on CC130342 on 1 February 2002.
During the investigation the CF-378 modification’s flexible 28.25 inch steel braided hydraulic hose was found to be in contact with the hydraulic pump motor power cable.
…revealed there was a 2 mm diameter hole in the stainless steel braiding, at the site where it was found in contact with the hydraulic pump motor power cable…
Examination of the hose braid indicated deposits of copper and tin consistent with the material in the auxiliary hydraulic motor power cable.
Examination of the hydraulic pump power motor cable revealed broken wire strands…consistent with the contact point to the flexible hydraulic hose.
When hydraulic fluid is released under pressure from a small orifice, an atomized spray of oil droplets is produced that may extend as far as 10m from the break. This atomized spray can be readily ignited by open flames, hot surfaces or electrical arcing and the resulting fire is usually torch-like, with a very high heat release rate.
It is likely that after the initial jet of orange flame, the flame self-extinguished. However, hydraulic fluid continued to spray and atomize under high pressure from the leak, dispersing as a fine mist within the cargo ramp area… It is likely that a second arcing event ignited this rapidly expanding cloud of hydraulic mist, resulting in the subsequent orange fireball from the ramp area.
Secondary combustion of material then likely occurred…sustained by hydraulic fluid continuing to…burn …accelerated by internal airflow patterns and by sources of pure oxygen. Of particular note, the aluminium tubing carrying gaseous oxygen to the regulator at FS737 just aft of the para door crosses the ceiling in the vicinity of the main frame at FS737. Aluminium melts at a relatively low temperature of approximately 660 degrees Celsius (ºC). The high temperature of the secondary fire likely consumed the oxygen line, allowing gaseous oxygen to escape. This breach of the oxygen line would have greatly intensified the fire until the gaseous oxygen had been depleted, resulting in the rapid burn through of the aft fuselage ceiling.
Aerossurance has previously discussed an oxygen fire on a US Air Force (USAF) RC-135V Rivet Joint reconnaissance aircraft 64-14848 at Offutt Air Force Base in 2015. In that case, the oxygen leak was due to mis-assembled oxygen couplings but also resulted in a cabin roof burn-through.
The technical investigation also noted that the circuit breakers for the auxiliary hydraulic pump motor did not open during the accident. The circuit breakers are an older bi-metallic thermal activation type (MS25244), which are not designed to respond to short duration arcing events. As a result, power continued to be supplied to the pump motor, creating the potential for multiple arcing events.
The Modification – Design Changes Made During Installation
The first aircraft to be fitted with modification CF-378 using the MPP was CC130341. While attempting to fit CF-378, the contractor had difficulty in implementing the modification in accordance with the modification leaflet. The contractor developed new routing for the hydraulic lines to “meet the intent” of modification CF-378, and raised and staffed a MPP to embody the revised configuration as a “standard repair”.
Readers with design experience will rightly immediately be suspicious that a ‘repair’ was being used as an inappropriate means to approve a modification. The investigators comment that these changes:
…were significant enough that they should have been dealt with as a new modification instead of a standard repair. In effect, the urgent MPP, approved as a standard repair, short-circuited the modification process and resulted in an insufficient review of the design changes proposed.
This short-cut appears to have been taken because of time-pressures to complete the aircraft maintenance but there were also further time pressures to quickly approve the MPP:
The contractor submitted the MPP to the Life Cycle Material Manager (LCMM) with the instruction to “treat as urgent”, as flight test of the aircraft was scheduled for the following week. The date of this MPP request was 30 June 2000, the Friday before the Canada Day long weekend.
…the LCMM relayed this sense of urgency to a member of the Director of Technical Airworthiness (DTA) staff who was requested to review and comment on the MPP. The DTA staff member concurred with the work procedure and engineering disposition given in the MPP on 04 July 2000, the first day back to work after the long weekend. The LCMM signed the DND approval…on 06 July 2000. No evidence was found in the MPP documentation as to the extent of the DTA or LCMM review conducted.
The rushed approval of this ‘repair’ is likely to have meant an abbreviated examination of the design. The investigators examined the installation differences (emphasis added):
The MPP stated “locate bracket so that cut-out in bracket assy clears FWD INBD pump mount” and also directed that the bracket assy be located in-board so that it sat flush against the in-board lip of the tray. By contrast, the original CF-378 detailed that the test connection support bracket was to be located 2 ½ inches from the forward edge of the tray and 5 inches from the in-board lip of the tray.
Among the resulting changes:
The 28.25 inch hose from the pressurized test connection to the aircraft system also had to span an extra 5 inches, also changing its natural routing and creating greater potential of chafing against the back edge of the pump tray and the electrical power supply wiring for the pump. This hose was lengthened from 28.25 inches to 38.25 inches in the MPP.
However, the investigators concluded that the original CF-378 design would have still been compatible if the bracket had not been relocated. So, why was the bracket relocated? The investigators found:
A comparison of the original  and re-issued  CF-378 modification leaflets revealed that the installation drawing, provided as Figure 2, had been converted from three orthographic views from the front, bottom and side (views A, B and C) in the original CF-378 leaflet, to a single three-dimension isometric view in the re-issued CF-378 leaflet. However, the detail information placing the support bracket 2 ½ inches from the forward edge of the tray and 5 inches in-board was omitted in the re-issued CF-378 leaflet.
Both the original and re-issued modification leaflet CF-378, Figure 6 provided a drawing of the test connection installation. The CF-378 modification documentation folder was found to contain a photograph of the original prototype installation which was used to create this drawing.
The photograph shows that the correct placement of the bracket was to be placed outboard of the drain channel for pump tray. However, in converting from the photograph to the drawing, details of the drain channel and forward in-board mount were omitted, creating an ambiguity with respect to the correct placement of the bracket, in that it appears that the bracket is placed along the in-board edge of the tray.
The investigation concluded that the absence of placement information in Figure 2 of the re-issued CF-378 leaflet, combined with the ambiguity created by the Figure 6 drawing, likely resulted in the contractor misinterpreting the correct placement of the support bracket as being along the in-board edge of the pump tray.
This led to the contractor erroneously placing the support bracket five inches in-board of its intended location, leading to an otherwise unnecessary series of configuration changes.
The use of the repair process as a short-cut also resulted in latent deficiencies in the RCAF technical documentation. The MPP listed the 28.25 inches and 38.25 inch hoses erroneously as alternatives to each other, not specific to the different individual aircraft and their different bracket locations.
The seven H(T)90 and H-30 aircraft with the MPP modification were examined after the fire. All were found to have had the shorter 28.25 inch hydraulic hose installed. Consequently the clearances did not meet the original intent. Three were found with signs of chafing by squadron personnel who were concerned at the lifting of the post-accident operational pause (see below).
Missed Warning Signs – Investigation Bias?
[Flight Safety Occurrence Management System report] FSOMS 123703 (17 Oct 2005) states that “during an engine run-up, the cargo compartment of CC130334 was found to be “filled with mist and smelled like hydraulic fluid” and “the cargo floor was soaked with hydraulic fluid”. In that occurrence, the source of the leak was found to be the 28.25 inch braided line from the CF-378 pressurized test connection that was chafing against the auxiliary pump tray. The tray was found to have a relief hole/slot cut into the back edge and the corner of the cut-out had caused the rupture in the line, allowing 11 litres of pressurized hydraulic fluid to be sprayed out into the cargo compartment.
…it was thought [at the time] that the tray had been modified without authorization and the preventive measure was to brief personnel on the hazards of making unauthorized modifications.
It was not identified that the 28.25 inch was in fact incorrect, due to the alternative part error introduced in technical documentation when the MPP was introduced.
However, on 04 May 2012, investigators were examining CC130334 at 8 Wing Trenton while investigating modification CF-378 and noticed that the “cut-out” still existed on the tray, nearly seven years later.
Additional examination revealed that both the tray and the shelf support bracket had been subject to abrasion by the braided steel hydraulic line.
The accident investigators concluded that:
…it was possible that the “unauthorized modification” cause factor in FSOMS 123703 was in fact the result of long-term chafing of the steel braided line against the aluminium alloy tray and shelf support.
There is a hazard in safety investigations, particularly if investigators are trained specifically as maintenance error investigators (for example), of approaching with the assumption of maintenance errors or violations and due to that bias, incorrectly identifying the failure mode and so not uncovering the real factors behind an occurrence. Prof Erik Hollnagel called this cognitive bias ‘What You Look For Is What You Find’ (WYLFIWYF).
Design Standards and Maintenance Standards
The investigators concluded that:
…the CF-378 modification leaflet does not contain adequate provisions for the routing and clamping of hydraulic hoses to prevent chafing and wear.
Detailed routing and clamping instructions are required to ensure the pressurized lines leading to/from the pressurized test connections are secured and do not create a chafing hazard.
While the investigation was not able to identify if the short hoses were installed at installation or during subsequent maintenance:
…investigators visited the current CC130 R&O contractor to conduct a Flight Safety survey in June 2013. During this visit, a walk-through of CC130337 undergoing Periodic Inspection revealed that several electrical wires were in direct contact with utility system hydraulic lines, and that these deficiencies had not been identified during the inspection phase of the Periodic Inspection.
Subsequently flight safety reports have continued to be generated on this topic, however:
It is of concern that problems are not being detected at squadron level and during third line R&O maintenance, in spite of numerous scheduled visual and detailed inspections of these areas.
Routing and clamping has been a previous issue upon which [RCAF Directorate of Flight Safety] DFS conducted a prevention campaign in the late 1990s and was on the “Top Ten” safety concerns list in that era. In 1998, the fire on-board [Boeing Vertol CH-113] Labrador CH11305, which resulted in the loss of the aircraft and six crew members, was likely initiated as a result of chafing on the main fuel line of the #2 engine. In 1994, [Sikorsky CH-124] Sea King CH12425 crashed, with two crew members killed and two seriously injured, after an on-board fire that was initiated by a leak in a main engine fuel line that was chafed through by a drain line.
Since 1996 [when the FSOMS taxonomy was changed], over 1000 occurrences have been identified [by the RCAF] as being caused due to routing and clamping and/or chafing, many involving wiring and/or flammable fluid carrying lines. While the FSOMS reports dropped from approximately 100+ per year initially, to approximately 20 to 30 per year by 2005, FSOMS occurrence reports seem to be gradually increasing with 39 reports generated in 2014. This issue spans across the entire Royal Canadian Air Force (RCAF) and no fleet is immune to this problem.
Computer based Electrical Wiring Interconnect System (EWIS) training is now being delivered to RCAF technicians.
The investigation also highlighted an unrelated routine non-compliance in the fleet:
Modification CF-694 requires the reservoir drip pan drain valve for the auxiliary hydraulic system to be lockwired in the open position. During the course of the investigation, closed reservoir drip pan drain valves were observed on other CC130 aircraft…Squadron personnel indicated that even though the modifications and inspections had been signed off, the actual practice was to close the drain valve as the opening to the external fuselage created a constant pressurization leak and an annoying whistling noise during flight.
Post-Accident Risk Management / Operational Pause
The fire damage to the upper fuselage initially lead to the incorrect suspicion that the cause was an oxygen fire. An initial post-accident operational pause was lifted following a fleet check of oxygen pipes in the cabin. The investigators highlight that the pause and its revocation was not communicated to the investigators Technical Airworthiness Authority or WSM. Lifting the pause did however have an unintended consequence:
…a 435 Sqn member, unsatisfied with the explanation for rescinding the operational pause, took it upon himself to conduct an unofficial inspection of the aircraft at 435 Sqn, resulted in the finding that hydraulic lines associated with the ground test connection modification were chafing the auxiliary hydraulic pump motor power cables on three additional aircraft [mentioned above]
The pause was re-initiated as it was realised that a hydraulic system problem was in fact more likely.
…new modification leaflet C-12-130-000/CD-154 (CD-154) removes the original CF-378 modification and hydraulic pump motor wiring. The redesign includes a new routing and clamping configuration for the hydraulic lines associated with the ground test connections, as well as new routing and clamping for the pump motor wiring harness. Modification CD-154 will use stand-offs and clamps to secure the hydraulic hoses and wiring to nearby structure to eliminate the potential for chafing. Modification CD-154 passed critical design review on 02 Oct 2014, with a proof fit and prototype installation completed. Full implementation is scheduled for 2016.
A change to the hydraulic fluid used was also made (although it is unlikely to have made a difference in this accident scenario). The opportunity to introduce a further mitigation, by using more robust, chaffing resistant wiring was not unfortunately taken.
Director of Flight Safety Remarks
Aircraft fires can be particularly hazardous and survival may be dependent on the ability to rapidly egress from a burning aircraft. The survival of all six crew members and 15 passengers, following an in-flight explosion and fire on CH147202 in Afghanistan, was due in no small measure to the ability to land and egress from the aircraft within 30 seconds of the initial explosion. Similarly in this case, the quick decision of the crew to land immediately almost certainly prevented a catastrophic accident with loss of life. Definitive procedures available to cover all emergency situations do not exist, and for this reason checklists, CFTO’s and other publications clearly stipulate that above all else, good judgement and professionalism must prevail. Standard training protocol dictates that multi engine aircraft continue the take-off following a V1/rotate call. However, in this case the decision to abort was a textbook example of exerting superior judgement during an exceptional circumstance.
In retrospect, the physical cause of the fire was evident. However, what is not evident is why the hazardous condition was not detected earlier. There were ample opportunities to detect this hazardous condition on any affected aircraft during any scheduled inspection. Complacency likely came into play in that the area was looked at many times, without actually seeing, understanding and/or correcting the problem. The short term solution to counter these human factors is through proper technical training, but the long term solution that will pay the greatest dividends is to train the minds of our technicians to expect to find problems every time they do an inspection.
You may find these Aerossurance articles of interest:
- AC-130J Prototype Written-Off After Flight Test LOC-I Overstress
- UPDATE 23 April 2016: Dim, Negative Transfer Double Flameout, or this article that discusses the latent defect introduced with a night vision modification
- UPDATE 9 January 2017: S-92A Flying Control Restriction on Wiring Loom
- UPDATE 16 March 2018: USAF F-16C Crash: Engine Maintenance Error
- UPDATE 13 November 2018: Inadequate Maintenance, An Engine Failure and Mishandling: Crash of a USAF WC-130H
- UPDATE 18 December 2018: USAF Engine Shop in “Disarray” with a “Method of the Madness”: F-16CM Engine Fire
- UPDATE 27 December 2018: Inadequate Maintenance at a USAF Depot Featured in Fatal USMC KC-130T Accident
- UPDATE 26 January 2019: MC-12W Loss of Control Orbiting Over Afghanistan: Lessons in Training and Urgent Operational Requirements
- UPDATE 30 March 2019: Contaminated Oxygen on ‘Air Force One’ Poor standards at a Boeing maintenance facility resulted in contamination of two oxygen systems on a USAF Presidential VC-25 (B747).
- UPDATE 30 May 2019: PA-34 Electrical Short Melted Rudder Cable
- UPDATE 30 October 2019: ‘Crazy’ KC-10 Boom Loss: Informal Maintenance Shift Handovers and Skipped Tasks
Aerossurance is pleased to be supporting the annual Chartered Institute of Ergonomics & Human Factors’ (CIEHF) Human Factors in Aviation Safety Conference for the third year running. This year the conference takes place 13 to 14 November 2017 at the Hilton London Gatwick Airport, UK with the theme: How do we improve human performance in today’s aviation business?
Aerossurance is pleased to be both sponsoring and presenting at a Royal Aeronautical Society (RAeS) Human Factors Group: Engineering seminar Maintenance Error: Are we learning? to be held on 9 May 2019 at Cranfield University.