USAF Tool Trouble: “Near Catastrophic” $25mn E-8C FOD Fuel Tank Rupture
On 13 March 2009 USAF Northrop Grumman E-8C JSTARS 93-0597 (a heavily modified Boeing 707-300 surveillance aircraft) was in the process of refuelling from a KC-135T tanker according to a USAF Accident Investigation Board.
After uplifting around 30,000 lbs of fuel the crew heard a “loud bang throughout the midsection of the aircraft.” Refuelling was paused while the E-8C crew checked for damage. None was identified and fuelling recommenced but “another series of loud noises and vibrations” were “heard and felt throughout the aircraft.” Onboard the KC-135T the boom operator saw fuel streaming from the E-8C. Once alerted the E-8C crew also spotted fuel pouring from “at least two holes in the left wing, just inboard of the number two engine.” The E-8C diverted back to its base at Al Udeid Air Base in Qatar.
On inspection, it was found that $25 million worth of damage had occurred when the no 2 main fuel tank had been over pressured and ruptured is a way that was described as “near catastrophic”.
Investigators found that a maintenance sub-contractor, FFC, employed by Northrop Grumman, had left a fuel vent test plug in one of the fuel tank’s climb relief vents during Programmed Depot Maintenance (PDM) tank de-seal/re-seal maintenance at the Northrop Grumman Lake Charles Maintenance and Modification Center in Louisiana.
The sub-contractor had reportedly failed to follow the USAF Technical Order (TO) procedures when using the plug during base maintenance. “The PDM subcontractor employed ineffective tool control measures,” say investigators.
The procedures required tool checks at the start and end of each shift, but there was no evidence the tool was identified as missing. In fact, there were no records of any tools being taken into the fuel tanks, which was not credible considering the work that was completed. The plug was required to be fitted with a red ‘streamer’ but it was found without one too.
Due to the relatively short period of time between take-off and aerial refueling, E-8C did not have the opportunity to burn much fuel from the number two fuel tank which would have allowed a secondary dive valve, to open (as occurred during the aerial refuelling when the aircraft deployed to theatre) and provide an alternative means to vent pressure.
Other Safety Resources
During preflight inspection a fuel leak was discovered… Maintenance personnel were unable to locate or repair the fuel leak, so they inserted fuel tank vent plugs and pressurized the fuel tank The leak was found and repaired. Once the repair was completed, the fuel systems technician exited the fuel tank but failed to document or request an in-progress inspection, to ensure all tools and materials are removed from the fuel tank; and fuel tank plugs (if used) are extracted from the fuel tank vents. Thus a fuel vent plug was not removed.
As the aircraft was being [subsequently] refueled [on the ground], upon reaching approximately 120,000 lbs of jet fuel, the interior left wing fuel tank pressure exceeded wing structural tolerances, as pressure was unable to vent due to the forgotten fuel vent plug. Over-pressurization resulted in catastrophic failure of the left wing structure at the wing root.
We previously discussed another E-8 contractor error at Lake Charles: What a Difference a Hole Makes: E-8C JSTARS $7.35 million Radar Mishap
FOD in military converted Boeing aircraft remains topical: US Air Force forces Boeing to make changes after problems with KC-46 Pegasus production and Boeing Has ‘Severe Situation’ After Parts Left in Tankers, Says Top USAF Buyer
The company has also declared this Friday [15 March 2019] to be “FOD Amnesty Day,” according to an internal company email reviewed by Defense One. The email, which does not mention the KC-46 by name, cites “multiple foreign object debris escapes involving some of our most critical programs.”
“This includes the identification and removal of unnecessary tools, equipment, electronic files, documents and refuse,” said the email from Gena Lovett, vice president for manufacturing, safety and quality, and Ursula English, vice president of total quality. “We all have a responsibility to maintain safe, clean, efficient and FOD-free working areas and it’s so vitally important to who we are we’re making it a [Boeing Defense, Space and Security]-wide priority.”
“FOD is really about every person, everyone in the workforce, following those procedures and bringing a culture of discipline for safety,” he said. “This is an issue of safety and culture.”
And [Will] Roper [US Air Force acquisition chief] does not expect a cultural change immediately, even if Boeing starts delivering plans without debris.
“I’m going to believe it [a culture shift] when I see month after month for a long time, that yes, those practices are now things that aren’t just being done because they have to be done, they’re being done because the workforce says this is a product that we deliver to the Air Force, they’re counting on it being high quality and we own the quality of our product,” he said. “That is how we feel [about] our depots and our production lines and I expect to see that reflected in industry partners that are building critical systems for us.”
UPDATE 17 June 2019: KC-46 deliveries are still only factionally above 1 per month rather than the intended 3 per month because of the ongoing FOD problem, with debris found “as recently as last week” said the Assistant Secretary of the Air Force (Acquisition, Technology, & Logistics), Dr Will Roper, at the Paris Air Show. He went on:
“We expect all of them are going to have foreign object debris,” or FOD, he said.
…he expects improvements in the quality-assurance inspections that are supposed to occur at every station on the assembly line.
Boeing should reward an assembly line worker who shuts down the production line, Roper said.
We also have written:
- Rotor Blade Tool Control FOD Incident
- Professor James Reason’s 12 Principles of Error Management
- Back to the Future: Error Management
- Safety Performance Listening and Learning – AEROSPACE March 2017
- Maintenance Human Factors: The Next Generation
- Airworthiness Matters: Next Generation Maintenance Human Factors
- Lost Tool FODs Propeller Blade, Penetrating Turboprop’s Fuselage
- FOD and an AS350B3 Accident Landing on a Yacht in Bergen
- Micro FOD: Cessna 208B Grand Caravan Engine Failure & Forced Landing
- FOD Damages 737 Flying Controls
- Crew Bag FOD Shatters Hawk Canopy
- C-130J Control Restriction Accident, Jalalabad
- Flying Control FOD: Screwdriver Found in C208 Controls
- B1900D Emergency Landing: Maintenance Standards & Practices
- Robinson R44 Power Loss: Excessive Lubricant
- United Airways Suffers from ED (Error Dysfunction)
- USAF RC-135V Rivet Joint Oxygen Fire: A military accident investigation has paradoxically determined that a $62.4mn fire was due to a maintenance error but that no human factors were involved.
- Inadequate Maintenance, An Engine Failure and Mishandling: Crash of a USAF WC-130H
- Inadequate Maintenance at a USAF Depot Featured in Fatal USMC KC-130T Accident
- USAF F-16C Crash at Joint Base Andrews: Engine Maintenance Error
- USAF Engine Shop in “Disarray” with a “Method of the Madness”: F-16CM Engine Fire
- MC-12W Loss of Control Orbiting Over Afghanistan: Lessons in Training and Urgent Operational Requirements
- UPDATE 9 February 2019: Meeting Your Waterloo: Competence Assessment and Remembering the Lessons of Past Accidents: No one was injured in this low speed derailment in London after signal maintenance errors but investigators expressed concern that the lessons about maintenance errors from the fatal triple collision at Clapham in 1988 may have been forgotten.
- UPDATE 30 March 2019: Contaminated Oxygen on ‘Air Force One’ Poor standards at a Boeing maintenance facility resulted in contamination of two oxygen systems on a USAF Presidential VC-25 (B747).
- UPDATE 30 October 2019: ‘Crazy’ KC-10 Boom Loss: Informal Maintenance Shift Handovers and Skipped Tasks
UPDATE 31 May 2019: The Portuguese accident investigation agency, GPIAAF, issued a safety investigation update on a serious in-flight loss of control incident involving Air Astana Embraer ERJ-190 P4-KCJ that occurred on 11 November 2018. The aircraft was landed safely after considerable difficulty, so much so the crew had debated ditching offshore. GPIAAF conformed that incorrect ailerons control cable system installation had occurred in both wings during a maintenance check conducted in Portugal.
GPIAFF note that: “By introducing the modification iaw Service Bulletin 190-57-0038 during the maintenance activities, there was no longer the cable routing and separation around rib 21, making it harder to understand the maintenance instructions, with recognized opportunities for improvement in the maintenance actions interpretation”. They also comment that: “The message “FLT CTRL NO DISPATCH” was generated during the maintenance activities, which in turn originated additional troubleshooting activities by the maintenance service provider, supported by the aircraft manufacturer. These activities, which lasted for 11 days, did not identify the ailerons’ cables reversal, nor was this correlated to the “FLT CTRL NO DISPATCH” message.”
GPIAFF comment “deviations to the internal procedures” occurred within the maintenance organisation that “led to the error not being detected in the various safety barriers designed” in the process. They also note that the error ” was not identified in the aircraft operational checks (flight controls check) by the operator’s crew.”
UPDATE 1 June 2019: Our analysis: ERJ-190 Flying Control Rigging Error
Maintenance Observation Program (MOP)
Aerossurance worked with the Flight Safety Foundation (FSF) to create a Maintenance Observation Program (MOP) requirement for their contractible BARSOHO offshore helicopter Safety Performance Requirements to help learning about routine maintenance and then to initiate safety improvements:
Aerossurance can provide practice guidance and specialist support to successfully implement a MOP.
As one article, Retired Air Force Four-Star: How to Solve Aviation Crisis has discussed:
…the best solution is for senior officials to keep the door open and have ongoing conversations on potential problems. “It’s not as much oversight as much as it is … going out and talking with [individuals] … just seeing what’s going on…”
Aerossurance is pleased to be both sponsoring and presenting at a Royal Aeronautical Society (RAeS) Human Factors Group: Engineering seminar Maintenance Error: Are we learning? to be held on 9 May 2019 at Cranfield University.
E-8 JSTARS Background
The E-8C Joint Surveillance Target Attack Radar System, or Joint STARS, is an airborne battle management, command and control, intelligence, surveillance and reconnaissance platform. It evolved from US Army and Air Force programmes to develop, detect, locate and attack enemy armor at ranges beyond the forward area of troops. The first two developmental aircraft deployed in 1991 to Operation Desert Storm in the Middle East and then supported Operation Joint Endeavor in the Balkansn December 1995. The APY-7 antenna can develop a 120-degree field of view covering nearly 19,305 square miles (50,000 square km) and is capable of detecting targets at more than 123 nm (250 km). The aircraft are operated by th 116th Air Control Wing at Robins Air Force Base, Georgia.