Contaminated Oxygen on ‘Air Force One’
In April 2016 Boeing maintenance technicians contaminated the oxygen system of 92-9000, a USAF Boeing VC-25A, one of two converted B747-200Bs flown by the 89th Airlift Wing at Joint Base Andrews, Maryland as Presidential transports (using the call sign Air Force One, only when the President is on board). The damage, done at a Boeing facility at Port San Antonio, Kelly Field, Texas, was spotted before any Presidential flying. It cost $4 million to rectify, making this a Class A mishap in the US DoD classification system.
USAF Boeing VC-25A 92-9000 (Credit: USAF, Senior Airman Elizabeth Slate)
The Incident
To minimise the risk of fire the VC-25A Aircraft Maintenance Manual (AMM) requires all tools and components used on the oxygen systems to be ‘oxygen clean’. As the UK CAA note:
All oxygen system equipment must be kept absolutely free from traces of oil, grease or flux, as these substances will ignite spontaneously in contact with pressurised oxygen. When a form of lubricant is necessary (e.g. because of a binding thread) the approved or recommended lubricant must be used. Lubricant should be used sparingly to ensure that it does not enter the oxygen system.
Investigators say three Boeing technicians had used a contaminated regulator, tools and other components during leak checks conducted as part of a scheduled inspection 1-10 April 2016.
One technician gave another one items they initially claimed were ‘oxygen-clean’. But the second technician subsequently became aware this was incorrect. When challenged. the first technician allegedly said:
Here [is] some cleaning fluid. Do with it what you want. Use it if you want, but I don’t know anything about it.
The fluid could have been used successfully to decontaminate the items, but investigators found none of the three technicians involved in the leak check were actually trained in the appropriate cleaning technique. The second and third technicians attempted to clean the items but subsequently both the passenger and medical oxygen system became contaminated.
USAF Boeing VC-25A 92-9000 (Credit: USAF, Jeremy Mashek)
The Analysis
Fatigue and pressure were factors that potentially contributed. Investigators found that maintenance at the Boeing depot had fallen behind schedule and in December 2015 workers had been put on mandatory 12-hour shifts beginning. The three personnel involved has been working six or seven days each week. This strongly suggests they will have been fatigued. We have discussed fatigue previously, looking at a number of case studies: Maintenance Personnel Fatigue and Alertness
The investigators also noted that when Boeing has previously relocated maintenance to Texas from Wichita, Kansas, 172 personnel at the new facility would not meet a USAF 5 years experience requirements. Boeing successfully applied for waivers based on their assessment this was low risk.
The technicians had under gone training in oxygen safety, in one case just 3 weeks earlier. Investigators comment that they “either did not absorb or retain the information”.
The problem on the VC-25A was identified when a non-oxygen clean regulator was found connected and testing confirmed the systems were contaminated.
Boeing paid for the rectification of the damage.
The San Antonio site is now starting work converting 747-8 N894BA as the first VC-25B to replace the now dated VC-25As. The aircraft was ferried there on 27 March 2019.
Boeing quality, albeit in production, remains topical: US Air Force forces Boeing to make changes after problems with KC-46 Pegasus production and Boeing Has ‘Severe Situation’ After Parts Left in Tankers, Says Top USAF Buyer
The company has also declared this Friday [15 March 2019] to be “FOD Amnesty Day,” according to an internal company email reviewed by Defense One. The email, which does not mention the KC-46 by name, cites “multiple foreign object debris escapes involving some of our most critical programs.”
“This includes the identification and removal of unnecessary tools, equipment, electronic files, documents and refuse,” said the email from Gena Lovett, vice president for manufacturing, safety and quality, and Ursula English, vice president of total quality. “We all have a responsibility to maintain safe, clean, efficient and FOD-free working areas and it’s so vitally important to who we are we’re making it a [Boeing Defense, Space and Security]-wide priority.”
“FOD is really about every person, everyone in the workforce, following those procedures and bringing a culture of discipline for safety,” he said. “This is an issue of safety and culture.”
And [Will] Roper [US Air Force acquisition chief] does not expect a cultural change immediately, even if Boeing starts delivering plans without debris.
“I’m going to believe it [a culture shift] when I see month after month for a long time, that yes, those practices are now things that aren’t just being done because they have to be done, they’re being done because the workforce says this is a product that we deliver to the Air Force, they’re counting on it being high quality and we own the quality of our product,” he said. “That is how we feel [about] our depots and our production lines and I expect to see that reflected in industry partners that are building critical systems for us.”
UPDATE 2 April 2019: The USAF has paused accepting KC-46s for a second time after more debris and other quality issues were found.
UPDATE 17 June 2019: KC-46 deliveries are still only factionally above 1 per month rather than the intended 3 per month because of the ongoing FOD problem, with debris found “as recently as last week” said the Assistant Secretary of the Air Force (Acquisition, Technology, & Logistics), Dr Will Roper, at the Paris Air Show. He went on:
“We expect all of them are going to have foreign object debris,” or FOD, he said.
…he expects improvements in the quality-assurance inspections that are supposed to occur at every station on the assembly line.
Boeing should reward an assembly line worker who shuts down the production line, Roper said.
Safety Resources
We have previously written:
- Professor James Reason’s 12 Principles of Error Management
- Back to the Future: Error Management
- Safety Performance Listening and Learning – AEROSPACE March 2017
- Maintenance Human Factors: The Next Generation
- Airworthiness Matters: Next Generation Maintenance Human Factors
- Flying Control FOD: Screwdriver Found in C208 Controls
- B1900D Emergency Landing: Maintenance Standards & Practices
- United Airways Suffers from ED (Error Dysfunction)
- USAF RC-135V Rivet Joint Oxygen Fire: A military accident investigation has paradoxically determined that a $62.4mn fire was due to a maintenance error but that no human factors were involved.
- Inadequate Maintenance, An Engine Failure and Mishandling: Crash of a USAF WC-130H
- Inadequate Maintenance at a USAF Depot Featured in Fatal USMC KC-130T Accident
- USAF F-16C Crash at Joint Base Andrews: Engine Maintenance Error
- USAF Engine Shop in “Disarray” with a “Method of the Madness”: F-16CM Engine Fire
- What a Difference a Hole Makes: E-8C JSTARS $7.35 million Radar Mishap
- USAF Tool Trouble: “Near Catastrophic” $25mn E-8C FOD Fuel Tank Rupture
- Meeting Your Waterloo: Competence Assessment and Remembering the Lessons of Past Accidents: No one was injured in this low speed derailment in London after signal maintenance errors but investigators expressed concern that the lessons about maintenance errors from the fatal triple collision at Clapham in 1988 may have been forgotten.
- UPDATE 1 April 2019: Tree Top Autorotation for B206L1 After Loose Fuel Line B-Nut Leaks
- UPDATE 30 October 2019: ‘Crazy’ KC-10 Boom Loss: Informal Maintenance Shift Handovers and Skipped Tasks
- UPDATE 6 December 2019: Dash 8 Q400 Return to Base After Pitot System Contaminated By Unapproved Test Kit Lubricant
- UPDATE 30 June 2020: 17 Year Old FOD and a TA-4K Ejection
UPDATE 31 May 2019: The Portuguese accident investigation agency, GPIAAF, issued a safety investigation update on a serious in-flight loss of control incident involving Air Astana Embraer ERJ-190 P4-KCJ that occurred on 11 November 2018. The aircraft was landed safely after considerable difficulty, so much so the crew had debated ditching offshore. GPIAAF conformed that incorrect ailerons control cable system installation had occurred in both wings during a maintenance check conducted in Portugal.
Misrigged Flying Control Cable, Air Astana Embraer EJ-190 P4-KCJ (Credit: GPIAFF)
GPIAFF note that: “By introducing the modification iaw Service Bulletin 190-57-0038 during the maintenance activities, there was no longer the cable routing and separation around rib 21, making it harder to understand the maintenance instructions, with recognized opportunities for improvement in the maintenance actions interpretation”. They also comment that: “The message “FLT CTRL NO DISPATCH” was generated during the maintenance activities, which in turn originated additional troubleshooting activities by the maintenance service provider, supported by the aircraft manufacturer. These activities, which lasted for 11 days, did not identify the ailerons’ cables reversal, nor was this correlated to the “FLT CTRL NO DISPATCH” message.”
GPIAFF comment “deviations to the internal procedures” occurred within the maintenance organisation that “led to the error not being detected in the various safety barriers designed” in the process. They also note that the error ” was not identified in the aircraft operational checks (flight controls check) by the operator’s crew.”
UPDATE 1 June 2019: Our analysis: ERJ-190 Flying Control Rigging Error
Aerossurance worked with the Flight Safety Foundation (FSF) to create a Maintenance Observation Program (MOP) requirement for their contractible BARSOHO offshore helicopter Safety Performance Requirements to help learning about routine maintenance and then to initiate safety improvements:
Aerossurance can provide practice guidance and specialist support to successfully implement a MOP.
Aerossurance is pleased to be both sponsoring and presenting at a Royal Aeronautical Society (RAeS) Human Factors Group: Engineering seminar Maintenance Error: Are we learning? to be held on 9 May 2019 at Cranfield University.
Recent Comments