NTSB Probable Causes and Bad Apples (and the Bad Apple Index)
During an on-line discussion about an Aerossurance post to mark the 20th anniversary of a CFIT accident in Guam, there was criticism that the NTSB ‘Probable Cause‘ was:
The captain’s failure to adequately brief and execute the nonprecision approach and the first officer’s and flight engineer’s failure to effectively monitor and cross-check the captain’s execution of the approach.
Some contributors rightly felt this simplistically blamed the crew (i.e. taking a ‘bad apple’ ‘old view’ of human error) rather than explained the accident. Others pointed out the accident was 20 years ago and investigators take a more sophisticated approach today. So we decided to look at the full NTSB air accident reports issued since the start of 2016:
- AAR1601: The Probable Cause was: “the pilot’s conduct…” (see Aircraft accident Embraer EMB-500 Phenom 100 N100EQ)
- AAR1602: The Probable Cause was: “the captain’s inability” (see Delta MD-88 Accident at La Guardia 5 March 2015)
- AAR1603: The Probable Cause was: “the flight crew’s mismanagement…” (see Execuflight Hawker 700 N237WR Akron Accident: Casual Compliance)
- AAR1701‘s Probable Cause, after a lot of debate in the public hearing, was “(1) preflight hydraulic check, which depleted hydraulic pressure… and (2) lack of salient alerting…” but the pilot’s actions were determined to be contributory (see Crashworthiness and a Fiery Frisco US HEMS Accident)
- AAR1702: The Probable Cause was: “(1) the pilot’s decision to…and (2) the company culture…” (see All Aboard CFIT: Alaskan Sightseeing Fatal Flight)
- AAR1703: The Probable Cause was: “the pilot’s pattern of poor decision-making…” (see Balloon accident Kubicek BB85Z N2469L)
- AAR1801: The Probable Cause was: “the failure of the high-pressure turbine (HPT) stage 2 disk…” (see Uncontained CF6-80 Failure: American B767-300 28 Oct 2016)
- AAR1802: The Probable Cause was: “the flight crew’s decision to continue the VFR flight into deteriorating visibility and their failure to perform an immediate escape maneuver after entry into IMC…” (see Operator & FAA Shortcomings in Alaskan Accident)
That’s a 5.5 out of 8 bad apple index (69%) as far as we are concerned! In the NTSB Board debate over the one case (AAR1701) where the Probable Cause was not crew actions (or inactions), Board member Robert Sumwalt, who has recently been sworn in as NTSB Chairman, objected to the original draft Probable Cause saying:
This pilot did not do what he was supposed to do. But if we dig deeper — and that’s what really bothers me about the original probable cause is that it is saying that the pilot screwed up… To put the pilot as the primary factor of the probable cause is just wrong, because it points to the last person who made the last mistake. I think we need to dig deeper to get to the root cause of this.
So while the concept of Probable Cause is enshrined in US law, perhaps we will see a change of interpretation in future. Certainly the Contributory Factors identified by the NTSB are often better indications of what caused an accident than the official Probable Causes.
Note: This is an update of an article originally published on LinkedIn by Aerossurance’s Andy Evans.
Tracking the Bad Apple Index
UPDATE 7 March 2019: AAR1901: The Probable Cause was: “the jammed condition of the airplane’s right elevator…” However “Contributing to the survivability of the accident was the captain’s timely and appropriate decision to reject the takeoff, the check airman’s disciplined adherence to standard operating procedures….”.
If we score this as -1 ‘bad apples’, because of the mention of positive crew performance, this takes our bad apple index to 4.5 out of 9 (50%). UPDATE 19 June 2019: Discussed by NTSB further here: Why we care when things go right
UPDATE 30 March 2019: AAR1902: The Probable Cause was: “the pilot-in-command’s attempt to salvage an unstabilized visual approach, which resulted in an aerodynamic stall at low altitude”.
This takes the bad apple index up to 5.5 out of 10 (55%).
UPDATE 12 December 2019: AAR1903: The Probable Cause was: “a low-cycle fatigue crack in the dovetail of fan blade No. 13…”.
Not surprisingly this takes the bad apple index to 5.5 out of 11 (50%).
UPDATE 24 January 2020: AAR1904 The Probable Cause was: “Liberty Helicopters Inc.’s use of a NYONair-provided passenger harness/tether system, which caught on and activated the floor-mounted engine fuel shutoff lever…”.
This takes the bad apple index to 5.5 out of 12 (46%).
UPDATE 6 June 2020: AAR2001 The Probable Cause was: “Survival Flight’s inadequate management of safety, which normalized pilots’ and operations control specialists’ noncompliance with risk analysis procedures and resulted in the initiation of the flight without a comprehensive preflight weather evaluation, leading to the pilot’s inadvertent encounter with instrument meteorological conditions, failure to maintain altitude, and subsequent collision with terrain.
This takes the bad apple index down to 5.5 out of 13 (42%).
UPDATE 5 August 2020: AAR2002 The Probable Cause was: “…the inappropriate response by the first officer... Contributing to the accident was the captain’s failure to adequately monitor the airplane’s flightpath and assume positive control of the airplane to effectively intervene. Also contributing were systemic deficiencies in the aviation industry’s selection and performance measurement practice…. Also contributing to the accident was the Federal Aviation Administration’s failure to implement the pilot records database in a sufficiently robust and timely manner.
This takes the bad apple index up to 6.5 out of 14 (46%).
UPDATE 9 February 2021: AAR2101 The Probable Cause was: “…the pilot’s decision to continue flight under visual flight rules into instrument meteorological conditions, which resulted in the pilot’s spatial disorientation and loss of control. Contributing to the accident was the pilot’s likely self-induced pressure and the pilot’s plan continuation bias, which adversely affected his decision-making, and Island Express Helicopters Inc.’s inadequate review and oversight of its safety management processes.
This takes the bad apple index up to 7.5 out of 15 (50%).
UPDATE 16 March 2021: AAR2102 Probable Cause was: “…the pilot’s aggressive takeoff maneuver…”
This takes the bad apple index up to 8.5 out of 16 (53%).
UPDATE 15 May 2021: AAR2104 Probable Cause was: “…the inherent limitations of the see-and-avoid concept, which prevented the two pilots from seeing the other airplane before the collision, and the absence of visual and aural alerts from both airplanes’ traffic display systems…”.
This takes the bad apple index down to 8.5 out of 17 (50%). Note AAR2103 was a study not an individual accident report.
Background: Safety Resources
James Reason, Professor Emeritus, University of Manchester wrote in his classic 1997 classic Managing the Risks of Organizational Accidents that among some of the problems with managing human errors are:
- Focusing on the active failures (of people) not the latent conditions
- Not being informed by current human factors thinking regarding error and accident causation
Human error is not itself a cause, but merely the start of a human factors investigation, as explained by Sidney Dekker in The Field Guide to Understanding Human Error – A Review (discussed in our 2006 book review The Field Guide to Understanding Human Error presented to the RAeS).
ICAO Doc 9756 Manual of Aircraft Accident and Incident Investigation Part IV Reporting gives the following definitions:
Causes are those events which alone, or in combination with others, resulted in injuries or damage. Causes are defined as actions, omissions, events, conditions, or a combination thereof, which led to the accident or
incident.Contributing factors are actions, omissions, events, conditions, or a combination thereof, which, if eliminated, avoided or absent, would have reduced the probability of the accident or incident occurring, or mitigated the severity of the consequences of the accident or incident.
It also states:
The determination of causes and/or contributing factors should be based on a thorough, impartial and objective analysis of all the available evidence. Any condition, act or circumstance that was a causal factor in the
accident should be clearly identified. Seen together, the causes should present a picture of all the reasons why the accident occurred. The list of causes should include both the immediate causes and the deeper or systemic causes.An investigator should not write from the perspective of a regulator who is concerned about non-compliance with rules and regulations or from the perspective of a manager in a company, where determination of support for disciplinary or legal action may be an objective.
Even though the purpose of ICAO Annex 13 accident investigation reports is not to assign blame or liability:
The causes/contributing factors should be formulated in a way which, as much as practicable, minimizes the implication of blame or liability.
The Manual gives these examples of formulating causes and contributor factors, with the right had column being preferable:
Aerossurance sponsored the 9th European Society of Air Safety Investigators (ESASI) Regional Seminar in Riga, Latvia 23 and 24 May 2018.
Aerossurance is pleased to be both sponsoring and presenting at a Royal Aeronautical Society (RAeS) Human Factors Group: Engineering seminar Maintenance Error: Are we learning? to be held on 9 May 2019 at Cranfield University.
UPDATE 8 April 2020: NTSB Confirms United Airlines Maintenance Error After 12 Years
Recent Comments