ATR 72 Rudder Travel Limitation Unit Incident: Latent Potential for Misassembly Meets Commercial Pressure (Air Contractors, EI-SLG)
On 15 March 2011 ATR 72-202 EI-SLG of Air Contractors experienced an uncommanded yaw, resulting in a roll to the left as it accelerated through 185 kt. Directional control was regained and aircraft returned to Edinburgh Airport. Cockpit indications identified a fault with the rudder Travel Limitation Unit (TLU). The aircraft was makings its first flight following a ‘2 Year’ base maintenance check at Edinburgh and was being positioned to Paris. In this article we examine the airworthiness aspects.
System Design and Organisational Background
The rudder linkage on the ATR 72 is a mechanical system… The TLU, installed on the rudder rear quadrant shaft, reduces the range of available rudder deflection at airspeeds above 185 kt, in order to limit the structural loads on the rudder.
In the full authority (or low speed) position, rudder deflection is not limited; in the reduced authority (or high speed) position, rudder deflection is mechanically limited by the TLU. The TLU mechanism comprises an electrical actuator which drives a pivoting bracket on which two rollers are mounted.
In the reduced authority position the actuator retracts, engaging the rollers in two v‑shaped cams mounted on the rudder rear quadrant shaft, to limit the rudder deflection mechanically. In the full authority position the actuator extends, disengaging the rollers from the cams and rudder deflection is no longer limited.
Within the ATR family the TLU is unique to the ATR72.
The maintenance organisation that performed the check…
…had previously been owned by the aircraft operator but both were now sister companies and part of a larger group. Two of the operator’s aircraft had recently experienced significant delays at the Edinburgh facility…. Another of the operator’s aircraft was planned in for maintenance immediately following EI-SLG.
The management at the maintenance organisation considered that another delayed aircraft would have been viewed as a major failure on their part and would result in loss of revenue if the following aircraft could not be accommodated.
Of note however is that:
This operator had a policy of directly purchasing parts from the aircraft manufacturer, and forwarding them to the maintenance organisation. The maintenance organisation considered that this practice would often result in delays, causing a backlog of work towards the end of the maintenance check.
A check leader was assigned… This role involved allocation of job cards and manpower, ordering of spares and reporting on the progress of the check. A number of mechanics and LAEs were assigned to each aircraft, and the senior LAE would deputise for the check leader in his absence.
The maintenance organisation had a production manager.
The production manager was an experienced [Part-66 C Licenced] engineer who had worked for the organisation for two years. In this time he had been promoted to the role of check leader and was subsequently appointed as production manager, responsible for the overall management of the maintenance facility.
This post also entailed acting as the Accountable Manager for the company’s Part 145 maintenance organisation approval. In addition to this he also held the post of line maintenance manager.
As well as the potential for high workload and distractions from these multiple roles, AAIB note that:
This situation represented a conflict of interest between the production manager’s commercial priorities and his obligations as the Accountable Manager.
AAIB Safety Investigation
…subsequent visual inspection of the TLU confirmed that one of the cams on the rudder rear quadrant shaft had been incorrectly installed, such that it was misaligned with the other cam.
Testing demonstrated that if a cam was removed and transposed through 180° (such that the inboard face of the cam then faced outboard) it could be installed without encountering any resistance, resulting in misalignment between the two cams.
This is because the master spline is not located centrally between the two lobes on each cam, but is offset to one side.
Readers looking at the photographs above might conclude that any misalignment would be easily detected visually. However, the side-on view an engineer would actually have is shown below.
The cam lobes are not visible from this perspective, and although evident, the misalignment between the two cams is more difficult to detect.
If the right cam is misorientated…
…it was demonstrated that when the TLU was actuated towards the reduced authority position, both rollers were prevented from engaging in the cams. Instead, the right roller was observed to push upwards on the upper lobe of the right cam, causing the rudder rear quadrant shaft to rotate, deflecting the rudder surface and pedals.
The maximum rudder deflection produced during testing on the ground (in the absence of flight loads) was 21°. A condition could be produced where the right roller was pushing up against the upper lobe of the right cam and the left roller was pushing down against the lower lobe of the left cam, effectively creating a condition where the rudder surface and pedals were jammed in the deflected position.
The AAIB also identified that the AMM functional test would only correctly identify misalignment if the test button was pressed for at least 25 seconds, yet the AMM did not specify a time. The AAIB report that the aircraft had been under maintenance between 19 February and 15 March 2011:
During the check it was determined the TLU actuator had to be replaced due to excessive moisture indications in the actuator desiccant cartridge. This was completed on 7 March 2011 and, as required in the Aircraft Maintenance Manual (AMM) task, an operational test of the TLU was performed.
This work was certified by a [Part-66 B1] licensed aircraft engineer (LAE), who will be referred to as ‘Cert A’. Cert A had worked for the organisation for 3½ years, initially as a technician before undertaking his licence exams. He was awarded a ‘B1’ category licence in November 2009 and an ATR 72 type rating and company approvals in May 2010. [H]e was considered within the organisation to be a very capable engineer, frequently assisting the check leader and often deputising in this role.
As the actuator replacement involved disturbance of a flight control system, independent inspections of the installation and the operational test were required; this was carried out on 9 March 2011. The first part of the independent inspection was performed by Cert A, and the second by the check leader.
While conducting a final check of the area for any loose items following completion of the independent inspections, Cert A observed that the rudder cables seemed very tight and he raised a Non Routine Job Card (NRC) for the cable tensions to be checked. He also noted some play in the bearing of the TLU support arm.
The defect was discovered at a late stage in the check.
Cert A discussed his findings with the check leader who referred him to the production manager. The production manager advised Cert A to remove the TLU support arm for closer examination. The production manager stated after the incident that he should not have become involved in the decision about the repair to the TLU support arm and should have instead referred Cert A to technical services [note likely to be a reference to the Operator’s CAMO].
Cert A referred to AMM job card 27-23-30 RAI 10000-001 ‘Removal and Installation of TLU Mechanism Assembly’ which involved relaxing the rudder control cables, removing the TLU actuator and partially disassembling the rudder rear quadrant shaft. It was the first time he had performed this task.
Upon examination of the TLU support arm, Cert A and the production manager concluded that the bearing housing was showing signs of wear which was allowing the bearing to migrate. However they considered the wear to be minor and unlikely to prevent the bearing from functioning correctly.
At this point the maintenance organisations recent performance and their management’s expectations on how a delay would be viewed started to influence decision-making:
As the aircraft was due to leave the hangar in four days, the production manager considered that there would be little chance of obtaining a replacement part without delaying the aircraft, but he did not perform a stock check to verify this. Instead, he advised Cert A to reinstall the TLU support arm and suggested the fitting should be ‘peened’ to prevent further movement of the bearing.
Both considered that peening (using a centre-punch to create a small lip at a number of locations around the edge of the bearing housing) was a standard engineering practice to retain loose bearings. They did not consult the ATR 72 Structural Repair Manual (SRM) or AMM to determine if this practice was an approved repair on the TLU support arm. Neither document contains reference to such a repair. Cert A asked the production manager if an NRC should be raised to document the defect and the subsequent rectification, but the production manager decided to proceed without raising the appropriate repair documentation.
The AAIB explain that the production manager stated that the prior delays of other aircraft…
…directly influenced his decision not to delay the EI-SLG check by ordering a replacement TLU support arm and not to record the work carried out on this system.
If an NRC had been raised to document the defect with the TLU support arm, the requirements for an operational test and an independent inspection of the TLU would have been raised. The decision not to record this maintenance resulted in these protections being removed and [ultimately] the maintenance error remaining undetected.
The next day Cert A reassembled the TLU mechanism. Notably…
He was deputising for the check leader and was the only certifying engineer working on EI-SLG that day. With the aircraft due to leave the hangar in three days, he was interrupted from the reassembly task numerous times to perform check leader functions.
The shift patterns for the engineers were 4 days on followed by 4 days off, working 12 hours per day from 0700 to 1900 hrs.
Yet on the day of installation AAIB report that Cert A was working his fifth consecutive 12-hour day.
He did not consider that he felt physically tired. However, he stated that he may have been mentally fatigued as a result of the heavy workload, the time pressure towards the end of the check and the additional stress of deputising for the check leader.
AMM job card 27-23-30 RAI 10000-001 ‘Removal and Installation of the TLU Mechanism Assembly’ did not include any specific instructions regarding the orientation of the cams or include any warnings about the possibility of incorrect installation.
These circumstances, coupled with a design that could be assembled incorrectly, increased the risk of an error. The AAIB note that:
The associated distractions, time pressure and the possibility of fatigue are likely to have been detrimental to his focus on the task of reassembling the TLU.
The AMM did specifically state that only the right-hand cam should be removed, though did not highlight the potential for an assembly error.
Having initially installed the TLU support arm, spacers and the left cam on the rudder rear quadrant shaft and checked that both cams were correctly aligned, Cert A was unsure of the order in which two of the spacers should be fitted, as this was not very clear in the AMM diagram.
He removed the spacers and laid them out to compare them with the AMM diagram. The right cam also came off and he inadvertently placed it back on the shaft in the incorrect orientation.
Cert A reported that the cam slid easily onto the shaft and he was confident that it was correctly aligned because he believed that the master locating spline on the rudder rear quadrant shaft was specifically intended to prevent misalignment of the cams. Once satisfied with the order in which the components had to be fitted, he completed the reassembly of the TLU mechanism up to the point where the next step was the rigging and tensioning of the rudder cables, for which he had previously raised a separate NRC.
Following this step, the AMM task also required a functional test of the rudder control, an operational test of the rudder control and spring tab and an operational test of the TLU to be performed. As no job card had been raised for the repair, Cert A made a mental note to perform an operational test of the TLU at a later stage but he omitted to do this.
Without a specific NRC…
None of the required functional checks on the TLU were performed. Cert A performed check leader functions for the remainder of the day.
[Another] NRC raised [previously] for rigging and tensioning the rudder cables was completed and signed off on 13 March 2011 by the opposite shift…
The incident was caused by the incorrect fitment of a cam on the rudder TLU mechanism which was not detected by maintenance personnel. This resulted in rudder control restriction which caused the aircraft to enter an uncommanded roll to the left when the airspeed increased above 185 kt.
The required independent inspection of the work and the operational test of the TLU system were not carried out.
Commercial pressure was identified as the most significant factor which influenced the decision to perform unapproved and unrecorded maintenance on the TLU system.
A contributory factor was the design of the TLU cams, which allowed them to be installed in the incorrect orientation.
The maintenance organisation, perhaps inevitably, appointed a new Accountable Manager and both engineers received additional training in ‘Aviation Legislation’ and ‘Human Factors’.
As a result of the issues identified with the lack of adequate staffing cover of certifying engineers, the maintenance organisation has implemented a change in shift patterns. The hangar now operates a Monday to Friday operation, with two shifts: 0700 to 1600 hrs and 1000 hrs to 1900 hrs, in order to ensure there are more certifying engineers available during peak hours.
In addition, a minimum of two certifying engineers are assigned to each aircraft in addition to the check leader. Where a certifying engineer deputises for the check leader, handovers are completed in writing and the production manager will step into the role of check leader if the number of certifying engineers on the check is compromised.
On 15 April 2011, in Special Bulletin S1/2011, AAIB promptly made the following recommendations:
- Safety Recommendation 2011-10: It is recommended that ATR immediately informs all operators of ATR aircraft equipped with a Travel Limitation Unit that it is possible to install the cams on the rear rudder quadrant shaft in the incorrect orientation.
- Safety Recommendation 2011-11: It is recommended that ATR amends all relevant Aircraft Maintenance Manual tasks to include a warning to highlight that the cams on the rear rudder quadrant shaft can be installed incorrectly.
- Safety Recommendation 2011-12: It is recommended that ATR amends the Aircraft Maintenance Manual task ‘Operational Test of the Rudder Travel Limitation Unit’ to state that: (1) the test should be carried out for a minimum of 30 seconds and (2) should an asymmetric restriction of the rudder pedals be detected or if the FLT CTL light illuminates, further inspection of the TLU system should be conducted.
The AAIB report that:
ATR issued an All Operators Message (AOM) on 19 April 2011, to advise operators of this incident and to emphasise the importance of performing an independent inspection after any maintenance is performed on a flight control system. ATR have also updated the AMM task ‘Removal and Installation of TLU Mechanism Assy’ to include a requirement to record the position of the right hand cam before removal. In addition, the AMM task ‘Operational Test of the Rudder Travel Limiter Unit’ has also been amended to reflect the intent of Safety Recommendation 2011-12.
However, as the potential for misorientation of the cams remained, in their final report, AAIB also recommended:
- Safety Recommendation 2012-002: It is recommended that the European Aviation Safety Agency require ATR to modify the cams on the rudder Travel Limitation Unit on all applicable aircraft, to reduce the risk of incorrect assembly.
This was rejected on the basis the design was reportedly compliant with the requirements it was certified to (JAR25.671(b), Change 11) and an assessment based on past frequency and consequence did not justify action.
So this remains a latent failure mode to this day. A classic example of a poor design that is not error-proof (see Rockets Sleds, Steamships and Human Factors: Murphy’s Law or Holt’s Law?).
There is almost no human action or decision that cannot be made to look flawed and less sensible in the misleading light of hindsight. It is essential that the critic should keep himself constantly aware of that fact.
You might find these safety / human factors resources of use:
- James Reason’s 12 Principles of Error Management
- Back to the Future: Error Management
- This 2006 review of the book Resilience Engineering by Hollnagel, Woods and Leveson, presented to the RAeS by Aerossurance’s Andy Evans: Resilience Engineering – A Review and this book review of Dekker’s The Field Guide to Understanding Human Error: The Field Guide to Understanding Human Error – A Review.
- Airworthiness Matters: Next Generation Maintenance Human Factors Over the last 10-15 years, much attention has been focused on maintenance human factors training and reporting & investigating errors. While we could concentrate on simply doing more of these and certainly can find ways to do these things better, perhaps the next generation approach needs to include a much wider range of activities.
- Aircraft Maintenance: Going for Gold? Should we start treating maintenance personnel more like athletes who need to achieve peak performance every day?
- The Power of Safety Leadership
- Leadership and Trust
- Safety Performance Listening and Learning – AEROSPACE March 2017
- How To Develop Your Organisation’s Safety Culture
There are also these other past Aerossurance articles:
- BEA Point to Inadequate Maintenance Data and Possible Non-Conforming Fasteners in ATR 42 Door Loss
- B1900D Emergency Landing: Maintenance Standards & Practices
- When Down Is Up: 747 Actuator Installation Incident
- Flybe Fume Event (Part 1): Compressor Wash Maintenance Human Factors Case Study
- ERJ-190 Flying Control Rigging Error
- Luftwaffe VVIP Global 5000 Written Off After Flying Control Assembly Error
- C-130 Fireball Due to Modification Error
- Fatal $16 Million Maintenance Errors
- Engine Shutdown Results in Revised SOV Rigging Instructions
- SAR AS365N3 Flying Control Disconnect: BFU Investigation
- ‘Crazy’ KC-10 Boom Loss: Informal Maintenance Shift Handovers and Skipped Tasks
- Maintenance Issues in Fire-Fighting S-61A Accident
- United Airways Suffers from ED (Error Dysfunction)
- NTSB Confirms United Airlines Maintenance Error After 12 Years
- USAF RC-135V Rivet Joint Oxygen Fire
- ATR72 VH-FVR Missed Damage: Maintenance Lessons
- The Missing Igniters: Fatigue & Management of Change Shortcomings
- A319 Double Cowling Loss and Fire – AAIB Report
- B747 Landing Gear Failure Due to Omission of Rig Pin During Maintenance
- Maintenance Human Factors in Finnish F406 Landing Gear Collapse
- Hoist Assembly Errors: SAR Personnel Dropped Into Sea
- Crossed Cables: Colgan Air B1900D N240CJ Maintenance Error
- A Lufthansa MD-11F Nose Wheel Detached after Maintenance Error
- Inadequate Maintenance, An Engine Failure and Mishandling: Crash of a USAF WC-130H
Flight Safety Foundation BARS Maintenance Observation Programme (MOP)
Aerossurance worked with the Flight Safety Foundation (FSF) to create a Maintenance Observation Program (MOP) requirement for their contractible BARSOHO offshore helicopter Safety Performance Requirements to help learning about routine maintenance and then to initiate safety improvements:
Aerossurance and its partners can provide proven, practical expertise to successfully implement a MOP, introduce an effective competence assessment process or analyse the hazards of maintenance tasks before an occurrence.