Performance Based Regulation (PBR) – EASA A-NPA & UK CAA Seminar
On 23 May 2014 the European Aviation Safety Agency (EASA) issued an Advanced – Notice of Proposed Amendment (A-NPA) on aviation regulation in Europe and potential updates to Regulation (EC) No 216/2008 (the so called EASA Basic Regulation). The A-NPA is open for comment until 15 August 2014: http://easa.europa.eu/document-library/notices-of-proposed-amendment/npa-2014-12
The A-NPA addresses seven areas of potential change:
- A performance-based, integrated approach to safety (in line with the International Civil Aviation Organisation’s thinking),
- Updating the EASA’s safety remit (e.g. being more proportionate to General Aviation, potentially including regulating ground handling and maintaining a central repository of licences and approvals),
- Extending the EASA’s remit beyond safety (e.g. security, environmental protection, wider involvement in Single European Skies, a greater aviation policy coordination role and potentially an EU central aircraft register)
- Optimising the use of available resources (allowing Member States to transfer tasks to EASA),
- Ensuring an adequate and stable EASA funding (raising various alternatives to the current mix of 1/3 EU funding and 2/3 from fees from certain sectors on industry),
- Further integration of aviation aspects (potentially towards a one tier regulator), and
- Aviation regulation beyond the EASA’s facets (essentially a question: ‘Is there anything else’ EASA should do?’).
The European Commission commentary on the A-NPA is here: http://ec.europa.eu/transport/modes/air/consultations/2014-aviation-safety_en.htm
There are many aspects of the A-NPA that will drive passionate comment, as no doubt the authors intended, particularly as some of the concepts mirror the wider debate on European politics.
This article will focus on the first of the potential changes discussed, the concept of a performance based approach to regulation, which is a more fundamental issue of safety regulation policy.
EASA ran a conference on this very subject 18 months ago: http://easa.europa.eu/conferences/pbo/
EASA observe that, traditionally:
… the safety regulatory approach has been mainly reactive to lessons learnt from accidents, serious incidents, or from oversight activities. This approach employs a prescriptive, compliance-based system addressing the different aviation domains.
The rapid increase of traffic volume… encourages an even more proactive and even predictive system, thanks to the vast amount of data now available in…a digitalised world full of monitoring systems. This suggests the development of a full Performance-Based Environment (PBE) including, amongst others, the establishment of safety performance indicators and targets.
While a performance based approach is not intended to replace the current regulatory system, it was inevitable that as more aviation organisations implement and mature their own Safety Management System, improvements would be made by the industry faster than the traditional, prescriptive rule-making system. While standardisation is still important in some areas, perhaps we will see more industry standards and less prescriptive regulation.
UK CAA Seminar
In the same week UK CAA staged a seminar for industry in Gatwick on their planned transition to Performance Based Regulation (PBR), which gave an insight to what this new approach will look like.
Mark Swan (CAA Director, Safety & Airspace Regulation Group) explained that this involves “a more risk and performance-based approach…by building a picture of aviation safety performance across the total system”.
In many ways it was clear from the CAA presentations that this transformation will require a major cultural shift in how individual regulators work. In crude terms it means less compliance focused ‘auditing’ and more ‘reviewing’ of effectiveness backed by good exchange of critical safety data with industry. Regulatory compliance is still expected, but it is clear that the change in EASA regulations from to ‘quality’ to ‘compliance monitoring’ was not purely a semantic exercise. Increasingly, approved organisations will need to show that they themselves are assuring compliance, freeing the regulators to examine the overall performance of organisation. However, the regulators will need to pay careful attention to applicants for new approvals.
Ben Alcott, CAA’s Programme Director for Enhancing Safety Performance, explained that such PBR needed to be:
CAA discussed how they would examine organisations with multiple approvals (say an AOC, CAMO and maintenance organisation) holistically as one entity. They also explained in some cases they might review separate organisations whose activity was tightly coupled (such as an aerodrome and their air traffic provider) together.
One critical element in these reviews being what the CAA quaintly termed ‘conversations’ with the Accountable Manager of each approved organisations. These may sound like cosy fireside chats but will be more a strategic review of risk and safety performance.
The invited industry speakers welcomed moving from a discussion on relatively trivial non-compliances found at audit to one that focuses the higher risk aspects of their operations. Industry did make it clear though that as technology and systems are moving rapidly, regulators often struggle to grasp the true reality of operations. But how much of a technical specialists do ‘PBR regulators’ need to be in future? Do they instead need have a stature and influence more like non-executive directors challenging safety performance and assessments or risk? Undoubtedly the CAA and other regulators need a staff who collectively can cover the full spectrum. It will be tricky working out the balance, but it will be even harder ensuring that not only do they have people with the right skills in place, but that they can sustain their workforce over time.
For more on the UK CAA’s thinking see: http://www.caa.co.uk/cap1184
For more on the general topic of PBR see this 2002 paper from the Harvard John F Kennedy School of Government: Performance-Based Regulation Prospects and Limitations in Health, Safety and Environmental Protection
UPDATE 1 : The presentations from their seminar are now available.
UPDATE 2: EASA have published a paper on A Harmonised European Approach to a Performance-Based Environment (PBE): http://easa.europa.eu/system/files/dfu/Report%20A%20Harmonised%20European%20Approach%20to%20a%20Performance%20Based%20Environment.pdf
UPDATE 3: Giancarlo Buono, IATA’s director of safety and operations, Europe, wants to see a partnership, rather than a “Tom and Jerry” relationship between regulator and operator (discussed in this Flight International leading article).
UPDATE 4: See this piece on lessons from the formation of the UK Military Aviation Authority (MAA): Regulatory Reflections & Resisting the Seduction of the Risk Management Process
UPDATE 5: The first meeting of the Performance Based Regulation Industry Group (PBRIG) was held at CAA House on 26 February 2015.
- Meeting 1 held on 26 February 2015 – Report and Presentation Slides
- Meeting 2 held on 19 May 2015 Report and Presentation Slides
- Meeting 3 held on 8 September 2015 Report
- Meeting 4 held on on 26 November 2015 Report
UPDATE 6: We look at a Canadian air accident that has lessons on how to introduce and regulate Safety Management Systems: Culture + Non Compliance + Mechanical Failures = DC3 Accident.
UPDATE 7: An NTSB report on a series of accidents at the Metro-North railway also has implications for the evolving concept of Performance Based Regulation (PBR). We discuss these further in a follow up article: Performance Based Regulation and Detecting the Pathogens that also looks at the crude oil train derailment and fire that killed 47 people at Lac-Megantic, Quebec, Canada.
UPDATE 8: We like the article on Performance Based Oversight and Regulation by Jules Kneepkens of TEASAS, which appeared to the Journal of Civil Aviation Training Issue 5/2015. Kneepkens notes:
Oversight of performance based rules will require a different and more advanced approach as it will trigger discussions on the basis of SPIs and targets and on whether the applied method is best suited to meet the performance goal. An exchange of views, rather than one-way communication from the competent authority to the regulated entity, will need to take place.
UPDATE 9: The UK CAA have published a report (CAP1365) on a conference held in London in October 2015 on PBR. It also described the CAA’s new Regulatory Safety Management System (RSMS) that sits above its Performance Based Oversight (PBO) process.
UPDATE 10: Prof Sidney Dekker comments on the danger that an SMS can become a “self-referential system”: a system that just exists for itself and is a sponge for data but one from which intelligence never emerges.
UPDATE 11: We look at an EU research project that recently investigated the concepts of organisational safety intelligence (the safety information available) and executive safety wisdom (in using that to make safety decisions) by interviewing 16 senior industry executives: Safety Intelligence & Safety Wisdom. They defined these as:
Safety Intelligence the various sources of quantitative information an organisation may use to identify and assess various threats.
Safety Wisdom the judgement and decision-making of those in senior positions who must decide what to do to remain safe, and how they also use quantitative and qualitative information to support those decisions.
UPDATE 12: We discuss new UK CAA guidance: Performance Based Oversight: Accountable Manager Meetings (CAP1508)
UPDATE 13: The Rule Illusion: Organisations should beware of a ‘rules bias’, a tendency to give in to risk aversion by establishing ever more rules:
Just like in the real world, stricter rules in an organization are rarely effective in stopping the most egregious transgressions.
In practice, those who make the rules are often insulated from the true consequences. The lack of a good feedback mechanism to adjust the rules would be bad enough if the rules were based on evidence and logic. But it’s often even worse, because rules often originate from received (but dubious) wisdom, unproven ‘common sense’, or reactions to one-off events. Rules that are based on beliefs rather than evidence, and never tested, are unlikely to produce net benefits. Yet such rules are precisely the ones the makers feel most protective of. Attempts to budge them meet with the backfire effect. That’s how we end up with sledgehammers to crack a nut.
Many rules were originally intended to be guidelines to help shape behaviour, but instead over time get codified into rigid laws. Often most people in the organization no longer know why the rules were even implemented. And yet depressingly, “Rule Nazis …. They cling to the rules like Leonardo DiCaprio clung to that door in Titanic — as if their lives depend on it. And they make sure everyone else does too, even when the rule doesn’t make sense or stands in the way of productivity,” as Heidi Grant says.
A recent post on the Science For Work blog sets out how important this sense of justice at work really is. Poorly enforced rules can easily undo all the other efforts.
Rules mostly creep in one by one. As their number grows, they contribute to a culture of conformity, rather than one of intelligent judgement and empowered employees who do what is right.
Hence the need to only establish the right rules and still leave room for empowered good judgement.
UPDATE 14: Leadership and Trust
UPDATE 15: Performance-based regulations have changed oversight responsibilities: When it comes to regulating the aviation industry, focusing on an organisation’s performance can pay large safety dividends, says Stephanie Shaw, the UK CAA’s Head of PBR.