Despite a lot of papers published about how to assess occurrence safety risk, many times during the implementation of our Galiot SMS - Safety Management System we have encountered inadequate risk assessment method selection, lack of understanding of the difference between safety issues, and historical events assessment and what are the goals of event risk assessment.

Therefore, I believe that this brief review of main risk evaluation methods might be useful for safety practitioners involved in the investigation and classification of safety occurrences in aviation

Outputs of occurrence risk assessment should give answers to these two questions:

Typical answers to this question are: a) Investigate immediately and take actions, b) Investigate and carry out a further risk assessment, c) Use data for improvements

In the transition from traditional compliance-based prescriptive schemes towards a performance-based approach, measuring events' safety risk values is recognized as one of the top priorities in the context of a Safety Management System.

Quantitative risk values are the relative risk values between events, calculated differently in the methods discussed below.

This method has three shortcomings: a) qualitative output limited to max 3 categories, b) no quantitative risk value, c) focused only on actual consequences and neglecting potential but not reached outcomes.

Nowadays, when accidents are very rare events (less than 3 per 10M flights), from a service provider perspective, this method is even more limited to 2 output values, resulting in poor and insignificant performance measurement, even for several years period calculation.

ICAO Risk Matrix is a safety issue risk assessment method based on “projected likelihood and severity of the consequences or outcomes from an existing hazard or situation”.

The process of the risk assessments with this method could be divided into 3 steps:

1. Assessing probability (by selecting one of the predefined values such as Frequent, Occasional, Remote, Improbable, and Extremely Improbable) where probability is defined as “the likelihood or frequency that a safety consequence or outcome might occur”.

2. Assessing severity (by selecting one of the predefined values such as Catastrophic, Hazardous, Major, Minor, and Negligible) where severity is defined as “the extent of harm that might reasonably occur as a consequence or outcome of the identified hazard”.

3. Selecting safety risk index (from 1E to 5A) and related risk tolerability (Red/Yellow/Green) from the risk matrix.

While this method has been successfully used for safety issue risk assessments, it has some general limitations and some specific drawbacks for individual event risk assessments:

Qualitative output limited to 3 categories (Inacceptable, Tolerable, Acceptable) is unable to distinguish the relative importance of risk in the same category.

There are no predefined methods for event probability and severity evaluation

Changing the number of probabilities/severities options and/or risk coloring schema without an understanding of Cox’s Matrix Theorem can lead to “worse-than-random” risk level selection.

The probability part is based on a concept of overall probability (product of average event probability and average conditional probability) of a certain consequence for a certain hazard for ALL similar events and NOT on the probability of the consequence at the moment when a SPECIFIC event occurred.

The key objectives of the Event Risk Classification (ERC) method developed by the ARMS working group are to overcome limitations of the ICAO Matrix and to reduce analyst subjectivity about the probability of the consequence.

In contrast to the ICAO Matrix where risk evaluation is based on all similar events of the same type (events resulting from the same hazard), the ERC method is based on the concept of “event-based risk level” where event risk is considered as the risk which was there when the event happened. The event risk classification is performed by use of ERC matrix (ARMS propose a 4x4 matrix but it could be customized to meet specific requirements) where a safety expert is required to answer two questions:

1. If this event had escalated into an accident, what would have been the most credible outcome?

2. What was the effectiveness of the remaining barriers between this event and the most credible outcome?

While the first question related to severity assessment is similar to the ICAO Risk Matrix approach, the second one, related to the probability of the outcome, is based on the evaluation of the remaining barriers preventing event escalation to the accident. This approach follows the “Swiss-Cheese” accident causation model developed by James Reason.

The following answers to the second question are proposed by the ARMS group:

ERC model has two outputs:

Several authors proposed enhanced matrix versions with better resolutions (like Jochen Mickel), but these models will not be discussed here because they are following the same principles explained above.

The European Risk Classification Schema (ERCS) follows core principles of the Event Risk Classification (ERC) method:

a) event-based risk level assessment

b) probability assessment based on the effectiveness of the stoping and remaining barriers

c) qualitative and quantitative safety risk score of an occurrence and not its actual outcome

Compared to Event Risk Classification ERC, this model introduces identification of the

The ERCS shall consist of the following steps:

The so-called key risk areas are: a) airborne collision, b) aircraft upset, c) collision on the runway, d) excursion, e) fire smoke and pressurization, f) ground damage, g) obstacle collision in flight, h) terrain collision, i) other injuries and j) security.

Proposed categories are X=more than 100 possible fatalities or equivalent size aircraft for cargo, S = between 20 to 100 possible fatalities or equivalent size aircraft for cargo, M = between 2 to 19 possible fatalities or equivalent size aircraft for cargo, I = 1 possible fatality, E = serious and minor injuries as there are no fatalities and A =no likelihood of an accident.

Proposed severity scores are A - for no likelihood of an accident, E - for an accident involving minor and serious injuries, I - for an accident involving a single fatality, M - for a major accident with a limited (2-19) number of fatalities, S - for a significant accident with potential for 100 fatalities, X - for a catastrophic accident with the potential for more than 100 fatalities.

Stopping barrier is barrier prevented event to escalate into an accident (if exists).

The remaining barriers are barriers that were placed between stopping barrier and the potential outcome. Barriers placed before stopping barrier should not be considered in the calculation because they do not prevent accident causation.

The effectiveness of each barrier shall be classified as:

a) Stopped, b) Remaining Known, c) Remaining Assumed, d) Failed Known, e) Failed Assumed and f) Not Applicable.

Barrier weight sum and corresponding barrier score are calculated by summing barrier weights for all barriers classified as Stopped, Remaining Know and Remaining Assumed.

Barriers classified as Failed or Not Applicable shall not be counted for the final score).

The numerical value of risk magnitude is mandatorily required to facilitate the aggregation and numerical analysis of multiple occurrences.

a) it has been designed for safety issue risk assessment only

b) qualitative only risk level output

c) inappropriate probability assessment based on a concept of overall probability (product of average event probability and average conditional probability) which is not applicable for historical events

Galiot Aero Ltd

February, 2021