How to develop a bowtie risk assessment model

Building upon the bowties

In order to further integrate your day to day management activities with risk management strategies consider using additional capabilities of the bowtie.

For example:

• Identify and link the ‘activities’ that are required in order to maintain the controls as intended.
• Develop barrier-based audits according to the controls and activities identified in the bowties.
• Map safety intelligence onto your bowties (e.g. recording and displaying control failures identified via the safety reporting system).

Monitoring safety by assigning Safety Performance Indicators (SPIs)

As the aviation industry moves further towards ‘performance based’ safety management, it is increasingly desirable to define the existing acceptable level of safety and monitor changes in performance using a range of Safety Performance Indicators SPIs.

Further information on SPIs (activity and outcome) is available if you access the Significant Seven bowtie templates.

Activity and outcome indicators

By identifying safety controls, the bowtie provides a framework for targeting data collection. It also provides a means of demonstrating the real world effectiveness of the controls and on how risk is being managed. This can be achieved through the creation of Safety Performance Indicators (SPIs) and example SPIs relevant to the Significant Seven templates are available for your consideration.

What are SPIs?

Measuring safety performance is a key activity of an effective safety management system (SMS). The BowTie barrier modelling process enables the identification of the controls that are in place to manage risks. The objective of SPIs is to measure how effective these controls are at preventing undesirable safety outcomes. They are also used to identify trends in performance relative to an organisation’s safety objectives, allowing action to take place to address undesirable changes.

There are broadly two types of SPIs:

• Activity indicators
• Outcome indicators

In order to appreciate the effectiveness of a control in preventing or mitigating the impact of a particular type of undesirable event (such as the use of standardised radiotelephony to prevent runway incursions), it is useful to monitor and measure effectiveness using a combination of activity and outcome indicators.

Activity indicators

Often referred to as leading indicators, an activity indicator is a measure of whether or not a control is actually in place. Activity indicators require a routine systematic check that key activities are undertaken as intended. It can be inferred then that these indicators are designed to assess the positive inputs into a safety system. They are proactive measures and often measure performance against a level of tolerance for a particular event type. Thus, these indicators highlight the need for action when a tolerance level is exceeded.

Outcome indicators

Often referred to as lagging indicators, an outcome indicator is a measurement of events after they have occurred. In the context of safety, these are almost always considered to be undesirable events. These indicators can inform an organisation as to the effectiveness of the controls in place. They are, by their very nature, reactive indicators.

Generally, outcome indicators can fall into three categories:

• Accidents: measures of the frequency or consequences of events that result in some form of loss, such as personal injury/death or damage to assets. In aviation, such measurements may be the number of mid-air collisions or the total lost time from personal injuries in the hangar.
• Precursors: measures that indicate less severe system failures or near misses, which when combined with other events or factors may lead to an accident or serious incident. They are still outcome indicators because an undesirable event has occurred. Examples may be a level bust, a pilot taking a wrong turn during taxi out, or an engineer leaving a tool inside an engine after maintenance.
• Results: these are simply the outcomes of activities undertaken. They could range from audit findings to the results of a safety culture survey of operational staff.

Acknowledgement

The above information on Safety Performance Indicators has been developed from guidance provided to the UK rail industry by RSSB (Rail Safety & Standards Board).