A novel approach to runway overrun risk assessment using FRAM and flight data monitoring

Abstract

Runway overruns (ROs) are the result of an aircraft rolling beyond the end of a runway, which is one of the accident’s types that most frequently occurs on aviation. The risk of an RO arises from the synergistic effect among its precursors, such as unstable approaches, long touchdowns and inadequate use of deceleration devices. To analyse this complex socio-technical system, the current work proposes a customised functional resonance analysis method, called FRAM-FDM, as traditional techniques of risk and safety assessment do not identify the interactions and couplings between the various functional aspects of the system itself, especially regarding human and organisational components. Basically, FRAM-FDM is the coupling of a traditional FRAM with flight data monitoring (FDM) techniques, used here to quantify the variabilities of the flight crew performance while executing the required activity (i.e. the landing). In this proposal, these variabilities (i.e. the FRAM functions aspects) are aggregated by the addend of a logistic regression, resulting in a model to evaluate the flare operations and the brake application profile effect on the remaining distance to the end of the runway, used as a reference to classify the landing as acceptable or not. The present application of the FRAM-FDM assesses the operational risk of a sample fleet in overrunning the runway during landing, highlighting the brake pedal application profile as the most relevant contributor. The model improves the knowledge about the system behaviour, being useful to direct flight crew training.

Nomenclature

AFM

aircraft flight manual

AGL

above ground level

ATM

air traffic management

BBN

Bayesian belief network

BN

Bayesian network

DA

decision altitude

EMAS

engineered material arresting system

ETTO

efficiency-thoroughness trade-off

FDM

flight data monitoring

FDR

flight data recorder

FPA

flight path angle

FRAM

functional resonance analysis method

GS

glideslope

ILS

instrument landing system

JCS

joint cognitive systems

LOC

localiser

MDA

minimum descent altitude

MSAW

minimum safe altitude warning

NA

not applicable

PAPI

precision approach path indicator

QAR

quick access recorder

RAM

resilience analysis matrix

RESA

runway end safety areas

RNP

required navigation performance

RO

runway overrun

ROAAS

runway overrun awareness and alerting system

ROPS

runway overrun prevention system

SMS

safety management system

SPIN

Simple Promela Interpreter

TLA

thrust lever angle

tsv

tab-separated values

ULD

unfactored landing distance

v_ref

reference speed

1.0 Introduction

Runway overruns (ROs) are the result of an aircraft rolling beyond the end of a runway, which can occur on take-off or landing [1]. They are the most frequent accident type in aviation, representing one of the most serious risks and economically significant safety-event type [2]. ROs derive from a complex combination of factors involving different stakeholders in the aviation industry including airlines, aerodromes, aircraft manufacturers and air navigation service providers. The complexity of the RO prevention comes from the fact that the effect of the risk and resilience factors are highly cumulative – runway condition maintenance and reporting; aircraft performance and operations; collaborative flight path management; and adherence to robust policies for safe descent and approach planning, stable approach, safe landing and go-around are some examples [2].

The Functional Resonance Analysis Method, or FRAM, derived from a need to understand and describe how performance in complex dynamic socio-technical systems like this one unfolds, and how the “mechanisms” behind everyday performance variability may be modelled [3]. It provides a way to describe outcomes using the idea of resonance arising from the variability of everyday performance. This work applies a customised and quantitative FRAM, named FRAM-FDM, for the operational risk assessment of runway overruns during landing.

1.1 The runway overrun in literature

Due to its frequency and complexity, several papers are being published to address this issue. Some focuses on the prevention of a RO; others aim to reduce its consequences. The Runway End Safety Areas (RESA) (i.e. an area symmetrical about the extended runway centre line and adjacent to the end of the strip) and the Engineered Material Arresting System (EMAS) (i.e. a soft ground employed to decelerate or stop an overrunning aircraft) are practices available to prevent the consequences of the overrun, once it occurs. Yang et al. [4] provides an overview about EMAS, including statistics of the final locations of overrunning aircraft. Benedetto et al. [5] proposes an integrated method for the prediction of rut depths in natural soils induced by aircraft overrunning, based on a numerical streamlined model.

Moretti et al. [6] and Kirkland et al. [7] looked at runway excursion events from an aerodrome’s perspective. Moretti et al. [6] proposes the quantification of veer-off events risk by multiplying the severity and the probability. Severity was given through a damage model, which considers statistical mechanical damages and injuries observed in the examined accidents database. Probability distributions described the final position of an aircraft after a veer-off. Kirkland et al. [7] also suggests a combination of the mechanical damages with the location of wreckage to assess the risk of any particular airport, but also including the probability of overruns. Yet, their method offers a historical overview and does not address the operational conditions of the take-off or landing procedure.

Zhao and Zhang [8] used a back propagation neural network model to predict the aircraft landing distance on a dry runway based on airport altitude, runway slope, wind conditions, ambient temperature and aircraft weight. Kang et al. [9] and Tong et al. [10] proposed deep neural networks model which utilises flight data for predicting long landing and groundspeed, respectively. Puranik et al. [11] built a global predictive model to predict the true airspeed and groundspeed at the touchdown point about of 18 seconds in advance. Studies like these serve as basis for on-board real-time performance monitoring and alerting systems, used to prevent an overrun during landing by assisting the flight crew with the land/go-around decision and alerting when more deceleration force is needed during the landing roll (i.e. increasing the flight crew situational awareness). The Airbus Runway Overrun Prevention System (ROPS) and the Embraer Phenom 300 Runway Overrun Awareness and Alerting System (ROAAS) are commercial examples of this kind of functionality [12, 13].

Other articles aim to combine the effects of the RO contributors into a risk assessment model. Ayra et al. [14] and Calle-Alonso et al. [15] proposed the use of Bayesian networks (BNs) to merge the ROs precursors’ probabilities. They include the stable approach, height and speed at the runway threshold, wind and the use of the thrust reverser as well as autobrake in the model. To apply the BN, the variables are categorised (e.g. stable approaches (yes or no)), and the output is given by the remaining distance to the end of the runway when the groundspeed is greater than or equal to 80 knots, with the risk being associated with a distance shorter than 2,500 feet. Barry [16] also used BNs with flight data to assess risks but of a veer-off (i.e. an aircraft departing the lateral limits of a runway).

Li et al. [17], Alnasser and Czado [18], Reiser et al. [19] and Wang et al. [20] suggested methods to estimate the likelihood of runway overrun precursors without a previous categorisation. Li et al. [17] proposes a method to examine the effects of pilot behaviour characteristics on the risk of runway overrun during take-off based on flight data, using similarity theory to construct a data-based man–machine-environment system expression. Alnasser and Czado [18] used flight data of a specific aircraft type and landing airport and applied a D-Vine copula-based quantile regression, estimating the probability of the excess of energy at 2,500 meters of remaining runway. Reiser et al. [19] employed a customised FRAM for operational risk assessment to analyse the long touchdown. The customisation is related with a proposal to aggregate the functions usually executed during the touchdown procedure in a quantitative way through a linear regression, using flight data to estimate the functions’ variabilities. Wang et al. [20] examined the effects of pilots’ flare operation on landing incidents based on flight data, especially its influences on landing touchdown distance and touchdown vertical load, which are two parameters determining long landing and hard landing. Analysis of variance was used to look for differences of flare operations between normal and abnormal landings, including their parameter differences at the initial flare point and throughout the flare process. A multiple linear regression model was developed for analysing the initial flare operation’s effect on touchdown distance, which is defined, in their work, as the horizontal distance from the radio altitude of 50 feet to touchdown point in landing process. The logistic regression model was built for examining contributions of flare operation on the occurrence probability of hard landings. Unsurprisingly, they concluded that pilot inputs did effect touchdown distance and vertical acceleration at touchdown, but possibly of more value were their findings around pitch and thrust control technique, i.e. a positive and steady pitch up input, coupled with a relatively slow reduction in thrust led to better landing outcomes.

The present work goes in the same direction as the Alnasser and Czado [18], Reiser et al. [19] and Wang et al. [20] as the proposal aims to assess the risk of a RO and does not demand a previous categorisation of the precursors. None of the previous studies included the effects of the use of the deceleration devices after the touchdown.

1.2 FRAM in literature

FRAM-FDM is a proposal that integrates quantitative approaches to FRAM to enhance its strengths. Many authors have already tried to evaluate the value of performance variability that can arise from the different instantiations of a FRAM model quantitatively (or at least semi-quantitatively).

Oliveira et al. [21] combined FRAM with Bayesian belief network (BBN) in order to demonstrate, quantitatively, the existing variability in functions that are part of the complex navigation system based on the Required Navigation Performance (RNP) procedure, specifically in approach phase.

Adriaensen et al. [22] developed a FRAM model to understand the information flow propagation in a cockpit environment under normal conditions, following a joint cognitive systems (JCS) perspective. They used a semi-quantitative method to identify critical functional couplings through the number of downstream dependencies for each function.

Patriarca et al. [23] proposes the resilience analysis matrix (RAM) to enhance the strength of FRAM-based accident analyses by reducing the complexity of FRAM’s representation. The RAM offers a two-dimensional representation which highlights systematically connections among couplings. Patriarca et al. [24] presented a semi-quantitative application of FRAM by quantifying and aggregating functions’ variabilities in relation to the air traffic management (ATM) system based on Monte Carlo simulation.

Yang et al. [25] proposes a method based on FRAM by using formal verification as a supporting tool to efficiently check the functional resonance after identifying the couplings of functional variability. They used Simple Promela Interpreter (SPIN) to demonstrate the functional resonance in system and applied the proposed method to a developing ATM system where a minimum safe altitude warning (MSAW) subsystem is being introduced.

1.3 Aim and structure of the study

The objective of this study is to identify the most significant contributors of an anticipated runway overrun of a specific fleet. Actions to reduce the risk of this fleet may be then directed in a more effective way.

The FRAM-FDM method consists of the application of FRAM to model and understand the system in a more holistic way integrated with flight data analysis to quantify the system variabilities. It seeks to identify and measure the variabilities of the system, mainly caused by human involvement. A logistic regression model is then built for examining contributions of final approach and landing operation on the occurrence probability of runway overrun, including the evaluation of the use of the deceleration devices. As a result, it recognises the variabilities that have a stronger impact in the system outcome.

This paper is organised as follows. Section 1.0 introduced the paper objectives and approach. Section 2.0 explains how an overrun develops. Section 3.0 details the proposed method while Section 4.0 illustrates its application with the aid of the flight data of a sample fleet. Lastly, the conclusions summarise the outcomes of the research and discusses potential future paths.

2.0 The runway overrun

Most of the ROs occur in landing phase and are associated with undesired aircraft states, like unstable approaches and/or long touchdown [26]. Adverse weather, runway surface conditions, deficiencies in aerodrome facilities and inadequate use of deceleration devices also play a relevant role.

Unstable approach is defined as a consequence of the aircraft deviation of the following criteria: (1) the correct flight path; (2) the expected airspeed; and (3) the right landing configuration.

The aircraft must be on the correct flight path, normally given by a three-degree approach track [27]. This path is usually indicated by the instrument landing system (ILS) and visual aids (e.g. precision approach path indicator (PAPI) and runway surface markings). It typically results in a runway threshold crossing at a height of 50 feet and a touchdown point of 1,000 feet beyond the threshold (Fig. 1).

Figure 1. Landing references.

The ILS is a precision runway approach support employing two radio beams to provide pilots with vertical and horizontal guidance during the approach. The localiser (LOC) provides azimuth guidance, while the glideslope (GS) defines the correct vertical descent profile (Fig. 2). They usually guide the flight crew until the decision altitude (DA) in a precision approach or minimum descent altitude (MDA) in a non-precision one. After that, the flight crew performs a visual approach and uses the visual aids as references. The PAPI system is a set of lights that the flight crew can see during the final approach. The ratio of white to red lights seen is dependent on the angle of approach to the runway (Fig. 3). Above the designated glideslope a pilot will observe more white lights than red; at approaches below the ideal angle, more red lights than white will be seen.

Figure 2. Approach radio navigation aids – ILS.

Figure 3. Approach visual aids. (a) PAPI lights. (b) Some runway markings.

The aircraft airspeed should not be more than the reference speed (v_ref) + 20 knots and not less than v_ref during approach [27]. Still, it must be equal to v_ref at the runway threshold to ensure that the estimated unfactored landing distance (ULD) will be achieved. ULD is the distance used by an aircraft in landing and braking to a complete stop (on a dry runway at sea level) after crossing the runway threshold at 50-feet above ground level (AGL) with v_ref in landing configuration [28]. The ULD is determined from data obtained from aircraft certification campaign with maximum brake application and without the use of thrust reverser. Corrections for airport elevation, aircraft weight, wind and icing conditions are available at the manufacturer aircraft flight manual (AFM).

The aircraft must be in the correct landing configuration [27]. The flight crew must extend the landing gear and select the slat/flap position to obtain the correct configuration. Engine power/thrust must be appropriately set.

A long touchdown occurs when an aircraft touches the ground far ahead of the aiming point, which is usually 1,000 feet from the runway threshold. Snow, rain, fog and haze might reduce the crew’s ability to obtain visual contact with the runway, increasing the probability of a long touchdown [29]. A manoeuvre called flare could also be saw as one of the reasons for a long touchdown. Under 50 feet of height, the pilots are required to change the aircraft attitude in a few seconds for a safe and smooth touchdown. They raise the aircraft nose to both land the aircraft on the main landing gear first and decrease the descent rate and vertical load. Nonetheless, a firm touchdown is recommended, particularly on wet or contaminated runways, to minimise the risk of aquaplaning [2, 30]. If not executed correctly, lasting too much time, the flare may result in a long touchdown.

After the touchdown, the flight crew must decelerate the aircraft using the available devices, such as ground spoilers, wheel brakes and thrust reverser systems. The inadequate or late use of these devices as well as a poor runway surface condition may increase the distance required to stop the aircraft. Ground spoilers are panels mounted on the upper surface of the wing. When extended, they dump the lift raising the load on the wheels and thus improving the wheel-brake efficiency. They also increase aerodynamic drag contributing to aircraft deceleration. The spoilers usually deploy automatically (if armed) upon touchdown or upon thrust reverser’s activation.

Wheel brakes are located on the wheels of the main landing gears. Braking action results from the friction force between the tires and the runway surface. It depends on aircraft speed, wheel speed (i.e. free rolling, skidding or locked), tire condition and pressure, runway surface condition and its friction coefficient, the load applied on the wheel, and the number of operative brakes. Normal brakes are applied through pedals and, sometimes, differential braking is necessary to control the aircraft laterally. Anti-skid systems prevent tire skidding and maximise brake efficiency according to the runway surface. Autobrake systems provide automatic braking at maximum deceleration rates, which varies according to runway surface conditions and its mode selection (e.g. low, medium, high). Emergency brakes may be used in case of normal brakes failure. In this case, the pilot must pull the handle carefully and slowly, modulating the braking action as there is no anti-skid protection.

Thrust reverser systems, whose efficiency is higher at high speeds, must be selected as early as possible after touchdown. They provide a deceleration force that is independent of runway surface condition [31].

To summarise, as depicted on Fig. 4, an unstable approach may result in an increased height or speed over the runway threshold. The certified ULDs, provided in the AFM, are determined based on the assumption that the landing gear is positioned 50-feet above the threshold. According to Flight Safety Foundation, for every 10 feet above this standard, landing air distance (i.e. the distance between 50-feet and the touchdown point) will geometrically increase 200 feet [28]. An excessive speed over the runway threshold may also result in a long touchdown as well as a higher speed from which the pilot must stop the aircraft [32]. A 10% increase in final approach speed results in a 20% increase in ULD, assuming a normal flare and touchdown [28]. The long touchdown directly reduces the runway available for braking. When the crew does not identify a critical landing or is not aware of a real and unfavourable runway surface condition, they may start with a usual deceleration procedure. For each second beyond 2 seconds of delay to employ the deceleration devices, 200 feet may be added to the braking distance [28]. Besides, less than maximum brake also impacts this distance.

The risk of a RO increases when more than one precursor (i.e. unstable approach, long touchdown, inadequate use of deceleration devices) is present as multiple hazards create a synergistic effect. A proper safety management system (SMS) methodology should effectively identify increased-risk operations by combining the effects of precursors [33].

3.0 The proposed method: FRAM-FDM

FRAM is a method-sine-model, whose purpose is to build a model of how things happen rather than to interpret what happens in the terms of a model. It is built over the following four principles. First, failures and successes are equivalent in the sense that they have the same origin. In other words, things go right and go wrong for the same reasons. Thus, understand what goes right when the daily work is carried out is as important as understanding what failed in the system [34].

Second, the everyday performance of socio-technical systems, including humans individually and collectively, is always adjusted to match the conditions. Workers usually need to make some trade-offs between being efficient and to make sure the work can be completed as precisely as possible. These kinds of adjustments are named efficiency-thoroughness trade-offs (ETTOs). They are necessary and understandable; however, any changed system behaviour may raise variabilities in the system [35].

The third principle states that many of the outcomes we notice – as well as many that we do not – must be described as emergent rather than resultant. To be more specific, minor variabilities always exist in normal system operations and do not affect system safety. Nevertheless, a particular external environment may integrate variabilities and magnify their influence to generate an undesired outcome [35].

Figure 4. Summary of factors that may affect the landing distance.

Fourth, the relations and dependencies among the functions of a system must be described as they develop in a specific situation rather than as predetermined cause–effect links. This is done by using functional resonance [34].

FRAM does not imply that events happen in a specific way, or that any predefined components, entities, or relations must be part of the description. Instead, it focuses on describing what happens in terms of the functions involved. These are derived from what is necessary to achieve an aim or perform an activity, hence from a description of work-as-done rather than work-as-imagined. But functions are not defined a priori nor necessarily ordered in a predefined way such as hierarchy. Instead, they are described individually, and the relations between them are defined by empirically established functional dependencies (Fig. 5) [34].

Figure 5. Summary of the FRAM model.

The following paragraphs present the four (4) steps to perform the current customised and quantitative FRAM analysis, the FRAM-FDM. Figure 6 illustrates the proposed method. Note that the Step 3 adjusts the Step 1 due to the knowledge improvement about the system.

Figure 6. Correlation between the FRAM and FRAM-FDM steps.

3.1 Functions identification and description

FRAM’s first step deconstructs the complex sociotechnical system into ‘functions’, that represent an activity that is required to produce a certain outcome. The first step identifies these functions that are needed for everyday work to succeed and characterised by six different aspects as follows. Aspects are traditionally placed at the corners of a hexagon, which represents the function itself (Fig. 7).

Figure 7. A hexagon representing a function.

The input (I) activates or starts a function and/or is used or transformed by the function to produce the output (O), which is the result of the function. The output can be either an entity or a state change and serves as input to the downstream functions. Preconditions (P) are mandatory conditions that must exist before carrying out the function. They do not necessarily imply the function execution. The function needs the resource (R) when it is carried out, or consumes it, to produce the output. Controls (C) supervise, regulate or monitor the function. They are exemplified by guidelines, regulations or even social expectations. Temporal requirements or constraints of the function, regarding both duration and starting point, are given by time (T).

3.2 Performance variability characterisation

The idea of the second step is to characterise the variability of the functions that constitute the FRAM model, assessing how performance variability will show itself – either in the sense of how it can be observed or detected – or in the sense of how it may affect downstream functions. One way to do that is to evaluate subjectively the outputs variabilities. For instance, they could vary in terms of timing or precision. An output can occur too early, on time, too late or not at all. Regarding precision, it can be precise, acceptable or imprecise.

Instead of evaluating functions variability in a subjective way, this work uses FDM techniques to estimate the variabilities. An FDM program is designed to enhance safety by identifying airlines’ operational safety risks, based on the routine analysis of flight data during revenue flights. These data are compared against pre-defined envelopes and values, to check whether the aircraft has been flown outside the scope of the standard operating procedures.

Essentially, flight data are converted into engineering units in a process called decoding, which is normally described in a data frame layout document. The decoded aircraft parameters represent the entry point and are used for measurement calculations, which may be analysed by means of distributions or directly compared with a threshold to produce events (Fig. 8) [36]. The distributions and events are investigated to highlight any trend that could show a latent or potential risk. The FDM program is applied reactively through analysis of incidents or accidents; proactively through analysis of the airline’s activities; and predictively through data gathering to identify possible adverse future outcomes [37].

Figure 8. Basic entities of the FDM program.

For ROs, EASA Guidance for the Implementation of Flight Data Monitoring Precursors [36] proposes algorithms to address hazards like the identification and quantification of unstable approaches; the recognition of high energy over the runway threshold through the measure of height and speeds; and the detection of inadequate use of deceleration devices like late or insufficient activation of thrust reverser and wheel brakes during rejected take-offs and landings. These algorithms are dependent on the establishment of thresholds as well as they are independent from each other. The risk for the monitored fleet remains unknown as the interactive effect among the hazards is not explored.

The variabilities of the FRAM functions are here estimated using algorithms that are similar to the proposed by the EASA Guidance. However, instead of focusing on events (i.e. how many times the parameter crosses the stablished threshold), the purpose is to understand how the system works on a daily basis and estimate how each function varies in the ‘real world’. Note that the aircraft systems are usually working as expected. Thus, the everyday variabilities of the functions are related with the human performance (i.e. the flight crew handling during the landing execution).

Flight data is the information coming from aircraft sensors, onboard computers and other instruments that are recorded into a crash-survivable flight data recorder (FDR) and occasionally also into easily accessible quick access recorder (QAR). In this work, FDR raw data from a typical jet is processed into Tab-Separated Values (tsv) files by an internal and non-commercial tool and analysed via R functions. R is a programming language and free software environment for statistical computing and graphics [38].

For most aircraft, FDR file format follows the ARINC 717 standard, being composed by a continuous stream of bits. ARINC 717 divides data into subframes and frames. A subframe is one second long and contains anywhere from 64 12-bit words, up to 2048 12-bit words, depending on the aircraft. A frame is composed of four subframes. Every frame contains exactly the same information (i.e. parameters distribution). The four subframes within each frame are not identical. Some parameters may be found in every subframe, while other may appear only once per frame. A parameter may be discrete (e.g. air-ground transition) or continuous (e.g. airspeed, groundspeed and height). In this case, an equation is usually set to convert the recorded set of bits into the data. For example, the airspeed may be recorded into Subframes 1 to 4 at Word 37 from bits 1 to 10 in a resolution of 0.5 knots. This means that the airspeed is recorded in a 1 sample per second (sps) and multiplied by 0.5, specifying a range from 0 to 511.5 knots. Given the FDR file in an ARINC 717 format and the map with the parameters location as stated by the aircraft manufacturer, some commercial tools are able to convert the raw data into a table as shown in Fig. 9. Schwaiger and Holzapfel [39] proposes a method for the decoding of binary ARINC 717 recorder files.

Figure 9. FDR raw data converted into a tsv file.

Following an eight (8) sps basis of the current tsv file, the analysis through R functions starts by interpolating the selected parameters to fill the gaps inside the converted flight data file, caused by a smaller sample rate. Continuous parameters are usually linearly interpolated to enhance their precision (i.e. to avoid steps among the samples). Continuous parameters that represent a position (e.g. thrust lever and flap surface position) as well as discrete parameters remain constant until the next sample.

Further steps consist of splitting the data into flights, characterised by the engines’ turning on and off, and of capturing the final approach/landing phase, starting at circa of 100 feet AGL prior to the air to ground transition. The FRAM functions outputs are then captured or computed for each flight cycle.

3.3 Variability aggregation

FRAM models the potential couplings among functions, not showing the effects of a specific scenario. The variability aggregation step focuses instead on examining specific instantiations of the model to understand how the potential variability of each function can become resonant, leading to unexpected results. It is therefore necessary to identify the functional upstream-downstream couplings. The variability of a function results as a combination of the function variability itself and the variability deriving from the outputs of the upstream functions, depending on the function type and the linked aspects type.

This step may be addressed qualitatively, based on potential for dampening performance variability ranges from +1 to +3 and for increasing performance variability ranges from −1 to −3 [24]. Yet, this step is addressed quantitatively here, through a correlation check between the FRAM functions aspects to validate the model followed by a logistic regression to describe the system output in terms of its upstream functions aspects.

Correlation is a dimensionless quantity that can be used to compare the relationships between pairs of variables in different units [40]. Spearman’s correlation coefficient is a statistical measure that assesses the strength and direction of a monotonic relationship between two variables. It ranges from −1 and 1, where 0 indicates no correlation, 1 means total positive correlation, and −1 signifies total negative one. The correlation confirms or discards the influences between the FRAM functions, validating and improving the model that was initially proposed on an expert experience basis.

Logistic regression is a type of predictive model that can be used when the target variable is binary, like acceptable or unacceptable outcomes. The logistic function, commonly referred to as the sigmoid function, describes the association between the predictor variables and the likelihood of the binary output. Its coefficients estimation is carried out using maximum likelihood [41]. The logistic regression is used in the current work to classify the cycles into acceptable or unacceptable outcome of the FRAM model, and to recognise the influence of each predictor variable in the target one in terms of probabilities.

An important problem in many applications of regression analysis involves selecting the set of predictor variables to be used. This selection is performed with the use of FRAM final model and the correlation between the functions’ outputs. Only variables that are independent from each other are used as inputs in the final model. Their independence is checked using the Spearman method.

3.4 Variability management

This last step consists of monitoring and managing the performance variability, identified by the functional resonance in the previous steps. Performance variability can lead both to positive and negative outcomes. The best strategy consists of amplifying the positive effects, i.e. facilitating their happening without losing control of the activities, and damping the negative effects, eliminating and preventing their occurrence.

Using a logistic regression, the exponential function of its coefficients is the odds ratio associated with a one-unit increase in the exposure. Odds ratios (OR) are used to compare the relative likelihood of the outcome occurrence (i.e. the runway overrun), given exposure to the predictor variables (e.g. height and speed at the runway threshold). The odds ratio can also be used to determine whether a particular exposure is a risk factor for a particular outcome, and to compare the magnitude of various risk factors for that outcome.

4.0 Case study

The methodology described in Section 3.0 was applied to the landing procedure of a sample fleet, starting at a height of 50 feet and ending at a groundspeed of 30 knots. It comprises the flare execution and the brake application profile. A total of 632 landings were analysed.

4.1 Functions identification and description

Figure 10 shows a diagram with the functions considered relevant to explain the performance variability of the landing procedure, while Table 1 summarises its variables. The diagram was drawn using the FRAM Model Visualiser.

The landing procedure starts before take-off, during the flight planning phase. The flight crew selects a runway to land the aircraft, whose required length is based on the aircraft ULD. The ‘To Prepare the Flight’ is a function that incorporates the landing assessment, evaluating if the designated runway is long enough to accommodate the landing given the airport altitude, environmental conditions, and aircraft status/ configuration. The runway margins, given by Equation (1), influences the touchdown point and the brake application profile, which is more aggressive in shorter runways. It is also directed related to the remaining runway at the initial point of the taxiing procedure (i.e. a groundspeed of 30 knots).

(1) \begin{align} RunwayMargins = \frac{{RunwayLenght}}{{ULD}}\end{align}

Theoretically, the landing procedure starts at the runway threshold with the aircraft at 50-feet AGL in landing configuration with the reference speed (v_ref). The ‘To Cross the 50-feet Height’ is a function that initiates the analysis. Its outputs could be the Δv_ref (i.e. the difference between the airspeed and the v_ref), the thrust lever angle (TLA), the pitch attitude and the flight path angle (FPA). The value of Δv_ref and TLA at 50-feet AGL are intrinsically correlated with Δv_ref and TLA at the runway threshold, respectively. TLA and the pitch attitude are manipulated by the flight crew during the flare. The FPA, associated with the height at the runway threshold, influences the touchdown point geometrically.

‘To Cross the Runway Threshold’ usually succeed the ‘To Cross the 50-feet Height’ in the fleet under analysis. Its outputs, height, Δv_ref and TLA, affect the touchdown point.

The ‘To Perform Flare’ function consists of the descent rate reduction to accomplish a smooth landing. It is normally performed near to the ground (less than 50-feet AGL) through the increase of aircraft pitch attitude simultaneously with the reduction of engine thrust. A firm touchdown results in a smaller touchdown point. Its output is given by the descent rate at the wheel spin (i.e. the moment that the wheels start rotating). In addition, this function has some internal variables such as the TLA and pitch attitude change. The most significant internal variable for its output is the moment that the aircraft reaches the maximum pitch in relation to the wheel spin. An earlier maximum pitch results in a lower descent rate and, consequently, a longer touchdown point.

‘To Touchdown’ outputs are the air-ground transition, the touchdown point and the groundspeed at this moment. The air-ground transition starts the functions that decelerate the aircraft (i.e. ‘To Press the Brake Pedal’ and ‘To Command Pitch’). The fleet under analysis does not have ground spoiler, thrust reverser nor autobrake systems. The touchdown point is given by distance between the runway threshold and the aircraft position at the first air-ground transition, impacting the remaining distance to the end of the runway at 30 knots directly. The groundspeed at touchdown is the resource that must be consumed during the deceleration process.

Table 1. Summary of the FRAM diagram variables

¹Not applicable

Figure 10. Proposed FRAM diagram for landing procedure with focus on ROs.

The flight crew starts to press the brake pedal as soon as the aircraft touches the ground, typically prior to the first air-ground transition. Nonetheless, the brake pedal position varies during the landing run as well as does not necessarily reach the 100% in the daily operation. The brake pedal output is, thus, composed by the mean application between the aircraft wheel spin, that occurs prior to the air-ground transition, and the landing end (i.e. with a groundspeed of 30 knots). The brake pedal parameter affects the ‘To Decelerate the Aircraft’ function indirectly, via the ‘To Apply Brake Pressure’ function, as the anti-skid system modulates the brake pressure based on the pedal position and the runway surface condition.

The landing roll should be performed with pitch command in neutral position. A pitch down command moves the load distribution from the main landing gear to the nose one, diminishing braking efficiency and affecting the aircraft deceleration.

Thus, the ‘To Decelerate the Aircraft’ receives the brake pressure mean and the pitch mean during the landing run as inputs. Note that, when available for the designated runway, the information regarding runway surface conditions is checked by the flight crew during the approach, represented by ‘To Check Environmental Conditions’ function.

The ‘To Decelerate the Aircraft’ output is the longitudinal deceleration, that associated with the runway margins and the touchdown point, compiles the remaining distance to the end of the runway when the groundspeed is equal to 30 knots (i.e. the landing end).

The system output is given by the remaining distance. According to Flight Safety Foundation, the risk of an overrun is higher if the remaining distance to the end of the runway is less than 2,500 feet when the velocity is greater or equal to 80 knots [33]. This premise is achieved with a deceleration of about 0.11 g or 1 m/s². Despite being a conservative value as the deceleration is normally higher, this value is used here to classify the system outcome as expected or not. Our landing ends at 30 knots as this is the taxiing speed, and the aircraft usually does not stop prior taxiing. A remaining distance of 2,500 feet at 80 knots is equivalent to 400 feet at 30 knots, considering the deceleration rate constant during the landing run. Thus, a remaining distance over 400 feet at the current landing end is considered acceptable (i.e. equal to 0) while a distance equal or under 400 feet characterises an event (i.e. equal to 1). The remaining distance is the binary variable of the current logistic regression.

4.2 Performance variability characterisation

The analysis continues by capturing the behaviour of each FRAM function aspects. Looking forward, the correlation between these variables gives preliminary tips regarding the system performance.

For example, height is one of the ‘To Cross the Runway Threshold’ outputs and its variability is shown in Fig. 11. The threshold is usually crossed under 50-feet AGL (95% of the flights), having a median of 26 feet. This output does not have any correlation with the FPA, indicating that the approach path is being displaced to reduce the required landing distance.

Figure 11. ‘To Cross the Runway Threshold Output’ behaviour.

Some aspects like height, TLA, brake pedal position, brake pressure and pitch attitude are directly captured from the FDR. Others must be estimated using the recorded parameters as basis (Table 1). For instance, the designated runway is intercepted by the minimum distance between the aircraft position at 50-feet AGL, provided by its linearly interpolated latitude and longitude, and its threshold, given by an external database. This database contains circa of 23,000 runways, specified by their airport ICAO Code, direction, length, aiming point position, width, altitude, threshold latitude and longitude.

Speed increment (Δv_ref) is given by actual airspeed minus v_ref, which depends on ice conditions, aircraft gross weight and flap position. Their relationship is available at the AFM in a table format.

The runway surface condition is an index created for this work and exhibited in Equation (2), measuring the interaction between the aircraft and the runway through the anti-skid actuation. This interaction depends on the runway surface conditions as well as tire pressure and condition, besides other attributes. There is a curve that relates the brake pedal position with the expected brake pressure, which is dependent on the brake control system under analysis. However, the designed pressure is usually not reached due to the anti-skid protection as shown in Fig. 13. In a good interaction, the actual brake pressure is close to the projected one. In a bad one, it is much lower. The index varies from 0 to 1, being the latter an indication of an ideal interaction.

(2) \begin{align}Runway\;Surface\;Index = \frac{{Current\;Brake\;Pressure}}{{Expected\;One}}\end{align}

The ‘Brake Application Profile’ is given by the brake pedals mean application between the aircraft wheel spin and the landing end as the brake pedal position varies during the landing run (Fig. 12). The mean comprises, indirectly, the moment that the flight crew starts the brake pedal application, the moment that they reach the maximum value and the maximum pedal value by itself.

Figure 12. An example of brake application profile.

Figure 13. Some FRAM model functions outputs.

Figure 13 displays the variabilities of some aspects of FRAM functions. An adequate use of the deceleration devices refers to a brake pedal profile near 100%. The pitch command on ground is neutral at zero.

The analysis of each output improves the understanding of the landing procedure, being used for the FRAM diagram adjustment.

4.3 Variability aggregation

The system output is given by the remaining distance, which is composed by the margins available on the designated runway, the touchdown point, the aircraft energy at the touchdown moment (i.e. the groundspeed) and the deceleration rate. The latter is affected by the runway surface condition index, the brake application profile and the pitch applied on ground.

Based on this set of predictor variables, the logistic regression was built by the glm R function (Table 2) [38]. The ‘Runway Margins’, for example, has a coefficient of −7.334201. A negative coefficient means that when the runway margins increase, the chances of a critical landing decrease (i.e. remaining distance under 400 feet at 30 knots). Likewise, it is noted that there is statistical significance at p = 0 in the use of this variable in the model, showing that it is important to the proposed regression model.

Table 2. Summary of the logistic regression

A more enriched interpretation of the relationship between predictor variables and the logistic regression output is the OR, built by the logitor R function from mfx package [42]. To illustrate, for each unit increment in the ‘Groundspeed at Touchdown’, the chances of a critical landing increase circa of 13%. Or, for each unit increment in the ‘Brake Application Profile’, the chances of a critical landing are reduced by about 49%.

To summarise and as depicted in Table 2, larger runway margins, better runway surface conditions, a more aggressive use of the brakes and a pitch closer to the neutral reduces the probability of the critical landing. In the opposite direction, longer and faster touchdowns increase this probability.

Note that the ‘Brake Application Profile’ (i.e. the brake pedals mean application between the aircraft wheel spin and the landing end) is the most relevant contributor. To increase this variable, the flight crew can increase the maximum brake pedal deflection used during the landing run or to reach this deflection earlier, maintaining it. The maximum brake application used to estimate the ULD would be represented by a mean of 100%, while the median of the analysed fleet is circa of 47% as shown in Fig. 13.

To evaluate the regression model obtained, the value of the adjusted R ² (R ² _adj) is used. This criterion is a statistical measure of how close the data are to the fitted regression line, varying between 0 (the output cannot be explained by the inputs) and 1 (the output is perfectly explained by the inputs). In general, the higher the R ², the better the model fits your data. The RsqGLM R function from package modEvA was used to estimate the R ² _adj of the current model [43], resulting in a R ² _adj of 0.2525447 in accordance with Pearson method or 0.3434947 with the Nagelkerke one.

The model comprises different types of predictor variables. The runway margins and the runway surface condition are environmental conditions from which the flight crew has a limited or even no control. The groundspeed and the pitch on ground are available and directly manageable parameters. The touchdown point and the brake application profile are also controllable, but they are a combination of other inputs from the pilots. The touchdown point can be described in terms of the height, the Δv_ref and the TLA at the runway threshold. The flare procedure and its consequent descent rate decreases the touchdown point, whose influence is mainly given by the time between the maximum pitch and the wheel spin. The brake application profile comprises the moment that the flight crew starts the brake pedal application, the moment that they reach the maximum value and the maximum pedal value by itself. Table 3 presents the variables that the flight crew is able to handle.

Table 3. Pilot handling impact over the predictor variables

¹The height is already under the standard one

4.4 Variability management

The model emphasises the impact of the selected parameters and, consequently, recognises which points of the operation should be preferably managed to reduce the RO likelihood of the analysed fleet.

It is recommended, for the sample fleet, a decrease of the aircraft speed during the approach and landing phase as well as an increase of the maximum brake pedal deflection in order to reduce the likelihood of an overrun. On other hand, the height at the runway threshold is under the recommended one. But this variability is improving the current system output and does not need to be addressed, at least because of ROs.

After identifying the points of attention of the fleet under analyses, they may be addressed in the flight crew recurring training, or by procedure reinforcement in safety meetings, or even by changes in the standard operating procedures.

5.0 Discussion and conclusions

Industry invests significant effort in the prevention of runway overruns. Addressing them efficiently requires a good understanding of how they originate and what contributes to their occurrence. Nonetheless, traditional FDM program algorithms address hazards such as long touchdowns and inadequate use of deceleration devices in an independent manner [16], without exploring the interactive effect among these precursors. The risk of ROs for a monitored fleet remains unknown.

To address this gap, models like the proposed here aim to concatenate the precursors and thus estimate the risk of an overrun. Some of them require a previous categorisation, characterising the precursors as events through the establishment of thresholds [14, 15]. Others use gross variables as inputs, focusing on their behaviour [17–20]. This work goes in the same direction as the latter but also addressing the landing run phase, not only the final approach and touchdown.

FRAM was used as a primary tool in order to identify the interactions and couplings between the various functional aspects of the system itself, especially regarding human components. Its coupling with FDM techniques made possible the quantification of the variabilities of the flight crew performance while landing an aircraft. The set of aspects chosen to compose the model are related to flight crew actions. Highlighting the variabilities that present a more significant impact in the critical landing likelihood, it gave insights regarding the practices that must be enforced or even corrected in the flight crew training. FRAM-FDM is able to identify the critical functional coupling like Adriaensen et al. [22] and Patriarca et al. [23], but not through the number of downstream dependencies for each function and thus by each function variabilities measure. It goes in the same direction as Oliveira et al. [21] and Patriarca et al. [24], specifically using flight data as a source of information to estimate the flight crew handling.

Besides the model to assess the operational risk of a sample fleet in overrunning the runway during landing, a quantitative evolution of FRAM was proposed. Despite being applied to runway overruns in the landing phase, it remains, as a proposal for future study, the application of FRAM-FDM to analyse the risk of other unsafe events on aviation, such as hard landings or runway veer-offs.