Insurance analytics: prediction, explainability, and fairness

Abstract

The expanding application of advanced analytics in insurance has generated numerous opportunities, such as more accurate predictive modeling powered by machine learning and artificial intelligence (AI) methods, the utilization of novel and unstructured datasets, and the automation of key operations. Significant advances in these areas are being made through novel applications and adaptations of predictive modeling techniques for insurance purposes, while, concurrently, rapid advances in machine learning methods are being made outside of the insurance sector. However, these innovations also bring substantial challenges, particularly around the transparency, explanation, and fairness of complex algorithmic models and the economic and societal impacts of their adoption in decision-making. As insurance is a highly regulated industry, models may be required by regulators to be explainable, in order to enable analysis of the basis for decision making. Due to the societal importance of insurance, significant attention is being paid to ensuring that insurance models do not discriminate unfairly. In this special issue, we feature papers that explore key issues in insurance analytics, focusing on prediction, explainability, and fairness.

1. Introduction

Actuaries have been using econometric and statistical models for decades. And just as statistical learning has fundamentally changed the way predictive models are built, actuaries have had to adapt to these new techniques. Neural networks, a concept rooted in the 1940s and inspired by the structure of the human brain, have exploded in recent decades with the advent of massive data, enabling increasingly sophisticated architectures and capturing more complex effects. This progress has made it possible to implement the universal approximation theorem, which had previously existed only in theory. The 2024 Nobel Prize in Physics, awarded to Hopfield and Hinton, underscores their pivotal role in this revolution.

But the arrival of these artificial intelligence (AI)/machine learning models has not been without its problems. Breiman (2001) spoke of a cultural difference between data modelers and algorithmic modelers. But the difference is more profound. Econometric and statistical models are deeply probabilistic, whereas learning algorithms are not. In a Support Vector Machine (SVM), we try to place a separating plane in a cloud of points, based on distance, and if this allows us to separate images of dogs and cats or individuals who are sick and others who are not, the question of the probability of belonging in a given group rarely arises. Yet it is this quantity that is essential for actuaries, in order to construct a tariff. The actuary is not trying to predict who will die in a life insurance portfolio, but to estimate, as accurately as possible, the probability of death for each individual. Recent advances in insurance predictive analytics have brought new challenges, such as handling high-cardinality features, incorporating Poisson and Tweedie loss functions into machine learning models, and enforcing smoothness and monotonicity constraints.

As a highly regulated industry, insurance often requires models to be explainable, enabling regulators and stakeholders to understand the basis for decision-making. However, emerging machine learning and AI models is usually too complex and opaque to meet these explainability standards. This has created a need for new models and techniques that can harness the predictive power of these black-box models while maintaining the transparency and interpretability that insurance demands.

Issues of discrimination and fairness in insurance have long been debated. Yet, AI and Big Data have added layers of complexity, as opaque algorithms and proxy discrimination introduce new concerns. Addressing these challenges requires multiperspective and cross-disciplinary collaboration. Importantly, even to start addressing these challenges, interpretability and explainability is a fundamental prerequisite.

To encourage further research in this area and support recent innovations, the Annals of Actuarial Science (AAS) has launched a special issue titled “Insurance Analytics: Prediction, Explainability, and Fairness.”

2. Predictive analytics in insurance

Statistical models have long been used in insurance, but their use raises profound epistemological questions. Von Mises (1939) explained that the “probability of death” applies to a group or class of individuals, not to any single person, as it has no meaning when referring to an individual, even with detailed knowledge of their life and health. In the frequentist approach, probabilities are constructed as asymptotic limits of frequencies, grounded in the law of large numbers. By reasoning in terms of “homogeneous risk classes,” actuaries historically relied on robust statistical techniques, both mathematically and philosophically, to rate policyholders. However, modern machine learning techniques now allow for pricing individual risks and personalizing premiums with increasing granularity. This shift introduces new challenges when applying advanced analytics in insurance, including managing high-cardinality features, classifying policyholders into unique subgroups, and incorporating Poisson and Tweedie deviance loss functions in boosting and tree-based methods.

In this special issue, we present four papers that focus on predictive analytics in insurance.

Campo & Antonio (2024) proposed the data-driven Partitioning Hierarchical Risk-factors Adaptive Top-down (PHRAT) algorithm to reduce hierarchically structured risk factors to their essence by grouping similar categories at each level. They also utilize embeddings to encode textual descriptions of economic activities, aiding in the grouping of categories for inputs.

Lee & Jeong (2024) modified the alternating direction method of multipliers (ADMM) for subgroup analysis to classify policyholders into unique groups. They interpret the credibility problem using both random effects and fixed effects, which correspond to the ADMNM approach and the classic Bayesian approach, respectively.

Willame et al. (2024) reviewed the use of boosting under the Poisson deviance loss function and log-link (following Wüthrich & Buser, 2019) and apply boosting with cost-complexity pruned trees on Tweedie responses (following Huyghe et al., 2022). They introduced a new Boosting Trees package in R designed for insurance applications.

Wu et al. (2024) extended the traditional Lee-Carter model using Kernel Principal Component Analysis (KPCA) to enhance mortality rate predictions. They demonstrated the robustness of this model, particularly during the COVID-19 pandemic, showing its superior performance in volatile conditions.

3. Explainability and interpretability

Insurance, as a high-stakes business, faces stringent regulatory requirements, particularly regarding explainability and interoperability. Traditional statistical models, such as Generalized Linear Models (GLMs) and Generalized Additive Models (GAMs), are typically more interpretable than modern machine learning models like Gradient Boosting Machines, Random Forests, or Neural Networks. While a range of interpretability tools have been developed to increase transparency in these black-box models, they have not been without criticism (Hooker et al., 2021; Rudin, 2019; Xin et al., 2024). Balancing regulatory demands for explainability with the use of advanced machine learning models has become a pressing challenge for the insurance industry. Recent literature emphasizes the growing importance of model interpretation and transparency in this field, as highlighted by Aas et al. (2021), Delcaillau et al. (2022), and Richman & Wüthrich (2023).

In this special issue, we present four papers that focus on explainability and interpretability in insurance analytics:

Jose et al. (2024) developed a zero-inflated Poisson neural network (ZIPNN) by following the combined actuarial neural network (CANN) approach to model admission rates. They extend this with zero-inflated combined actuarial neural network (ZIPCANN) models and adopt the LocalGLMnet method (Richman & Wüthrich, 2023) to interpret the models.

Lindholm & Palmquist (2024) proposed a method for constructing categorical GLMs guided by information derived from a black-box predictor. They use partial dependance (PD) functions to create covariate partitions based on the black-box predictor, followed by an auto-calibration step and a lasso-penalized GLM fitting.

Maillart & Robert (2024) explored an approach to estimate a GAM with non-smooth feature functions. This method distills knowledge from an Additive Tree model, partitions the covariate space, and fits a GLM using binned covariates for each decision tree, followed by an ensemble approach and final GLM fitting for auto-calibration.

Richman & Wüthrich (2024) introduced ICEnet, a method that enforces smoothness and monotonicity constraints in deep neural networks. To train neural networks with these constraints, they augment datasets to produce pseudo-data that reflect the desired properties. A joint loss function is used to balance accurate predictions with constraint enforcement.

4. (Algorithmic) fairness

As in other industries, insurers are redefining their practices with the rise of Big Data and advanced AI algorithms, enabling them to detect previously unknown patterns, incorporate more rating factors, improve predictive accuracy, and move toward more granular risk classification. While these technologies expand the scope of what is possible, they do not fundamentally change the longstanding issues of insurance discrimination. In fact, in this rapidly evolving landscape, old challenges are becoming more pronounced. Concerns about indirect discrimination and the use of algorithmic proxies are growing, as insurers increasingly leverage vast datasets and sophisticated models.

Avraham (2017) argued that insurance faces unique moral and legal challenges. While policymakers seek to prevent discrimination based on factors like race, gender, and age, the insurance business inherently involves distinguishing between risky and non-risky insureds, which often correlates with those sensitive characteristics. Actuaries must remain vigilant to these issues and actively contribute to solutions that mitigate the risks of discrimination.

Recent research has begun addressing these issues for insurance applications from multiple perspectives (such as ethical, actuarial, statistical, economic, and legal perspectives), with contributions from Prince & Schwarcz (2019), Baumann & Loi (2023), Lindholm et al. (2022, 2024), Frees & Huang (2021), Xin & Huang (2023), Barry & Charpentier (2023), Charpentier (2024), Araiza Iturria et al. (2024), and Fahrenwaldt et al. (2024), as examples.

We encourage more contributions to this crucial area of research, addressing the ongoing challenges of discrimination and fairness in insurance via multidisciplinary research and collaborations.

5. Conclusion

The integration of advanced analytics, machine learning, and AI into the insurance industry has presented significant opportunities to enhance predictive accuracy, streamline operations, and deliver more personalized services. However, with these advances come complex challenges, particularly in maintaining the explainability and fairness of increasingly opaque models. As insurers adopt these powerful tools, the responsibility to ensure ethical and responsible use becomes even more critical.

The papers in this special issue of the Annals of Actuarial Science highlight cutting-edge approaches to prediction and explainability in insurance analytics. They collectively demonstrate the potential of advanced methods to address industry challenges, but they also emphasize the need for further research to reconcile the power of these models with business, regulatory, and ethical considerations. We encourage continued research contributions to this critical area.