The vast majority of risk models used by financial institutions uses historical data to characterize the distribution of future, potential events. The ubiquitous use of risk models in banking and insurance is understandable: risk models in principle allow us to learn valuable lessons from the past to take better decisions in the future by leveraging historical data. For example, underwriting risk models such as credit scoring models are used to improve profitability by influencing underwriting decisions including whether to accept a credit or insurance application and at what price. In addition, internal models play a critical role in determining capital requirements under risk-based capital regimes such as Solvency II for European insurers and Basel III for banks globally.
Unfortunately, the naïve use of data-driven risk models is often inappropriate if the past proves to be an inaccurate guide to the future or if there is not an historical precedent, circumstances that are especially relevant in rapidly changing environments that exhibit new and complex interdependencies such as we observe today. In such instances, complementary approaches to risk identification and risk evaluation also need to be used.
This article presents complementary approaches to risk modelling which are robust to these challenges and suitable for the interconnected geopolitical, social and economic risks which we face today. The approaches, the Top Risk Assessment and Scenario Analysis approaches, are derived from best practices in the insurance industry. The remainder of this article is organized as follows: a) the first paragraph outlines why traditional, data-driven risk modelling approaches used by financial institutions often prove to be insufficient given the increasing complexity and interconnectedness of global risk scenarios; b) the second paragraph presents a generic, judgement-based risk identification and evaluation framework which can be used to complement traditional risk modelling; c) the third provides a concrete example of the framework in the form of the Top Risk Assessment; d) the fourth paragraph provides a second example in the form a Scenario Analysis using the 2011-12 European sovereign crisis as an example.
The Failure of Data-driven Risk Models in a Complex, Interconnected World
On risk modelling generically
Data-driven risk modelling is used in many ways by financial services firms to improve decisions. For example, historical loan default data is used to support loan underwriting decisions by predicting the probability of default of a new applicant, often summarized by an applicant’s credit score; similar statistical underwriting support tools are used by insurers when accepting insurance applications and setting rates. Similarly, large amounts of historical market data are used by banks and insurers to understand the range of potential outcomes for an investment or trading portfolio and to determine the amount of economic capital needed to support the portfolio.
Fig. 1. The role of historical data in calculating value at risk
One example of an important data-driven risk models can be found in fig. 1, which illustrates how historical data is used to characterize the probability of worst-case future events; this measure, often referred to as value at risk, is often used as the basis for risk-based regulatory capital requirements. As the exhibit illustrates (fig. 1), historical data is either used “raw” in order to simulate future outcomes (“historical simulation”) or parameterized into an assumed statistical distribution which can be used to calculate the loss profile directly (“parametric”) or by sampling (“Monte Carlo simulation”). The illustration is based on financial market returns over time which can go up or down daily; for underwriting models, the historical data would cover loan defaults or insurance claims. In any case, the “events” are interpreted through the financial lens of the new application or investment portfolio being considered.
Tab. 1. WEF Global Risks 2015
Asset bubble in a major economy
Extreme weather events (e.g. floods, storms, etc.)
Failure of national governance (e.g. corruption, illicit trade, organized crime, impunity, political deadlock, etc.)
Deflation in a major economy
Failure of climate-change adaptation
Interstate conflict with regional consequences
Energy price shock to the global economy
Major biodiversity loss and ecosystem collapse (land or ocean)
Large-scale terrorist attacks
Failure of a major financial mechanism or institution
Major natural catastrophes (e.g. earthquake, tsunami, volcanic eruption, geomagnetic storms)
State collapse or crisis (e.g. civil war, military coup, failed states, etc.)
Failure/shortfall of critical infrastructure
Man-made environmental catastrophes (e.g. oil spill, radioactive contamination, etc.)
Weapons of mass destruction
Fiscal crisis in key economies
High structural un-/under-employment
Failure of urban planning
Breakdown of critical information infrastructure and networks
Large-scale involuntary migration
Large-scale cyber attacks
Profound social instability
Massive incident of data fraud/theft
Rapid and massive spread of infectious disease
Massive and widespread misuse of technologies (e.g. 3D printing, artificial intelligence, geo-engineering, synthetic biology, etc.)
Source: Global Risks 2015, World Economic Forum.
The three critical premises underlying data-driven risk models are that: a) past developments are a good representation of future uncertainty; b) there is sufficient past data available to characterize the uncertainty; c) there is a direct and predictable link between the modelled events and their impact on measures of interest.
Challenges in an increasingly complex, globally interconnected world
These three premises fail demonstrably when considering many of the major geopolitical, social and technological risks in our rapidly evolving, increasingly interconnected world. Consider as an example the top risks identified by the World Economic Forum (WEF 2015) based on a global survey of industry professionals, academics, politicians, etc. The WEF defines a “global risk” as an uncertain event or condition that, if it occurs, can cause significant negative impact for several countries or industries within the next 10 years.
The WEF’s Global Risks 2015 report lists 28 global risks, grouped into the five categories (economic, environmental, geopolitical, societal and technological risks), which were considered the most important by the survey respondents based on an assessment of likelihood and potential severity (tab. 1).
A cursory examination of the table is sufficient to conclude that the WEF’s top risks are not amenable to traditional, data-driven risk modelling. More specifically, the past is neither a good representation of future uncertainty, nor is there always sufficient data to characterize the future uncertainty since many of the risk events are triggered by developments never seen before. More specifically, it is impossible to collect historical data for such recent, game-changing developments as, e.g. climate change; a reliance on critical information infrastructure; new technologies and applications such as genetic engineering, nanotechnology, 3D printing, artificial intelligence; the collapse of colonies; etc.
In addition to an overall assessment of frequency and severity, the WEF survey also made transparent the postulated interdependencies connecting the global risks, illustrated in fig. 2.
Fig. 2. Global risks and interdependencies 2015
Source: Global Risks 2015, World Economic Forum.
A cursory examination of this interconnectedness indicates that these risks are not amenable to traditional risk modelling also because there is not a direct and predictable link between the modelled events and the impact on variables of interest.
Tab. 2. Known, unknown and unknowable risk management framework
Distribution of potential losses is well understood in terms of the drivers of both frequency and severity. Examples: credit card defaults or automobile insurance claims.
Data-Driven Risk Modelling Approaches
Potential events can be imagined but there is difficulty in assessing the probability of occurrence or severity. Examples: systemic crisis, terrorist attacks.
Judgement-Based Risk Evaluation Approaches
Events cannot even be imagined, although once they occur they enter into the realm of the unknown. Example, cyber attack in the 1950s.
Judgement-based Approaches for Unknown Risks
How to address the potential challenges to traditional, data-driven risk identification and evaluation approaches? Taking a step back, it is useful to define three categories of risks – known, unknown and unknowable risks, illustrated in the table 2. Looking at the table, it is clear that each of these categories requires a unique risk identification, evaluation and management approach.
The first category, “known” risks, are those whose frequency and severity can in principle be characterized by historical data; as a consequence, data-driven risk modelling approaches are primarily used for risk identification and evaluation of “known” risks.
The third category, “unknowable” risks, by definition falls outside of the realm of human imagination; because of this, they can only be effectively managed by building a robust and resilient organization.
Of particular interest in the present context is the second category, “unknown” risks. While the possible events and severity of “unknown” risks can be imagined and described by scenarios, they cannot be characterized by using historical data and traditional risk modelling techniques. This category represents the vast majority of the WEF global risks, the most important risks which are of global concern in an increasingly complex and interconnected world. In contrast to the data-driven approaches used for “known” risks, the only way to identify and evaluate “unknown” risks is to use judgement-based approaches.
There are two generic, judgement-based approaches useful for identifying and evaluating the global risks stemming from complex geopolitical, technological, social and environmental sources – the Top Risk Assessment and Scenario Analysis approaches. Both are based broadly on the COSO (2004) framework for risk identification and evaluation and both are described in the following Sections.
The Top Risk Assessment (TRA)
The WEF Global Risks Report 2015 is an example of a judgement-based Top Risk Assessment approach, albeit one where the “brainstorming” was supplemented by a broader survey to assess likelihoods and severities. Most banks and insurers have a formalized process called the Top Risk Assessment to assess their most important risk scenarios and exposures. The Top Risk Assessment (TRA) used by banks and insurers is described below at a high level.
Fig. 3. Top Risk Summary Report
Objectives, Scope, and Output
The TRA identifies and prioritizes the firm’s top risks and assigns “ownership” to the most senior level for remediation. This process covers known and unknown risks as well as modelled risks and risk scenarios, including financial market, credit, insurance, operational, reputational, business and regulatory risks (the most important risk categories for banks and insurers). It is applied across the businesses and at the group level.
The output includes a description of the firm’s top risks, a conscious comparison against the firm’s risk appetite and, finally, the definition and assignment of the risk mitigation activities to senior business leaders, as summarized in fig. 3. Backing up this overview are details regarding the risk scenarios and remediation plans.
As illustrated in fig. 3, it may be desirable to separate the top risks into three categories, each with its own presentation: 1) those risks with a sudden, immediate economic impact (e.g. global recession, systemic failures, large exposures, etc.); 2) those risks which predominantly impact the firm’s reputation and franchise value, without an immediate balance sheet or solvency impact (e.g. data security breaches, sales practices issues, etc.); 3) those strategic risks which have a longer-term opportunity cost if not appropriately addressed (e.g. technological changes impacting distribution paradigms; a failure to reduce expenses; demographic changes such as longevity, middle class in developing economies, mega-cities, etc.).
The TRA is often an annual, bottom-up process, building on the detailed analysis within each business and operational area. The scenarios and remediation activities are reviewed on a quarterly basis. The analysis involves four steps: risk identification, risk evaluation, risk mitigation and continual review: 1) risk identification is done through formal brainstorming sessions under the following ground rules: a) gather experts from a variety of disciplines and experiences, both internal and external, in order to broaden the insight pool and allow for greater association. Often involved in the discussions are individuals covering all aspects of the firm, including external academic and economic experts; internal resources from the strategy, economics, legal, compliance, risk, treasury and investment departments, operations and administration departments, etc.; b) encourage all to speak openly, introduce ideas and draw free associations.
Provide positive incentives to throw out ideas, even those which may at first seem bizarre or redundant, and do not criticize. Collect and record all the ideas; c) launch the conversation with familiar anchor points and then branch out through free association. For example, anchor points may include historical precedents from both internal and external loss experience, in the industry and in other industries; d) approach the problem from several different directions. This may yield similar results, but occasionally will lead to important incremental insights. For example: i) look at possible chains of events which might lead from A to B to C (e.g. the implication of sovereign debt restructuring on the balance sheets of banks and the consequences for derivative counterparty exposures); ii) look at large, material “things” and ask what might make them vulnerable (e.g. if 50 per cent of your business is concentrated in one product or one country or one customer, ask what would make the concentration vulnerable); iii) look at any large positions which have a low modelled risk profile, asking what assumptions the model makes that can go wrong; 2) risk evaluation places each of the identified risks in a two-dimensional matrix as illustrated in fig. 3, the first dimension representing the frequency and the second the potential severity. Typically, risk evaluation is done in parallel with risk identification, allowing the expert panel to jointly discuss scenarios, consequences and likelihoods and to iterate to a consensus; 3) risk mitigation.
As mentioned, “ownership” of the top risk scenarios is assigned to a senior member of the firm. In this case, “ownership” includes: the development of a remediation plan (if the risk is outside of the firm’s risk appetite), contingency planning (in case the situation should deteriorate) and on-going monitoring activities; 4) continual re-evaluation. By necessity, risk identification is an on-going process, both because the world is changing and because we learn, allowing us to make new associations and draw new conclusions.
As mentioned, the WEF Global Risks Report is an example of a TRA. The difference between the WEF report and the TRA described above is that the first two steps, identification and evaluation, were conducted by survey as opposed to brainstorming sessions, and the third step, mitigation, is understandably absent.
Three technical decisions need to be taken to set up the TRA. The first is the categorization of event probabilities and severities. It is useful to set ranges rather than use point estimates, for example, event frequencies in 1-in-5, 1-in-10, 1-in-50 and 1-in-200-year buckets. In addition, a decision has to be made on how to characterize severities (e.g. based on mark-to-market or accounting terms, relative to net income or solvency).
The second relates to reporting of the risks on an as-is evaluation (the level of risk today, including existing controls and management actions) or on both an as-is and an inherent basis (the risk without current compensating controls or risk mitigation). Inherent risk provides a stable baseline against which management actions can be evaluated; the drawback is that it is artificial. For example, do we evaluate a property theft scenario under the (arguably ludicrous) scenario that there are no locks on any of the windows or doors, without a security system or security guards?
Finally, how to coordinate with other, related functions? Similar processes are often run by internal audit, legal or compliance. It is beneficial to coordinate the activities and definitions as far as possible in order to eliminate confusion and the duplication of work.
As opposed to best-estimate forecasting or the single-step Top Risk Assessment, the end product of Scenario Analysis is a set of possible future events or “states of the world” and, in the case of scenario trees, the logical path leading to those states. Scenario Analysis is a valuable tool for going deeper into a specific risk scenario and leads to two primary benefits compared to the TRA: 1) it provides a better understanding and description of “unknown” risks arising from complex, interconnected events which can take many branches in a scenario tree; 2) it provides a better basis to define risk mitigation and more articulated contingency plans, e.g. also along the scenario tree.
Scenario Analysis proceeds generically in three steps, illustrated in the following by using the 2012 European sovereign debt crisis as a practical example.
Define Event Space and Factors to be Considered.
In the case of the euro sovereign crisis, in early 2011 Allianz pulled together experienced professionals from the risk, investment, economics, treasury and legal departments as well as business professionals from the affected local markets. They were sequestered in a workshop with the objective of developing a scenario tree. Materials were distributed before the workshop, including current news, economic research and a synopsis of previous historical events of similar nature; also distributed was an overview of the portfolio to trigger connections from both directions. The workshop was facilitated, but it was done with a “light touch”, allowing the conversation to range widely before reining in the discussion. The scenario tree that emerged to describe the possible events remained relatively stable during 2011-12: a) “muddle through”, characterized by financial market volatility but with growth sufficient to correct the underlying fundamentals; b) “escalation”, marked by a crisis in confidence triggering a flight to quality, a precipitous decline in asset values and risk-free rates due to aggressive monetary intervention; c) following “escalation”, either i) an eventual step “Back from the cliff”, returning to the “muddle through” scenario (where we still seem to be); ii) a “United States of Europe”, characterized by a stronger and more credible fiscal union addressing the underlying fundamentals; iii) “partial or full break-up of the euro.”
While these scenarios could potentially be mapped into the more traditional “best-case”, “baseline” and “worst-case” scenarios, we found it useful to retain the labels as they were more descriptive. Underlying each scenario were more specific and detailed steps in the scenario tree. For example, we considered different possible sequences of affected countries; restructuring versus exit scenarios; possible actions of regulatory forbearance; possible legal scenarios with regard to contract redenomination, etc.
Analyse Impact and Alternatives
The next step analysed the impact of the scenarios. In the case of the euro crisis, Allianz focused primarily on the first-order effects on economic and IFRS balance sheets as well as regulatory solvency ratios; however, we also evaluated likely competitor behaviour for both new business and M&A opportunities.
In addition, different management alternatives were analysed, for example shifting assets to safer counterparties or jurisdictions, redrafting contracts to explicitly address potential redenomination risks, enhancing systems to include multi-currency capabilities, etc.
The final step was to discuss and decide on management actions. In order to do so, some statements about management preferences needed to be made. The high-level goals which Allianz committed to during the crisis were: 1) the group’s ability to withstand up to a 50 per cent haircut on peripheral sovereign bonds and the likely devaluation of other risk assets in sympathy; 2) the local entities’ ability to continue to write profitable business from a operational systems and balance sheet perspective.
Meeting these objectives required Allianz to take immediate action, reducing fixed-income exposures to peripheral sovereign issuers and banks, reducing equity exposures, increasing group liquidity reserves and adapting local administration systems.
Common Challenges in Scenario Analysis During a Crisis
Scenario analysis is a useful risk tool, providing a structured approach for eliciting, interpreting and consolidating expert judgment on complex issues. However, it also has some limitations: a) expert judgment will be wrong, possibly in the headlines and certainly in the details, during dynamically evolving crisis situations; b) it can be difficult to converge to a few scenarios. If you ask 100 experts, you will get 100 different answers. Scenario Analysis multiplies this by a factor of 20, with experts disagreeing on whether it is a 1 per cent or a 5 per cent probability or whether there shouldn’t be a fork in the tree with a different commodity price development, etc.; c) it is difficult to take events seriously which fall outside of our comfort and experience zone. For example, during the first months of 2011, many pundits discounted the probability of a collapse of the euro, even though there is no immutable, physical law holding the euro together; d) analysis is easy; taking action is difficult even with consensus. Put simply, Scenario Analysis is not a guarantee of effective decision taking and the execution of those decisions.
Mitigating the Issues
Scenario Analysis is a means to an end, and not the end in and of itself; a good Scenario Analysis is necessary, but not sufficient, to effectively steer through a crisis. More important is taking the right decisions and executing them consequently. This is too often forgotten, leading to more analysis and less management action. Nonetheless, there are some “tricks” which can support effective Scenario Analysis: 1) consensus on details is not necessary; consensus on the “headlines” is. Although 100 people will have 100 opinions, some consensus is necessary in order to progress to the next step – management action. To resolve this paradox: a) recognize that it is better to get consensus around something directionally correct (such as “things are likely to get much worse with a high probability”) and to act decisively than to reach consensus in all details with no action at all; b) choose scenarios which “tell a story” – the scenarios we chose (“muddle through”, “escalation”, “back from the cliff”, “United States of Europe” and “breakup”) conveyed an intuitive message that was easily understood by all; c) focus on ranges, not spot estimates – it doesn’t matter whether it is a 15 per cent or a 25 per cent probability that equity markets will plummet.
What does matter is that it is a real possibility; d) focus on the first-order effects – as a European insurer it was important to focus on interest rates, bond spreads, equity markets and consumer demand for liquidity; not as important was the impact on gold or oil prices or a host of other variables which have only second-order impact; e) recognize diminishing returns to analysis. Your scenario will be wrong in the details with absolute certainty; the key is to make sure that it is directionally correct on the major points and build organizational resilience for the remaining uncertainty; 2) be prepared to take decisions.
The objective is to take the right decisions and to execute them effectively. Decision taking in large organizations is inherently complex even in normal times; taking decisions under accelerated time lines and difficult circumstances even more so. The following are some things which support better decision making during a crisis: a) avoid analysis paralysis; refresh the scenarios only periodically. While a detailed analysis at regular intervals feels more “structured”, the reality is that it occupies your best resources when they could be better used actually managing the crisis. Working with ranges gives some leeway, as does regularly reviewing but not refreshing the analysis unless there are material changes; b) adjust your behaviour and expectations.
By definition, crises are times of significant uncertainty; taking large bets during such periods may lead to great rewards, but it can also lead to ruin. Examples include John Corizine’s exaggerated bet on the Eurozone and its disastrous results for MF Global (Protess 2012) as well as the cost paid by RBS, Bank of America (Duhigg 2008; Story and Dash 2009) and others for transformational M&A deals during the 2008 financial crisis; c) don’t take decisions based on the mean. In normal times, decisions can be taken based on the best estimate, a natural inclination when worst-case scenarios are distant. However, more weight needs to be put on the tails during a crisis.
This can be done intuitively or mechanically, for example by making an explicit trade-off between the (scenario-weighted) expected value and some measure of risk such as the scenario-weighted volatility or the distance between best and worst cases; d) take the actions which can be taken, not the ones you would like to take but cannot. Some actions represent immediate, no-regret moves, for example shifting deposits to safer banks in less affected countries, raising liquidity, etc. Others may be desired but not possible due to reduced market liquidity, for example selling subordinated bank debt or illiquid alternative assets from the affected countries. If your desired move is stymied due to market conditions, consider actions further afield: the sale of senior debt of banks in the rest of Europe may be the best that you can do to minimize exposure to a systemic financial sector event; e) contingency planning to act when required. Because of the accelerated time lines and high uncertainty, opportunities come and go rapidly, leaving the ill-prepared behind. Preparing the organization is critical and contingency planning can help by building an appropriate sense of urgency and consensus beforehand.
However, there is a caveat on contingency planning: There can be situations where triggers are met, but the anticipated actions cannot be taken. Consider the hypothetical rule to “sell Cedulas or European bank subordinated debt if markets deteriorate to a specific point”. In a declining market, you are likely to find yourself standing in a long queue of sellers with no one buying. If your contingency plans are to provide more than illusory comfort, think critically whether the actions can be taken in the scenario which triggers them; it may be more reasonable to sell into a situation of temporary euphoria; f) alter traditional decision channels. During normal times, decisions may be taken by committees which do not react generally quickly. Delegate authorities to individuals or to a Crisis Task Force, on call 24/7, to execute contingent actions; g) train for and test the contingency plan, potentially under “surprise conditions”, to ensure that actions become second nature and are effective. For example, at Allianz we conducted “war games”, having the Crisis Task Force react to a default scenario over the weekend and evaluating their preparedness via Monday morning conference calls; h) set the right incentives. Selling an asset into a soft bid will likely realize a loss relative to its current carrying value.
This provides a disincentive to sell if bonus targets will be missed with certainty, whereas there might still be a chance to reach targets if the position is left open. Similarly, there may be an incentive to continue to write new business even if of questionable value due to changing market conditions. The solution is to adjust targets, either implicitly or explicitly, for example by setting up an operating profit or net income “budget” to be used by the businesses to close risky positions.
The views expressed in this article are those of the author and do not necessarily reflect those of Allianz.
 Value at risk is often defined as the worst-case loss within a pre-defined confidence interval (e.g. 1 per cent, 0,5 per cent, etc.) over a pre-defined time horizon (e.g. 10 trading days, 250 working days, etc.). See Wilson (1998) for an overview.
 In addition to these criteria, a further premise (or assumption) is that the model used for characterizing the future also has to be appropriate both in representing the risks and for the purpose for which it is used.
 The following summarizes the risk measurement and management framework described in Wilson (2015).
 See Diebold et al. (2010).
 See also Wilson (2015) for a discussion on how to build a resilient organization.
 The following is based on Wilson (2015; 2013a; 2013b).