Models get a thrashing

Was it all in vain? More than 200 credit experts and their regulators spent two days deep underground last month thrashing out the virtues and vices of credit risk modelling. At the end of this session, two floors down in The Pit cinema at London’s Barbican Centre, a panel including a trio of European regulators concluded that the models were “half-baked” and could not be used to set regulatory capital.

The financial risk industry is not discouraged. The fact that there was a conference at all, organized by the Bank of England and the UK’s Financial Services Authority (FSA), “shows they’re taking us seriously, which they wouldn’t have done a year ago”, says one industry source.

The modellers’ interim goal, besides the holy grail of modelling credit risk perfectly, is to persuade regulators to modify or replace the crude credit risk ratios imposed by the 1988 Basle Accord on capital adequacy. The accord demands an across-the-board capital to risk assets ratio of 8%, which is acknowledged to have forced most banks to stock up on capital – good. But the risk weightings are absurd for today’s conditions – for example they are five times more favourable to an OECD bank, however poorly rated, than to a non-bank company, even if it’s triple-A-rated – bad. Regulators acknowledge that this needs to be fixed, but they’re not convinced that credit risk models are the way to go.

The modellers would like regulators to apply the principle – which the regulators already accepted last year in the case of market risk – that banks’ internal risk models should be used to help determine a level of regulatory capital for credit risk. That would allow banks to benefit from the portfolio effect of diversified credit risk which the models are designed to identify. And it would allow them to reduce their regulatory capital accordingly. It would also better identify concentrations of risk and in some areas indicate that an increase in regulatory capital might be required, the modellers hasten to point out. So the aim is not less prudence, but a closer fit between what banks calculate they need as economic capital and the capital requirement set by regulators.

But there is heavy scepticism among the European regulators, who have a big vote in the Basle committee on banking supervision. In fact it needs a unanimous vote to change the Basle capital accords, although ironically those accords carry no statutory weight: they are only recommended minimum standards which bank supervisors around the world can accept, or reject, or strengthen. The UK authorities, for example, have more stringent standards. “No bank in the UK has its regulatory capital set at 8%, it’s always more,” says Oliver Page, in charge of complex group supervision at the FSA. To confuse matters, EU banks must comply with the EU capital adequacy directive (CAD), which partly conflicts with the Basle capital standards.

Convincing the regulators

The modellers know they have a battle to persuade the regulators of their case. Continental European regulators are particularly sceptical. US and UK regulators are regarded as more friendly and the US Federal Reserve Board and the Bank of England have done extensive comparative studies of the better-known credit risk models.

The Barbican conference mostly examined the robustness of credit risk models and whether they could be used, beyond risk management, to set regulatory capital.

The four best-known models on offer are CreditMetrics (developed by JP Morgan), CreditRisk+ (Credit Suisse), KMV (KMV Corporation), and CreditPortfolioView (McKinsey). Each has its strengths and weaknesses – if not downright flaws – as several speakers pointed out.

A credit risk model is designed to predict the probability of default or a rating downgrade, or both, and the likely recovery of assets after a default or bankruptcy. The model does this either for portfolios of credits averaged according to rating bands or by individual obligor based on its asset value and price volatility. CreditMetrics, the earliest popular model, is based on historical analysis of transitions from one credit rating to another (including the transition to default). It makes several assumptions: that credits within the same rating band have the same risk, although it can vary by business sector; that interest rate changes have no impact on credit risk; that equity price is a good proxy for asset value; and that macroeconomic factors such as unemployment and GDP growth have no impact on credit risk. This means that CreditMetrics can “allow one to identify trading opportunities in the loan/bond portfolio, where concentration and overall risk can be reduced without affecting expected profits”, says Robert Mark executive vice-president for treasury and risk management at CIBC in Toronto. It can’t handle the credit risk of collateralized loan or mortgage obligations, or derivatives such as options, cross-currency swaps or other non-linear products. (CIBC has an internal model CreditVaR I closely based on CreditMetrics which makes the same assumptions. It is refining a model CreditVaR II which should handle derivatives.)

CreditRisk+ is based on actuarial calculation of expected default rates and expected losses from default. Like CreditMetrics it assumes there is no market risk, and it gives no significance to changes in credit rating. It doesn’t deal with non-linear products such as options and cross-currency swaps. Nor does it take account of macro-economic changes.

KMV‘s model is based on calculating the expected default frequency (EDF) for each company based on its current asset value and the volatility of its equity returns. Unlike the two previous models the formulas used are not publicly available. It works best for publicly traded companies whose equity value is easily determined. According to the model a company will default when its asset value falls below the value of its liabilities. The default tends to occur “when the asset value is somewhere between total liabilities and the value of the short-term debt”, says Mark in a paper delivered at the conference. Mark complains that because some details are proprietary there is “no way to check the accuracy of the estimates”. The model also assumes no impact from market risk and implies that the credit spread on a bond tends towards zero at maturity.

CreditPortfolioView comes at the problem from almost the opposite direction, positing that the probability of default is a factor of macroeconomic variables, such as unemployment, the level of long and short-term interest rates, GDP growth, government expenditure, the aggregate savings rate, and exchange rates. The model uses default data going back 20 years, where available, from many countries and from business sectors within each country, matching that to business cycles. The argument is that these variables tend to affect the aggregate number of business failures. This model, far from ignoring market risk, is practically driven by it. CreditPortfolioView may be more flexible than its three forerunners, but to be accurate it will need a huge amount of reliable and well-calibrated data.

Too crude, too narrow

And this is the snag, as far as the regulators are concerned. Most of the models on the market tend to extrapolate data and observations from the world’s best-developed markets – predominantly the US bond and equity markets – and apply them to markets where the parameters may be completely different. For example, default rates and loss recovery from US companies, with the benefit of Chapter 11, are likely to be very different from the aggregate default and loss recovery rate in Japan, Germany or Italy. Yet these models aren’t sensitive to the right data, or don’t have enough of it, to apply their technology in more than, at best, a handful of countries. Another potential weakness is the reliance of some models on public ratings, which tend to lag market information.

During the conference it became clear that considerable work must be done on broadening and calibrating databases, and that, if banks and other sources cooperate, there are many gold mines of data lying unused. Westdeutsche Landesbank boasts a 12-year database of default and recovery rates among its big and medium-size customers, mostly in Germany but also around the world. Using actuarial techniques and the Poisson distribution equations also used by CreditRisk+ it is able to generate probabilities that have a basis in individual companies. The Banque de France has its centrale des risques, a database of bank loan exposures to French borrowers, collected since 1948 with records of default; the Bundesbank has a similar Evidenzzentrale recording any bank loan over Dm3 million ($1.8 million). But the Evidenzzentrale has no records of default. A better database, say Bundesbank sources, is the records of commercial bills that German banks discount with the Bundesbank, recording the company name and history of creditworthiness. Such databases may be useful if pooled and calibrated across countries. But sceptics point out that next year the landscape in Europe if not the world will change radically with the introduction of the euro. Companies will be facing a different kind of competition and there will be a fresh set of winners and losers. That’s nothing new, says Frank Lehrbass, head of credit management analytics at WestLB. “In our database companies have come and gone over the years.”

Delegates at the conference challenged the regulators to help develop a global database of default and credit history. Imagine the power of a data pool fed by many banks, along the lines of the fictitious Sigma Bank’s credit risk model StreetCred40, which pooled credit data from 120 subscribing banks. Unfortunately this excellent model was bad at predicting credit events in new industries, such as mobile phone companies.

But the regulators didn’t appear that keen to collaborate on a database, prompting a diatribe from one delegate, who refused to give his name. He accused the Basle committee of lack of leadership and a lazy policy of wait-and-see. This was too much for panellist Jochen Sanio, deputy president of the Bundesaufsichtsamt für das Kreditwesen in Berlin. Self-declared sceptic and advocate of extreme caution before accepting rocket-scientists’ models, Sanio retorted: “You want us to develop a database, and next you’ll want us to develop a model and give it to you as a gift!” He had earlier made his view clear that “the burden of proof that credit risk modelling works rests firmly on the shoulders of the banks”, and he wasn’t swerving from that now. Daring Japanese risk manager Katsuto Ohno of IBJ compared Sanio to “an army general trying to regulate the air force”. Ohno’s IBJ colleague Daisuke Nakazato had earlier highlighted the culture gap by mentioning that his final capital adequacy formula for credit derivatives was difficult to display on the screen, since it was eight pages long.

But Sanio wasn’t alone. Claes Norgren, deputy chairman of the Swedish finance inspectorate also wondered whether these complex models were reliable for regulators and the simple rules that regulation requires. Even the FSA’s Page, known as model-friendly, drew gasps from delegates when he declared: “Some of the issues raised by credit risk models just may not be soluble for regulatory capital purposes.”

Models may be good for credit risk management but, he warned, “we [the regulators] focus very much on the tails of distribution, you [the banks] focus on the centre of the distribution.”

A practitioner on the same panel, Joe Rickenbacher of UBS, also pronounced himself a sceptic and called for more liquidity [of credit instruments] as a means of risk reduction, and “first and foremost, data collection, and calibration of credit ratings around the world”.

This practical talk may have upset those modellers in search of the holy grail. But one delegate reminded everyone that the near-term goal is “not competing with some Platonic ideal: we’re competing with the 1988 Basle Accord”. Even regulators agree that’s not much of a contest. New York Federal Reserve chairman Bill McDonough had predicted earlier that a change to the accord was likely to come “in the next one to two years”. He also admitted: “Whatever we choose as a direction for future capital requirements is likely to be imperfect and will eventually need to be replaced, no doubt sooner than we would like.”

The compromise may be an interim accord which allows, not complete risk models, but more refined risk buckets to take account of credit spread correlations. Always remembering that, in times of stress, correlation is the first to go.

Complex global banks are kidding themselves if they think their endeavours will allow them to carry less regulatory capital overall. Daniel Zuberbühler, bearded Swiss regulator of “systemic” giants Credit Suisse and UBS, for one is not prepared to compromise. “Whatever these models suggest,” he said, enjoying his pipe after two days wrangling in The Pit, “the answer is these banks need more capital.”