Some may choose to defend it as model arbitrage, but in reality, in some cases what is actually happening should be more accurately labelled as thieving. So I read with interest a recent report from consultancy firm Celent called Model Validation Best Practices: Achieving Value Added, authored by Umit Kaya and Il-Dong Kwon.
The report could have been sub-titled Stating the Bleeding Obvious, but that in no way detracts from it, for it highlights some pertinent truths about today’s financial markets. “The last decade has seen a maturing process for the management of financial risks….One of the drivers of this process has been the increasingly complex dynamics of the global financial markets, which have brought about the necessity for commensurately complex models. The upsurge in usage of models, sophistication of underlying theories, and complexity of the IT environment have firmly placed model risk (the risk that models do not perform as intended) on regulator and senior executive agendas…..Accounting troubles and tight regulatory controls around risk measurement models led the first model validation wave, primarily as a compliance exercise. The next wave will be characterized by repositioning beyond compliance towards a streamlined, value-adding model validation process. Completely eliminating model risk through controls is impossible; therefore institutions need to strike the right balance between costs and benefits,” the report states.
The authors go on to outline a framework that will improve model validation, and in their conclusion they write. “The past few years have seen the establishment of model validation units. However, best practices documented in this report are only employed by a minority, and it would be optimistic to conclude that the state of the market is close to the end game. A significant portion of market participants have approached model validation with the mindset of a one-off exercise and/or a pure compliance activity.” While that may sound shocking, I hear that in some cases it is even worse and actually entails little more than a risk manager asking the quant who designed the model whether it works or not. Engineers in other industries would not get away with such an approach. Could you imagine a tester basing his validation of a process, such as construction, on the calculations handed over to him by the very same person he is evaluating? Why should financial engineers be treated differently? There is more, inevitably, to follow on this.