FX: Different testing no guarantee of better outcomes


Paul Golden
Published on:

While the ability to run simulations based on hypothetical future as well as historic data is appealing, there is no guarantee that testing algos against both types would make them operate more efficiently in stressed market environments.

Advanced simulation models are not a recent phenomenon – they have been used for many years in disciplines such as climatology and physics to study 'what if' scenarios. But increased availability and affordability of computing power have brought these techniques within the reach of banks and other FX market participants.

Justin Lyon 160x186
Justin Lyon, Simudyne
According to Justin Lyon, CEO of Simudyne, advanced simulation models based on synthetic future data are able to prepare algos to deal with unprecedented market conditions rather than simply shutting down in the face of volatility, which can mean immediately exiting or hedging all current positions or simply not sending any orders at all until the unprecedented condition passes.

"Historical data can quickly lose relevance as market conditions and structures evolve, but execution is increasingly being undertaken by trading algorithms which can only learn from the historical data they have been fed," he says. "Agent-based modelling and simulation allows key aspects of the data-generating process to be captured, thereby enabling the creation of synthetic data."

Lyon reckons that testing algos against this data would improve market participants' understanding of how their strategies could impact the broader market.

The right data

Using simulations to create unprecedented market conditions is key to risk management, observes Aite Group senior analyst Audrey Blater. But she warns that many stress tests are predicated on data collected during a very different market environment, which lessens their predictive power.

Another potential issue with historical data is that it often comes from historical price feeds that are the sum total of prior trading decisions, of trades criss-crossing the spread numerous times, which means that the market impact of others is impounded in the price. Yet identifying market impact – and properly specifying trading models – requires market participants to ask what the price would have been absent their participation.

Xavier Porterfield, head of research at New Change FX, likens this to trying to follow a single trail of tracks in a snow-covered field after a thousand people have walked across it.

Neill Penney 160x186

Neill Penney, Refinitiv

Most quantitative trading strategies are not put live until they have been proven in simulation through back-testing against actual historical data. While in the past such simulations were run primarily to satisfy the trader running the strategy, increasingly the compliance and risk function of a trading firm must also be satisfied with these simulations before a strategy can go live.

Neill Penney, managing director and co-head of trading at Refinitiv, remains confident that simulations that make use of high-quality historical data are the most convincing form of simulation because, by definition, that data encapsulates participants’ actual reactions to market events.

"Where historical data is incomplete or its time series is short, we have observed that many participants simply shut off their strategies when live market conditions do not reflect market conditions that appeared in the back-test," he says.

The way in which back-test simulations are designed can address what otherwise might be viewed as limitations in only using historical data. By observing the market impact of other trades that have occurred in the market and appear in the historical data, traders can estimate the impact of their own trades and incorporate this into the simulation.

Modelling rules

Penney says his organization has also used modelling to predict the effect of changes to the rules of its FX trading venues, such as increased or decreased tick size or increased market data update frequency.

"Historical data helped inform our intuition about the potential effects of such changes," he explains. "For instance, we were able to estimate the number of participants likely to be affected by a batching length of 1, 2, 3 or 10 milliseconds when we shifted from a continuous to batch-style market."

The fragmentation of the FX market and the lack of a consolidated tape further limit the opportunity to build effective simulations, as does the fact that the data available to market participants is not consistent in taking into account factors such as last look compared to no last look pricing or indicative versus executable pricing.

Synthetic data could help make algos more robust during unprecedented market conditions, but approaches to obtaining this synthetic data involve studying, transforming or deriving it from actual historical data, says Penney.

“It may be useful to modify historical data for the asset class to amplify price movements or speed up events,” he adds. “In the context of agent-based modelling, the study of real historical data might lead to insights about what realistic behaviours of agents in the market are – and this is important because these simulations are only as convincing as the realism in their assumptions about those behaviours.”

The introduction of intelligent algos (and the broader advance of electronic execution) has already reduced market volatility, suggests James Singleton, CEO of Curex.

"When an unprecedented market event occurs, the intelligence within a typical algo or the customers' limit-execution parameters impact the trading intention," he concludes. "Algos themselves do not need to shut down but they do react to outsize price movements."