What went wrong with the ASX blockchain landmark?

The Australian Securities Exchange took a leap of faith in commissioning Digital Asset to build a blockchain replacement for its clearing and settlement engine five years ago – perhaps too big a leap. Here, Digital Asset’s CEO explains what went wrong and what was learned.

It was supposed to be a game changer, the great step forward in the use of blockchain technology that would elevate it from a clever but nebulous idea to the heart of real-world financial markets.

After two years of study, the Australian Securities Exchange appointed a company in 2017 called Digital Asset Holdings to build a distributed-ledger system to replace the ASX’s 25-year-old clearing and settlement systems, called Chess (Clearing House Electronic Subregister System).

Five years on, plenty of examples of blockchain technology exist in stock exchanges – at Deutsche Börse, for example, or Switzerland’s SIX Digital Exchange – but back then it was seen as highly ambitious and attempting something on a greater scale of complexity than either of those examples.

Too complex, it turned out.

After multiple delays, in August Accenture was commissioned to look into the state of the project. The consultant’s report identified problems from timelines to communication to complexity, and suggested that its implementation, which the ASX had indicated as near-ready, was no more than 63% complete.

In the aftermath, in November, the ASX halted the project and said it would write off as much as A$255 million ($173 million) in technology costs. A parliamentary committee has followed and questions have been raised about the ASX’s culture, governance and technology capability. There are calls for the whole board to go.

Yuval-Rooz-ASX-960.jpg
Yuval Rooz, Digital Asset

Digital Asset

One voice that has been relatively quiet in the fallout is that of the blockchain vendor, Digital Asset. Here, its chief executive and co-founder, Yuval Rooz, speaks to Euromoney about what happened.

The first thing we talk about is the scale of the whole challenge.

“At the time it was extremely groundbreaking,” he says. “There was no blockchain application running in production that would do something as scalable and complex.”

Then there is Chess.

“More specifically, Chess itself is significantly more complex than the rest of the world when it comes to clearing and settlement,” says Rooz. It requires the underlying system to hold stock in the customer’s name on the register, and “encompasses a number of unified functions beyond clearing and settlement that stretch across many different market conditions.”

Many of the non-functional capabilities are either to be built or in build stages

Accenture report

Chess constitutes a monopoly position in Australia over clearing, settlement, asset registration and some post-trade and issuer services. The assignment covered six core issues – holdings, batch settlement, bulk process support and resiliency, bilateral matching, issuer (HIN) notifications, and support for ex-transactions – and required the replacement system to consider several underlying drivers. One was latency: distributed systems are meant to introduce higher latency, but bring more complications. Others were concurrency, batch processing and technical constraints.

Digital Asset proposed to deal with these issues using a language it had devised called Daml. This underlying language comes through the Accenture report reasonably unscathed – “the review highlighted the existence of a high quality Daml implementation providing considerable efficiencies relative to the current Chess and is not contributing to the core issues,” it says – but the problems arose elsewhere.

Specifically, there was a difference between the functional requirements of the system and what were known as non-functional requirements, or NFRs, relating to things like scale and performance.

So, when Accenture came up with its 63% number, another way of putting that was that most of the functional stuff was up and running, but the non-functional side was not.

“Many of the non-functional capabilities are either to be built or in build stages,” Accenture said.

Later in the report, Accenture says 77% of the non-functional capabilities have not yet been delivered for testing.

One might think of this as successfully building a high-performance car without getting around to thinking about whether it can fit in the lanes of the roads it will drive on. In the heart of the tech, all is reasonably well; making it work for the real-world purpose for which it is intended, less so.

The Chess replacement is extremely ambitious, and was a very large project to take on for a company of our size in 2017

Yuval Rooz, Digital Asset

So, what went wrong with the non-functional work?

“In order to be able to meet NFRs, we needed to push data through the system,” says Rooz.

An accomplished runner, he uses the analogy of wanting to run a marathon in three and a half hours: “You go for a long run to see how much training you are going to need to do to get to that number. We were unable to go out for that long run on the system, and when we were asked to give an estimate on how long it would take to run our marathon, it was hard to answer.”

It is not clear why this data has not been pushed through the system.

Rooz suggests a technological constraint that became important: “In computers you can use a process called threading, in which each thread will have some level of capacity that it can process. In modern computing, you can have multi-threading: running a lot of things in parallel. But if the business requirements force the computation to be put through a single thread, it doesn’t matter if it is a strong computer because you are physically forcing everything through that one thread.”

The Accenture report also questions the quality of communication between ASX and Digital Asset. Rooz refutes this.

“From our perspective, there has been no miscommunication,” he says. “In every one of our projects with our clients we are very open with our estimates and what we think is the status. This project is no different.”

If there was a communication issue, he suggests, it was more about engagement with external stakeholders: “The report also called out engagement with customers, expectation management, and that is critical, internal and external.”

The lessons

We ask what lessons have been learned? What should have been done differently? And this prompts Rooz’s longest and most reflective answer.

“The Chess replacement is extremely ambitious, and was a very large project to take on for a company of our size in 2017,” he says. “We did so earnestly and have learned too many lessons to count.

“Two points really stand out for us, and are points of emphasis when working with all of our customers,” he continues. “First, aggressively scope a minimum viable product and get that to production as soon as possible. That early moment of getting to production and then being able to drive additional functionality on the back of user feedback and vision can be quite fulfilling.”

You can run, in my opinion, the top three equity markets in the world on this technology

Yuval Rooz, Digital Asset

Second, he says, working with a new technology “benefits from a universal shift in mindset. Lift and shift in many ways can defeat the purpose of the technology decision.”

‘Lift and shift’ refers to the idea of moving an application or system to another platform without redesigning it. “The most successful projects are those that embrace the opportunities the technology provides,” Rooz adds.

This suggests that the ASX didn’t go far enough in its spirit of reinvention: that rather than just moving to a new underlying system, it ought to have had a rethink about some of the things it was proposing to do on that system. (That is Euromoney’s read; Rooz doesn’t explicitly say this.)

“There were things that existed in the Australian market for 30 years that are not scalable to today’s market conditions, and the ASX can take the opportunity to make those better,” he says. He gives the example of net broker obligations as “one of those workflows that doesn’t scale very well at today’s volumes.”

Still, he is keen to reflect the fact that the exchange was bold in what it sought to do: “We applaud ASX for the ambitious decision they made years ago and would be happy to support them going forward.”

Broader question

There is a broader question, now understandably being aired in Australia and beyond, about whether the technology is fit for purpose. The question that keeps arising with blockchain is: sure, it works, but is there a specific problem that it fixes? Does it actually make anything better in a way that is scalable and usable in the real world?

On this, Rooz’s conviction hasn’t budged.

“You can run, in my opinion, the top three equity markets in the world on this technology,” he says. “The most important NFR in the system has been achieved a long time ago. If I look at some of the systems that are in production – Deutsche Börse, Goldman, Broadridge – we are able to meet all those NFRs. It is not just that I have confidence in the tech, it has proven itself.”

Here, he is referring to Deutsche Börse’s D7 digital post-trade platform – “Deutsche Börse will say D7 is their future infrastructure,” Rooz claims; Broadridge has introduced distributed ledger repo; and Goldman Sachs is trading some bonds and other debt securities on blockchain-based networks such as Daml.

Christoph Böhm, CIO at Deutsche Börse, has indeed stated: “D7 reflects our strategic ambitions and the clear focus we put on fostering innovation and new technologies. We are creating the next generation of future-proof financial infrastructure for the digital era, delivering cutting-edge technology and services, and paving the way for the transformation towards digital markets for our industry.”

Current activity on the Chess replacement project is paused as we revisit solution design

ASX

Rooz also believes ASX, once it has taken a period of review, might still come back to blockchain as a solution: “The point about this pause is that after you suffer some blowback, it gives you an opportunity to progress in the right way.”

Let’s hear from ASX. A spokesperson tells Euromoney what happens next.

“Current activity on the Chess replacement project is paused as we revisit solution design,” he says. “This process will consider a number of broad options, including consideration of the work that has been completed that can be leveraged into a new solution.”

So, indeed, blockchain might still be part of the ASX’s future picture.

The ASX confirms that the functional side of the project was largely done and the complexity of the challenge.

“The functional requirements of Chess replacement were substantially delivered and the project had moved to industry testing,” the spokesman says. “However, there were a number of concerns relating to non-functional requirements that were not being fully met, including performance, scalability and supportability.”

That caused ASX to communicate delays and then commission Accenture in August.

“Replacing Chess is a large and complex undertaking,” the spokesman adds. “There is no ‘off-the-shelf’ solution available to meet the needs of the Australian market.”

In the background of all of this is a debate about whether or not ASX ought to maintain its monopoly over clearing and settlement. This is perhaps a red herring: the world’s biggest exchanges tend to have only one clearing and settlement provider, whether as a matter of stated policy or just a natural practical outcome.

Asked about the debate, the ASX says: “Structural separation is a matter for government.”

Publicly, the ASX’s top brass have been acutely apologetic, knowing that many linked providers have had to spent millions of dollars readying themselves for a system that may now never be operational.

“On behalf of ASX, I apologise for the disruption experienced in relation to the Chess replacement project over a number of years,” said ASX chairman Damian Roche on November 17. “ASX provides critical market infrastructure. What we do matters. We must do it right, and we will.”

The Australian Securities and Investment Commission will take some convincing. Its chairman Joseph Longo said in an interview in December: “The commission and I have sent the strongest possible message to the [ASX] board that this must be their top priority, dealing with Chess replacement. The confidence in that has been shaken by these recent events.”

Backdrop

All of this is happening against the backdrop of the collapse of the crypto exchange FTX and deep-seated institutional fears about the structural utility of blockchain ideas.

Rooz acknowledges that crypto has not helped with acceptance of distributed-ledger technologies.

“Crypto opened the imagination of what is possible and did a lot of us a lot of favours,” he says. “But what it did very badly was that rather than focusing on real-world utility applications, it went for something very easy to get a lot of retail people interested – which is speculating.

“One of our board members offered the analogy that crypto looks and feels a lot like Napster in the early days,” he adds. “It provided the means to establish market demand but in illicit ways. Apple and Spotify then worked with the music industry to satisfy and expand that demand in a scalable, safe way.

“I can see a similar outcome for the digital asset space.”

Along the way, Rooz says he has learned that pitching blockchain as an end in itself is pointless.

“We don’t talk to clients about blockchain,” he says. “Instead we ask, at the end of the day, are we providing you utility that improves your business?

“Whether it is on blockchain or not does not matter.”