Busting the Myth of Public Blockchains for Business

It’s time to talk about transaction finality. Last week’s 51% attack demonstrates that Ethereum-style blockchains are not ready for business

A belief took hold amongst some of the tech community in 2018: “If you have an enterprise blockchain use-case you should build it on a platform based on Ethereum.”

The argument was pretty well constructed and relied on several plausible-sounding claims so it’s understandable how it seemed pretty convincing. However, as 2018 unfolded, these claims began to be challenged. And as we enter 2019, the final remaining argument has been undermined with a public demonstration of how the lack of settlement finality in public blockchains such as Ethereum renders their immutability and security guarantees worthless for business.

In this piece, I will argue that it is now time to conclude that Ethereum’s core technologies are the wrong foundation upon which to build business blockchain solutions. My argument is: 1) the core Ethereum technologies are due for abandonment, leaving businesses at risk of technology dead-ends, 2) the Ethereum developer skill-pool has been massively overstated and is in fact far tinier than that for the purpose-built business blockchains based on existing languages, and 3) the idea of building on Ethereum in order to securely ‘anchor’ private blockchains to a public chain is now discredited.

In short, business blockchain applications should be built on technologies designed for the enterprise, not Ethereum.

What was the argument for why businesses should build on Ethereum?

To understand how we reached this point as a community, it’s helpful to review the thinking that led here. Here’s how the argument for why businesses should build on Ethereum went:

  • “Go where the skills and innovation are: Ethereum has the largest community and the broadest availability of skills.”
  • “Use the tools that will best let you interoperate with the public chain: Even if you’re not using the public Ethereum network you should use platforms that are based on the EVM, and use languages like Solidity so you can inherit the innovation from the public chain and maximise the chances of interoperability in the future”
  • “Overcome the ‘weak’ security of private chains by ‘anchoring’ in the public chain: Public chains are more immutable than ‘insecure’ private networks and so you should ‘anchor’ your private transactions to prevent malicious parties rolling back your transactions behind your back.”

By the end of 2018, there was ample evidence to debunk the first two claims, but the third claim persisted. Indeed, this third claim, that a public blockchain such as Ethereum offers a degree of transaction confirmation permanence that is otherwise unobtainable, has been repeated over and over again, even as late as December 2018.

Until last week, that is, when a 51% attack against the Classic (original) Ethereum network demonstrated for real what we already knew in theory: that history on a public blockchain like Ethereum can be arbitrarily rewound, money double-spent and network participants defrauded.

The rest of this article will review each of the three claims above in depth to explain why they are incorrect and how that makes Ethereum – and Ethereum-based platforms – unsuitable for business. But it’s important to note that the purpose of this blog post is actually to make a positive message. Because the broader picture is actually one of success: Ethereum is proving to be a valuable tool for a wide range of isolated social and economic experiments. And plenty of blockchains purpose-built to solve business problems, such as Hyperledger and Corda, are live and are changing the world of commerce.

So my key message is that it’s the inappropriate application of Ethereum technologies to the unforgiving world of real business problems, for which it was not designed, that we need to guard against. These two worlds have very different requirements.

It’s time to declare in public what has been openly discussed in private: Ethereum is currently unsuited to the world of business and we should have the courage as a community to say so.

So let’s now review the arguments for using Ethereum in the enterprise, that have now shown to be incorrect.

Claim 1: “Go where the skills and innovation are: Ethereum has the largest community and the broadest availability of skills.”

This argument starts well. For example, ConsenSys claim that the “Ethereum developer community” has 250,000 members, by which they presumably mean the number of people who can code using Solidity, the language in which almost all Ethereum apps are coded.

But when you scratch the surface, reality begins to intrude:

  • Hundreds of thousands of Solidity developers sounds like a big number until you realise that there are over a million developers with the knowledge to build applications for Hyperledger Fabric using the language Go and twelve million developers with the knowledge to build applications for Corda using Java. In the latter case, our experience shows that any competent Java developer can pick up the Corda library and be productive in a couple of days. This means the Hyperledger and Corda developer skillpools are at least one, maybe even two, orders of magnitude bigger, even using ConsenSys’s figures.
  • But we need to challenge ConsenSys’s figures, small as they now seem. This is because there is minimal evidence to support even the 250k figure. The claim seems to be based on looking at how many people have downloaded one of the development tools that pretty much every Ethereum developer has to use, and assuming half of them became Ethereum developers. But that methodology doesn’t work. To see why, let’s apply the same logic to the Java ecosystem to generate an estimate for how many developers there are and see if it matches the correct figure, twelve million. Now, we know that one tool for developing Java applications, IntelliJ, had almost twenty five million downloads in 2017 alone, and that product had barely ten percent of the huge and diverse market for Java development tools (Eclipse, Android Studio and NetBeans were all larger). This means we can estimate there were at least 250 million downloads of Java development tools in 2017, which would mean there must be over 125 million Java developers by ConsenSys’s logic. Except, there aren’t… we know the correct number is about twelve. It’s out by a factor of ten. So the true number of people with Ethereum skills is almost certainly much smaller than 250k; I would be surprised if it was even 50k or 10k, a rounding error in the world of developer communities. And the number of those who can write Solidity contracts securely, critical to avoiding another DAO-style bug, is smaller still.
  • And on top of this, we also need to add the huge productivity gains that come from being part of established ecosystems. For example, the range of development environments, debuggers, testing frameworks, profilers and libraries available for the Java ecosystem is staggeringly larger than that for the Ethereum and Solidity ecosystems.

The reality is that the developer ecosystem and momentum is with the Hyperledger and Corda communities, not Ethereum. So it’s perhaps no surprise that the overwhelming majority of truly ground-breaking, successful enterprise blockchain deployments to date run on Hyperledger Fabric and Corda, not Ethereum.

Claim 2: “Use the tools that will best let you interoperate with the public chain: Even if you’re not using the public Ethereum network you should use platforms that are based on the Ethereum Virtual Machine (EVM) so you can inherit the ‘innovation’ from the public chain and maximise the chances of interoperability in the future”

This argument is more pernicious than the previous one. It says to developers: “even if you’ve correctly determined that a public Ethereum network is wrong for you, you should still use the Ethereum toolset for your private project.” It is an argument that plays on people’s deep fears: stick with the crowd; after all, you won’t be fired if you make the same mistake that everybody else made!

The problem is: as we demonstrated above, there is no crowd and the Ethereum community plans to throw all the current technology away in any case: the EVM is set for total replacement. The plan, “Ethereum 2.0”, is to build a new design from scratch.

So the world faces the possibility that, long after the public Ethereum community have moved on to something new, business leaders will wake up one day to discover critical parts of their business are running on technology that isn’t even being used any more for the purpose for which it was built. Talk about buyer’s remorse…

This might be OK if the Ethereum Virtual Machine was a sound technology but, as the team from Kadena documented, the EVM is “fundamentally unsafe”. And the team at Aion also independently reached a similar conclusion and have written eloquently why they didn’t use the EVM and chosen the Java ecosystem instead. And yet consultants, some from reputable firms, are pushing this technology hard in to organisations that don’t always possess the technical expertise to realise the advice may not be appropriate.

Genuinely ground-breaking work is, of course, being done by some very talented and committed people in the Ethereum community on the public Ethereum network, but it is – and should continue to be done – safely away from the back offices of the businesses upon whose data integrity the world depends.

However, 2018 ended with one, last, killer plank in the argument for why businesses should nevertheless build on Ethereum rather than a platform like Hyperledger Fabric or Hyperledger Sawtooth or Corda.

And it was this last argument that was severely undermined this week.

Claim 3: “Overcome the ‘weak’ security of private chains by ‘anchoring’ in the public chain: Public chains are more immutable than insecure private networks and so you should ‘anchor’ your private transactions to prevent malicious parties rolling back your transactions behind your back.”

This argument was actually pretty clever. Here’s how it went:

  • ‘The security of public blockchains is “backed” by the work performed by billions of dollars worth of mining equipment and electricity. To reverse a “confirmed” transaction would be economically infeasible and, since only public blockchains use proof of work, only public blockchains can provide this “immutability” guarantee.’
  • ‘By contrast, blockchains that rely instead on identifiable parties to provide consensus cannot deliver this level of security and immutability; there is always the chance that parties could “collude” to reverse a transaction.’

And so, the proponents of Ethereum for the enterprise propose a clever idea: by all means, use a peer-reviewed fault-tolerant algorithm for your business transactions – you need rapid and final confirmation, after all.

But then, as an additional layer of safety, “anchor” a summary of your transactions in the public Ethereum network. The network that is massively more secure and resistant to mutation. Its proponents even claim this would provide ‘greater “proof of settlement finality”’ and that ‘any chance of counterparty disputes about membership is eliminated’.

This sounds perfect: the privacy, performance and settlement finality of a private chain and the security and immutability of a public chain!

Except… there was always a problem with this argument: finality.

In short, the two unanswered questions were:

  • If your enterprise blockchain needs settlement finality but the chain into which it is ‘anchored’ provides only probabilistic finality, when is it safe to tell a user of the private chain their transaction has been confirmed? What happens if two conflicting hashes might be vying for inclusion at the same time? Are users expected to constantly monitor the underlying chain to check the private chain hasn’t gone bad? And what exactly are they supposed to do at that point in any case?
  • If the ‘anchor’ gets washed away by a ‘reorganisation’ of the underlying public probabilistic blockchain, what are you supposed to do then?

The problem is: technically savvy people knew these questions made the concept highly suspect but the fact that there had never been any high profile examples of where this would ever have been a problem, nobody seemed to care. And the concepts were complicated in any case – probabilistic settlement, reorganisations. All too abstract! So the response seemed to be: “sure… this could happen in theory but it never happens in practice, so who cares?”.

Until last week.

When a high profile Ethereum network suffered a devastating and unprecedented attack, that caused transactions over one hundred blocks deep to go from “confirmed” to “unconfirmed”. Any “anchor” that had been in one of those hundred blocks would have been washed away, opening up the possibility that a simultaneous attack on the private network could result in a conflicting anchor taking its place.

In other words, the trivial ease with which the supposedly secure and immutable chain was rewritten means it failed in its only and single purpose for an enterprise deployment.  

The right approach to settlement finality for business blockchains is to acknowledge things can go wrong and to plan for them up-front: accept that you need to know the identity of the consensus providers, which also ensures provider diversity rather than increasingly centralised mining providers; and that you need a governance process and dispute resolution forum for problems that cannot be solved solely with clever math or novel technology.

Conclusion

So, here at the start of January 2019, what is left of the “Ethereum in business” story?

  • The number of developers with skills in Ethereum is far lower than Ethereum’s proponents claim and is orders of magnitude smaller than the programming language ecosystems supporting Hyperledger and Corda
  • The core ‘engine’ of Ethereum, the EVM, has been publicly disowned by the communities that spawned it and the platform is being expensively rewritten, yet enterprise Ethereum vendors continue to push tools based on this dead-end into unsuspecting businesses.
  • And the only remaining plausible argument for using Ethereum in the enterprise, that it somehow makes it easier to secure your network by ‘anchoring’ into the public network, has been shown by the Ethereum Classic debacle to be false.

Be in no doubt: blockchain for the enterprise is real and it is here to stay. But if you’re doing it on Ethereum, you’re doing it wrong.

 

[Update 2019-01-14 Reworded subtitle to clarify I’m making a broader point about probabilistic finality]

Corda: Open Source Community Update

The Corda open source community is getting big… it’s time for a dedicated corda-dev mailing list, a co-maintainer for the project, a refreshed whitepaper, expanded contribution guidelines, and more..!

 

It feels like Corda took on board some rocket fuel over the last few months. Corda’s open source community is now getting so big and growing so fast that it’s just not possible to keep up with everything any more — a nice problem to have, of course. And I think this is a sign that we’re reaching a tipping point as an industry as people make their choices and the enterprise blockchain platforms consolidate down to what I think we’ll come to describe as “the big three”.

Read the rest of this post over at the Corda Medium blog…

Introducing the Corda Technical Advisory Council

I’m delighted to announce the formation of the Corda Technical Advisory Council (the Corda TAC). This is a group of technical leaders in our community — who most of you know well — who have volunteered to commit their time over and above their existing contributions to the Corda ecosystem to provide advice and guidance to the Corda maintainers.

Members of the TAC are invited by the maintainers of the Corda open source project (Mike and Joel) and will change over time — the inaugural members are listed below. If you’re also interested in contributing to the TAC, please do let us know — most usefully through your technical leadership and contribution to the ecosystem!

Read the rest of this post over at the Corda medium blog…!

Universal Interoperability: Why Enterprise Blockchain Applications Should be Deployed to Shared Networks

Business needs the universal interoperability of public networks but with the privacy of private networks. Only the Corda network can deliver this.

The tl;dr of this post is:

  • Most permissioned blockchains use isolated networks for each application, and these are unable to interoperate. This makes no sense.
  • We should instead aspire to deploy multiple business applications to an open, shared network. But this needs the right technology with the right privacy model.
  • Corda, the open source blockchain platform we and our community are building, was designed for just this from day one. But there was a piece missing until now: the global Corda network.
  • In this post I describe the global Corda network for the first time in public and how it will be opened up to the entire Corda community in the coming months.
  • If you’re building blockchain solutions for business, you need to read this post…

Think back to how excited you were (well, was!) when you first heard about Ethereum. The idea of a platform for smart contract applications, all running across a common network, with interoperability between all these different applications written by different people for different purposes. It was mind-blowing.

And it’s not just a vision, of course. The public Ethereum community have actually delivered it! Indeed, emerging standards such as ERC20 are a demonstration of the power of a shared, interoperable network and the power of standardisation.

So the question we asked ourselves at R3 back in 2015 was: imagine if you could apply that idea to business… imagine if different groups of people, each deploying applications for their own commercial purposes, woke up one day and discovered that those apps could be reassembled and connected in ways unimaginable to their creators but in a way that respected privacy and which could be deployed in real-world businesses with all the complexity that entails.

It seemed obvious to us that this was the right vision. And that it would require a universal, shared, open network, the topic of this post.

But it dawned on me recently that this is not how everybody in the permissioned blockchain space sees it. The consequences for users could be serious.

The rest of this post is continued at our medium site here!

View story at Medium.com

What problem are we trying to solve with Corda?

Todd pointed me at a great piece about “crypto finance” versus “regular finance” by Bloomberg’s Matt Levine earlier.

I thought he did a good job of nailing the essential contradiction that arises if one tries naively to apply Bitcoin or Ethereum principles directly to traditional finance. He uses the example of an Interest Rate Swap (IRS) and how a fully pre-funded model would kind of defeat the point…

This caught my attention because an IRS was the very first project we ever did on Corda! So it’s something I know a little about… Anyway, I think the key to understanding the mismatch is captured in a post of mine from 2015 about how two revolutions are playing out in parallel.

Anyway… full details over on my Medium page.

New to Corda? Start here!

Are you just hearing about Corda for the first time? Want to understand how Corda differs from other platforms and how its unique architecture is perfectly suited to address the real problems faced by today’s businesses?

I just posted to the Corda Medium page with a list of links and background info that should help answer that question…

 

 

Not all business blockchain platforms are alike. To succeed they need to reimagine business computing.

Lessons from the world’s largest multi-year collaborative Blockchain research programme

If you’re not part of the blockchain bubble, you probably think people in the blockchain world are all mad! Especially those of us in the enterprise space. How on earth could we believe a technology, originally built by a group of idealistic anarchists to solve a problem no financial institution actually has, could possibly be relevant to the challenges facing those trying to build and maintain complex IT infrastructures across the world?

In this article, I explain how the work we’ve done at R3 over the last two years with hundreds of senior technologists from dozens of leading financial institutions has led me to some fundamental conclusions:

  • The promise of blockchain technology is real: new business solutions will soon be deployed which can eliminate huge amounts of cost, redundancy, error and needless reconciliation across entire business ecosystems, as well as opening up previously hidden new revenue opportunities. I provide concrete examples below.
  • This will be achieved by applying the fundamental blockchain insight: “I know that what I see is what you see”. This is the key to helping us move from a world of isolated, custom, inconsistent IT infrastructures in each firm, to one based on shared business logic, securely shared data, and common processes.
  • The new IT architecture that is needed to capture these opportunities is one that converges the complex world of application servers, messaging engines, workflow managers and databases into a unified, secure and private design that works seamlessly within and across enterprises.
  • But not all blockchain platforms are alike: Only some designs will be architecturally suited to the challenge.
  • Our research has revealed several key characteristics that are needed to succeed. An enterprise blockchain platform must:
    • Provide an integrated application, messaging, workflow and data management architecture.
    • Build on an existing technology “mega-ecosystem” to maximise skills and code reuse.
    • Eliminate unnecessary data “silos” to unlock new business opportunities by allowing real-world assets to move freely between all legitimate potential owners.
    • Embed legal entity-level identification into the programming model to enable legally enforceable and secure transactions.
    • Support inter-firm workflows for negotiating updates to the shared ledger to support real-world business scenarios.
    • Enable the inevitable move to the public cloud without requiring high-risk technology bets.

For the last two years, I’ve been privileged to lead an intense research and development effort amongst the membership of the most vibrant consortium the financial world has seen. Hundreds of senior technologists from dozens of companies have worked with R3’s engineers to identify where blockchain technology can make a difference in large enterprises and also how it needs to be designed to achieve this potential.

This work has convinced me that the judicious application of key blockchain principles is key to rescuing the world’s banks and other large companies from the corner into which they have painted themselves after decades of investment in previous generations of technology.

In particular, we could be looking at the holy grail of enterprise software: convergence of application servers, messaging engines, workflow managers and databases in a way that works seamlessly within and across businesses.

The result is a model that delivers on the promise of blockchain in the enterprise: a world where applications can be securely developed and deployed across trading partners, enabling them to transact securely and accurately and without endless reconciliation, inconsistent data, and duplication. It is why we are now able to think about building or replacing payments systemssyndicated loans processing systems, asset issuance and custody systemsFX matching business processes and much more.  And these are just examples from the banking sector being addressed by just one platform, Corda, which itself turns out to be applicable far more broadly, in areas as diverse as healthcare and identity management.

But our belief is that it is only by converging previously diverse technologies that we can dramatically simplify how companies develop, deploy and manage applications that manage their business relationships with each other.

Here’s what I mean. Through one lens, a blockchain is like an application server: it hosts business logic and ensures it runs at the right time and for the right reasons. But a blockchain is also like a messaging engine: it allows people and, importantly, their computers to exchange information and ensure that the right information gets to the right places in the right order – and to do so between companies as well as within. And some blockchain platforms also have characteristics of workflow managers; they help coordinate the activity of different parties across time and space to achieve some business outcome. And, of course, they can be akin to databases.

If we could somehow devise a platform that built on this insight and delivered a platform that could combine the function of these hitherto diverse technologies, and could do so in today’s adversarial security environment, with the necessary levels of privacy, and in a way that was easy to use and could reuse rather than replace what already exists, imagine how much simpler our future enterprise IT landscape would look!

But building a platform that combines these separate concepts is easier said than done. There’s a reason existing vendors haven’t already built something that delivers on this vision.  In what follows, I’ll share the output of the last two years of our work, which confirmed to me that, by judiciously weaving together existing technologies and key advances from the blockchain world, it is indeed possible. I highlight some of the specific, key design choices you need to make in order to achieve this convergence and hence unlock the massive opportunity to transform the IT estates of existing companies.

I will refer at times to a key output from our work: the open-source Corda platform. We developed it in parallel with the research effort I describe above and, as a result, it benefited from a huge amount of expert input from across the R3 membership. No other enterprise blockchain platform has enjoyed remotely the same level of expert input. But I also refer to other enterprise blockchains where contrasts are helpful, paying particular attention to one that has come to prominence of late, Quorum, because it makes some very different choices and hence acts as a useful comparison point. This is intended to help make some of the points concrete and draw readers’ attention to areas of legitimate disagreement.

The ideal enterprise blockchain is a next-generation application server…

First and foremost, the opportunity that blockchain technology presents to enterprises is the ability to write applications that are shared between those who use them to transact. The key idea is this: rather than you and all of your trading partners each writing an expensive application to manage your participation in that trading network, you write it once. We thus share the cost and effort, whilst each running our own instance of the application in order to maintain our own books and records.

But to enable this vision, we concluded from our research that we need features that are unavailable in traditional application servers but which are common in the blockchain world. One such feature is cryptographic chains of provenance for all data in the system. When freely sharing relevant data with your trading partners, you must take nothing on trust but verify all proposed updates to the shared application state. This is also why all consensus-critical code on the platform needs to be digitally signed so you know that the code that is running is the code that should be running.

And we need to enable as much reuse of skills and existing technology as possible: you don’t save money by throwing away all the existing technology you have or by forcing all your staff to retrain! So you should be able to write applications in languages that have as large a population of developers as possible. Working with our diverse membership, we concluded that Java is by far the most widely deployed language across the business world and so we focused on that ecosystem. There are almost ten million people in the world who have these skills. But we also found that large enterprises hunger to bring their infrastructures right up to the contemporary cutting edge, so support for techniques such as reactive programming, approaches like functional programming and new languages like Kotlin and others are also required.

In short, the ideal application server for the modern era is one built for security, productivity, data integrity and cost minimisation.

But don’t all enterprise blockchains do this? Actually, no. To see that, we can compare the approach we took with Corda – the result of our two year journey – with a competing blockchain platform such as Quorum.  Although superficially similar to Corda in the way it describes itself it is, in fact, a relatively small fork of an existing public blockchain platform called Ethereum – which was designed to solve a completely different set of problems. Quorum inherits all of the associated advantages and disadvantages as a result.

For example, rather than supporting a full range of modern and mature languages to suit different needs, for which there are abundant skills and existing libraries, as Corda does through its support for the Java Virtual Machine, Quorum only supports languages that run on something called the Ethereum Virtual Machine. The EVM is a young, purpose-built virtual machine maintained by a community primarily focused on executing cryptocurrencytransactions and associated business logic. No mainstream languages run on the Ethereum virtual machine. This raises obvious questions around skills and integration with heritage systems. But it also opens up serious new risks for those who try to apply Quorum to real-world business problems. For example, the most popular language for writing applications on Quorum has been repeatedly shown to be impossible to use safelysuffering from countless bizarre language features that have led to people losing real money.

Not all enterprise blockchains are alike.

The ideal enterprise blockchain is also a next-generation messaging platform…

However, as I argued above, the exciting realisation during our research effort with our member banks was that good blockchain platforms are also inspiration for how to rethink enterprise messaging systems. Messaging systems are used by big companies to link computers and applications to each other securely and reliably.

The need for this becomes acute when you talk about deploying shared applications betweentrading partners. The applications running on different firms’ nodes need to be able to communicate. We need all parties to be securely identifiable and to be able to communicate with each other near-instantly over the internet or private network, addressing each other by legal name.  It’s not good enough to take existing enterprise messaging platforms that let you communicate with an address or a queue; you need to be able to talk to a named legal entity, anywhere in the world.

In our work with the diverse group of technologists from across our membership, we identified that the way to achieve this was to take existing technologies – in this case, TLS, X.500 and more, and make them consumable and accessible to today’s application developers. It seems obvious when you think about it yet no mainstream platform offered it in this way until we supported it in Corda. It was necessary to weave insights from the cryptography community, where public key infrastructure is well understood, and the banking community, where financial networks between identifiable peers are common, to come up with the idea of “send to legal entity” as the natural and obvious way to communicate between parties.

A separate finding from our work was that large institutions are institutionally allergic to anything that looked like it could create new islands of connectivity.  They needed a platform that would allow multiple independent Business Networks to operate within a single global namespace – perhaps something we could call a Compatibility Zone.  No more tolerance for isolated silos of communication and data distribution (unless you want it of course…). And this turned out to be key for the delivery of the holy grail: true representation of real-world assets on a shared network, with no artificial barriers to exchange or transfer, which I talk about more below.

Achieving this level of seamless interoperability across multiple business networks, whilst retaining flat, transparent, legal-entity-level addressing is hard. But I think we’ve achieved it with Corda.  And it’s something that is lacking in platforms like Quorum, where network membership is defined in text files which must be copied to each node, and where behaviour is undefined (or, rather, may have “adverse effects”) if they don’t match perfectly.

Identity and legal-entity addressing can’t be an after-thought in a text-file: they need to be designed in from the start.

The ideal enterprise blockchain is also a next-generation workflow manager…

When I worked with clients in my previous jobs, I would find they deployed workflow management tools, also sometimes called “Business Process Management” platforms, inside the organisation to control and optimise the flow of work between people and systems.

But all too often these systems would stop at the edge of the firm.  The “other side” was treated as a black box.  And yet, it is in the interaction between firms that the risk and cost creeps in.  Did they receive the message? Are they working on it? Did they understand it and process it correctly?

Do they see what I see?

This isn’t actually something that existing blockchain platforms provide much help for. And yet it is a real need even for some simple cryptocurrency transactions such as implementing an escrow arrangement or telling somebody how to pay you! In essence, for every meaningful transaction on a blockchain platform, there is usually an out-of-band negotiation flow that needs to take place.  Just like in real business.

And as we worked through countless use-cases during our research, what emerged was a need to make it easy for developers to write this back-and-forth business logic for the collaborative negotiation of a deal or construction of a transaction.

We discovered a need for an inter-firm workflow automation technology where developers don’t have to worry about any of the complex technical work necessary to orchestrate such co-operation at scale across a global network. In particular, it needs to be possible to define business processes that flow between, within and across firms, yet which can be coded in a simple and natural programming style, where you can express what each party in a transaction should do and have that work automated within and across your firms.

Unfortunately, developing something so ambitious would be hard if implemented from scratch. So it provided further weight to our emerging belief back then, a certainty today, that a successful blockchain platform for the enterprise has to build on the work of an existing, massive ecosystem, where it can reuse as much existing infrastructure and code as possible.

If we now turn to the specific case of Corda, we were able to take cutting edge technologies from the Java ecosystem and add some seriously clever engineering by the R3 engineering team to deliver something unique called the Flow Framework. In particular, recent advances in automatic checkpointing of business logic, with restore across system restarts, means developers can write what looks like entirely normal straight-line logic and yet, behind the scenes, a complex international workflow is being orchestrated. The first time you use it, it feels a bit like magic.  This is an example of why it’s so important to build on the foundations of an existing and robust ecosystem, such as that provided by Java, whilst at the same time bringing the programming model and tools right up to the contemporary cutting edge.

The ideal enterprise blockchain is also a next-generation asset ledger…

A repeated theme from our work with members was that the real wins come when the new enterprise blockchain applications become systems of record for real transactions.

This means they need to provide legal certainty.  Application developers need to link their contract code to associated contract legal prose to give certainty about how disputes will be managed.  But, for this to work, one must go further. Just as a successful enterprise blockchain identity layer needs to be based on legal names, the actual nodes – the machines running the business logic – also need to be clearly associated with a specific legal entity.

Such a framework, provided by the legal prose, identity-driven messaging and legal entity association with nodes, means that contracts signed on platforms that support it can be legally binding if you choose. It’s a prerequisite to being safely able, at scale, to support direct issuance of assets and the direct formation of contracts.

It means that a bank or other firm can issue assets (cash, equities, even commercial paper or syndicated loans) on to the ledger and have them be directly owned and seamlessly transferred to other members of an appropriate business network in near-real-time and with settlement finality.

And make no mistake: that is a hard trick to pull off.  For dematerialised assets to have full utility, they must be easily transferrable to any and all legitimate potential owners: isolated islands won’t do. So it turns out that this also has strong implications for the most fundamental aspects of the design: how data on the blockchain is managed, distributed and evolved, which I describe more below.

The issue we found during our research was that if you get this wrong, you risk trapping assets in silos, where they can only be transferred amongst members of a preconfigured list or through the involvement of third-party market-makers, or by asking the issuer to impose friction by cancelling and reissuing them.

It’s a tough problem to crack, as the link above makes clear in its analysis of another platform. The criticism in that link also applies to Quorum: they both suffer from this stranded asset problem and it means apps you write for those platforms will almost certainly have to be extensively rewritten in the future when those platforms are redesigned to correct these shortcomings. A costly prospect.

It’s one of the reasons we were so focused on delivering a version of Corda with “API Stability”: members and other developers told us loudly and clearly that they highly valued knowing they wouldn’t have to redesign their apps in the future

And the ideal enterprise blockchain is also first-of-a-kind decentralised database…

At the heart of the enterprise blockchain vision is the idea that, if two or more parties are transacting with each other, then they should all have an identical copy of the associated data.

“I know that what I see is what you see”.

To achieve this, we have to ensure that all parties have fully validated that their shared state was indeed calculated correctly.  At heart, it is this that distinguishes blockchains from what went before: we don’t take things on trust; we verify.  And to verify we need to run the logic for ourselves. We have to check the provenance of data.

What we want is, in effect, a globally shared database, but where each party only has the rows of the database that pertain to transactions they’re part of.  So we need to make sure that you receive the rows you need and if anybody tries to change any of them, they can only do so if the rules are followed and that you get to sign off on it and/or get notified afterwards as appropriate. To get an idea of how this works under one architecture, see this simple analogy.

The design we concluded that best achieves this is one that builds on and heavily extends and generalises ideas that originated in Bitcoin rather than Ethereum. This model, which Corda adopts, encourages developers to think of the fundamental units of information in their business problem and how they can evolve over time in isolation. Corda then adds the powerful flows I described earlier to orchestrate their updates via transactions that specify precisely which data units – “state objects” – are being referenced, created or replaced.

This approach allows for arbitrarily rich scenarios and sophisticated negotiations and business processes to be modelled with ease, whilst allowing the system to tell precisely who needs to receive what data, when, whilst revealing nothing more than is needed to prove provenance.

Other platforms take a different approach, based on an Ethereum-like model.  They typically start with the idea of an “everybody sees everything” full broadcast blockchain, representing a globally shared computer, and go fractal in order to correct for the privacy issues that result: they spin up tens or hundreds or thousands of separate mini shared-computers, one for each group that wants to transact in privacy. This approach can work but our analysis, confirmed by later experience, was that, for real world scenarios, it gets very complex, very quickly.

In the case of Quorum, each “confidential contract” can be thought of as a mini Ethereum universe that is visible only to the participants in that contract. This idea of millions of mini-blockchains works for some scenarios, but as I described in this post the approach, used by some other platforms too, fails completely if you ever need to support asset mobility. As soon as you need to prove provenance of one piece of data in a confidential contract to somebody else, you have to show them everything in that contract. Game over: you either lose privacy in the hunt for provenance or your assets are stranded, with their provenance impossible to be demonstrated to outsiders.

And… the ideal enterprise blockchain must also be designed for the cloud

The world doesn’t stand still. The enterprise software market in 2017 is different to the market in 2000…  In particular, most new applications that get deployed in the future will run in the cloud, increasingly in a public cloud.

So this was inevitably a hot topic of debate throughout the consortium’s deliberations over the last couple of years: how can we support application developers who need to write applications that will form legally binding contracts and be the basis of their audited books and records yet which will run on other people’s computers in a potentially hostile environment?

We considered the obvious options: zero-knowledge proofs, homomorphic encryption, secure hardware and more, and combinations of them all.

Our first conclusion was that the answer needs to be multi-layered: deterministic sandboxes, signing of all code and transactions, everything underpinned by legal-entity-level addressing, support only for a well-understood and tested managed runtime (in Corda’s case, the JVM, which we heavily lock down) and so forth.

But we also made a decisive choice: when weighing up risk, availability of skills, time to market and a collection of other factor, we concluded that the right initial approach is first to support hardware security technologies and, Intel’s SGX technology in particular.

This allows users to deploy Corda applications to other people’s computers (ie on public clouds) in a way that prevents those operators from interfering in transaction verification or accessing historical transaction information yet with no material limitation on the business logic that can be run. And we found that making this work well required it to be designed in from the ground up, which we did.

We concluded that this approach contrasted favourably with a higher-risk one being taken by platforms such as Quorum, who are recommending today the use of zero-knowledge proofs. Zero-knowledge proofs are a very fine piece of engineering and mathematics. They’re almost certainly part of the future of privacy in the long-term on blockchains – and Corda is designed to adopt them when they have matured.  But, today, the story is different.

First, and as Mike and Kostas in our engineering team discussed at CordaCon in September, the world of Zero Knowledge Proofs is young, very few people have skills, it is almost impossible to know what one of these things actually does when written and the security assumptions they rely on are unproven. Indeed, the “zk-SNARK” technology being promoted in February this year by the inventor of Ethereum is now being challenged by something new, called zk-STARKs This is a promising sign of a vital research community, rapidly advancing a young field.  But it’s a risky foundation upon which to build enterprise solutions before it has matured more.

Secondly, the integrity of the financial system rests on the idea of atomicity: related activities should either all happen or not happen at all. Inconsistent, half-complete exchanges just won’t do.

This requirement was hammered home to us by our members during the design process for Corda and so the ability to do complex atomic “payment versus payment” and “delivery versus payment” transactions is a fundamental part of the architecture.

Unfortunately, Quorum, which needs to use zero-knowledge proofs to achieve acceptable privacy in their system cannot, according to their own documentation, achieve delivery-versus-payment:

… the POC solution will not support cryptographically-assured DvP (i.e. atomic exchange of assets). This is because ZSL currently has no means of supporting shielded DvP-style functionality.”

This is why it’s so important to distinguish between the generic promise of enterprise blockchain technology and the practical reality of specific implementations.

Thirdly, we need to remember at all times that this whole emerging industry is about consensus, about being sure your counterparts really do see the same things you do and that you agree on all important data, when it happened and in what order.

But we learned from our members that how you reach this consensus may need to differ based on the business context: maybe you’re trading amongst a group of peers who you know well: you may each need to participate in the decentralised consensus forming process by running a node as part of a consensus cluster and reach consensus quickly and with finality amongst yourselves.  But, perhaps for some other line of business, you’re transacting with people all over the world and you want an independent, decentralised group of impartial observers to timestamp the transactions and help you reach agreement about ordering. That must also be supported: you need to be able to use a fully byzantine-fault tolerance decentralised cluster of consensus nodes, operated by mutually distrusting entities. And a successful platform needs to support all these modes – and more – and, here’s the key part: on the same network, at the same time!

So when we designed Corda, we engineered it so that you’re not forced to pick one consensus model and expect everybody to accept a one-size-fits-all for every transaction they perform on their blockchain network. That would be the road to disconnected, isolated blockchain networks… and the fast lane to stranded assets.

Why I believe Corda is the future of enterprise software

This piece began by outlining what I consider to be some of the critical findings of our journey so far. And we’ve embedded as many of them as possible into the design of the platform we jointly defined with this unprecedented consortium of institutions and the hundreds of technologists who so willingly gave their time to help us get it right.

It’s why I am so careful to stress that all enterprise blockchain platforms are not the same.

There is a reason Corda looks different to platforms like Fabric and Ethereum.  It’s because we’re not merely trying to take public blockchain technology – designed for a completely different purpose – and force it inappropriately into the enterprise. 

Instead, we’re doing something altogether more exciting and ambitious: we’re building the future of enterprise software. We’re tapping into one of the largest technology ecosystems ever seen, the Java ecosystem, and building on the work of countless thousands of skilled engineers to deliver an enterprise blockchain that’s designed directly to solve real problems faced by businesses today.

It’s why I believe the Corda enterprise blockchain is the future of enterprise software.

Indeed, firms like FinastraCalypsoHPE and others discovered Corda for themselves after reaching similar conclusions as us about what the right design for a consensus system between identifiable parties in a regulated environment needs to look like and our large and growing roster of partners, including Microsoft and Intel, is a testament to the momentum.

Perhaps I shouldn’t be surprised: we were immensely fortunate to have the deep and insightful input of literally hundreds of senior technologists from across dozens of the world’s largest companies help with the design. I hope one day to be able to list each and every one of them.

Indeed, I hope we will look back on Corda in a few decades as one of the largest and most successful collaborative design efforts in the history of enterprise technology!

Finally: a note on terminology. In this post I used the word ‘blockchain’ to describe Corda.  I tried for a long time to retain engineering purity (“Corda is very much like a blockchain and has chains of transactions but, strictly speaking, it doesn’t batch them into blocks – it’s real-time – so we probably shouldn’t call it a blockchain!”) But the reality is that the market uses the term Blockchain to describe all distributed ledger technologies, including ours. So I’m not going to fight it any more…!