Not all business blockchain platforms are alike. To succeed they need to reimagine business computing.

Lessons from the world’s largest multi-year collaborative Blockchain research programme

If you’re not part of the blockchain bubble, you probably think people in the blockchain world are all mad! Especially those of us in the enterprise space. How on earth could we believe a technology, originally built by a group of idealistic anarchists to solve a problem no financial institution actually has, could possibly be relevant to the challenges facing those trying to build and maintain complex IT infrastructures across the world?

In this article, I explain how the work we’ve done at R3 over the last two years with hundreds of senior technologists from dozens of leading financial institutions has led me to some fundamental conclusions:

  • The promise of blockchain technology is real: new business solutions will soon be deployed which can eliminate huge amounts of cost, redundancy, error and needless reconciliation across entire business ecosystems, as well as opening up previously hidden new revenue opportunities. I provide concrete examples below.
  • This will be achieved by applying the fundamental blockchain insight: “I know that what I see is what you see”. This is the key to helping us move from a world of isolated, custom, inconsistent IT infrastructures in each firm, to one based on shared business logic, securely shared data, and common processes.
  • The new IT architecture that is needed to capture these opportunities is one that converges the complex world of application servers, messaging engines, workflow managers and databases into a unified, secure and private design that works seamlessly within and across enterprises.
  • But not all blockchain platforms are alike: Only some designs will be architecturally suited to the challenge.
  • Our research has revealed several key characteristics that are needed to succeed. An enterprise blockchain platform must:
    • Provide an integrated application, messaging, workflow and data management architecture.
    • Build on an existing technology “mega-ecosystem” to maximise skills and code reuse.
    • Eliminate unnecessary data “silos” to unlock new business opportunities by allowing real-world assets to move freely between all legitimate potential owners.
    • Embed legal entity-level identification into the programming model to enable legally enforceable and secure transactions.
    • Support inter-firm workflows for negotiating updates to the shared ledger to support real-world business scenarios.
    • Enable the inevitable move to the public cloud without requiring high-risk technology bets.

For the last two years, I’ve been privileged to lead an intense research and development effort amongst the membership of the most vibrant consortium the financial world has seen. Hundreds of senior technologists from dozens of companies have worked with R3’s engineers to identify where blockchain technology can make a difference in large enterprises and also how it needs to be designed to achieve this potential.

This work has convinced me that the judicious application of key blockchain principles is key to rescuing the world’s banks and other large companies from the corner into which they have painted themselves after decades of investment in previous generations of technology.

In particular, we could be looking at the holy grail of enterprise software: convergence of application servers, messaging engines, workflow managers and databases in a way that works seamlessly within and across businesses.

The result is a model that delivers on the promise of blockchain in the enterprise: a world where applications can be securely developed and deployed across trading partners, enabling them to transact securely and accurately and without endless reconciliation, inconsistent data, and duplication. It is why we are now able to think about building or replacing payments systemssyndicated loans processing systems, asset issuance and custody systemsFX matching business processes and much more.  And these are just examples from the banking sector being addressed by just one platform, Corda, which itself turns out to be applicable far more broadly, in areas as diverse as healthcare and identity management.

But our belief is that it is only by converging previously diverse technologies that we can dramatically simplify how companies develop, deploy and manage applications that manage their business relationships with each other.

Here’s what I mean. Through one lens, a blockchain is like an application server: it hosts business logic and ensures it runs at the right time and for the right reasons. But a blockchain is also like a messaging engine: it allows people and, importantly, their computers to exchange information and ensure that the right information gets to the right places in the right order – and to do so between companies as well as within. And some blockchain platforms also have characteristics of workflow managers; they help coordinate the activity of different parties across time and space to achieve some business outcome. And, of course, they can be akin to databases.

If we could somehow devise a platform that built on this insight and delivered a platform that could combine the function of these hitherto diverse technologies, and could do so in today’s adversarial security environment, with the necessary levels of privacy, and in a way that was easy to use and could reuse rather than replace what already exists, imagine how much simpler our future enterprise IT landscape would look!

But building a platform that combines these separate concepts is easier said than done. There’s a reason existing vendors haven’t already built something that delivers on this vision.  In what follows, I’ll share the output of the last two years of our work, which confirmed to me that, by judiciously weaving together existing technologies and key advances from the blockchain world, it is indeed possible. I highlight some of the specific, key design choices you need to make in order to achieve this convergence and hence unlock the massive opportunity to transform the IT estates of existing companies.

I will refer at times to a key output from our work: the open-source Corda platform. We developed it in parallel with the research effort I describe above and, as a result, it benefited from a huge amount of expert input from across the R3 membership. No other enterprise blockchain platform has enjoyed remotely the same level of expert input. But I also refer to other enterprise blockchains where contrasts are helpful, paying particular attention to one that has come to prominence of late, Quorum, because it makes some very different choices and hence acts as a useful comparison point. This is intended to help make some of the points concrete and draw readers’ attention to areas of legitimate disagreement.

The ideal enterprise blockchain is a next-generation application server…

First and foremost, the opportunity that blockchain technology presents to enterprises is the ability to write applications that are shared between those who use them to transact. The key idea is this: rather than you and all of your trading partners each writing an expensive application to manage your participation in that trading network, you write it once. We thus share the cost and effort, whilst each running our own instance of the application in order to maintain our own books and records.

But to enable this vision, we concluded from our research that we need features that are unavailable in traditional application servers but which are common in the blockchain world. One such feature is cryptographic chains of provenance for all data in the system. When freely sharing relevant data with your trading partners, you must take nothing on trust but verify all proposed updates to the shared application state. This is also why all consensus-critical code on the platform needs to be digitally signed so you know that the code that is running is the code that should be running.

And we need to enable as much reuse of skills and existing technology as possible: you don’t save money by throwing away all the existing technology you have or by forcing all your staff to retrain! So you should be able to write applications in languages that have as large a population of developers as possible. Working with our diverse membership, we concluded that Java is by far the most widely deployed language across the business world and so we focused on that ecosystem. There are almost ten million people in the world who have these skills. But we also found that large enterprises hunger to bring their infrastructures right up to the contemporary cutting edge, so support for techniques such as reactive programming, approaches like functional programming and new languages like Kotlin and others are also required.

In short, the ideal application server for the modern era is one built for security, productivity, data integrity and cost minimisation.

But don’t all enterprise blockchains do this? Actually, no. To see that, we can compare the approach we took with Corda – the result of our two year journey – with a competing blockchain platform such as Quorum.  Although superficially similar to Corda in the way it describes itself it is, in fact, a relatively small fork of an existing public blockchain platform called Ethereum – which was designed to solve a completely different set of problems. Quorum inherits all of the associated advantages and disadvantages as a result.

For example, rather than supporting a full range of modern and mature languages to suit different needs, for which there are abundant skills and existing libraries, as Corda does through its support for the Java Virtual Machine, Quorum only supports languages that run on something called the Ethereum Virtual Machine. The EVM is a young, purpose-built virtual machine maintained by a community primarily focused on executing cryptocurrencytransactions and associated business logic. No mainstream languages run on the Ethereum virtual machine. This raises obvious questions around skills and integration with heritage systems. But it also opens up serious new risks for those who try to apply Quorum to real-world business problems. For example, the most popular language for writing applications on Quorum has been repeatedly shown to be impossible to use safelysuffering from countless bizarre language features that have led to people losing real money.

Not all enterprise blockchains are alike.

The ideal enterprise blockchain is also a next-generation messaging platform…

However, as I argued above, the exciting realisation during our research effort with our member banks was that good blockchain platforms are also inspiration for how to rethink enterprise messaging systems. Messaging systems are used by big companies to link computers and applications to each other securely and reliably.

The need for this becomes acute when you talk about deploying shared applications betweentrading partners. The applications running on different firms’ nodes need to be able to communicate. We need all parties to be securely identifiable and to be able to communicate with each other near-instantly over the internet or private network, addressing each other by legal name.  It’s not good enough to take existing enterprise messaging platforms that let you communicate with an address or a queue; you need to be able to talk to a named legal entity, anywhere in the world.

In our work with the diverse group of technologists from across our membership, we identified that the way to achieve this was to take existing technologies – in this case, TLS, X.500 and more, and make them consumable and accessible to today’s application developers. It seems obvious when you think about it yet no mainstream platform offered it in this way until we supported it in Corda. It was necessary to weave insights from the cryptography community, where public key infrastructure is well understood, and the banking community, where financial networks between identifiable peers are common, to come up with the idea of “send to legal entity” as the natural and obvious way to communicate between parties.

A separate finding from our work was that large institutions are institutionally allergic to anything that looked like it could create new islands of connectivity.  They needed a platform that would allow multiple independent Business Networks to operate within a single global namespace – perhaps something we could call a Compatibility Zone.  No more tolerance for isolated silos of communication and data distribution (unless you want it of course…). And this turned out to be key for the delivery of the holy grail: true representation of real-world assets on a shared network, with no artificial barriers to exchange or transfer, which I talk about more below.

Achieving this level of seamless interoperability across multiple business networks, whilst retaining flat, transparent, legal-entity-level addressing is hard. But I think we’ve achieved it with Corda.  And it’s something that is lacking in platforms like Quorum, where network membership is defined in text files which must be copied to each node, and where behaviour is undefined (or, rather, may have “adverse effects”) if they don’t match perfectly.

Identity and legal-entity addressing can’t be an after-thought in a text-file: they need to be designed in from the start.

The ideal enterprise blockchain is also a next-generation workflow manager…

When I worked with clients in my previous jobs, I would find they deployed workflow management tools, also sometimes called “Business Process Management” platforms, inside the organisation to control and optimise the flow of work between people and systems.

But all too often these systems would stop at the edge of the firm.  The “other side” was treated as a black box.  And yet, it is in the interaction between firms that the risk and cost creeps in.  Did they receive the message? Are they working on it? Did they understand it and process it correctly?

Do they see what I see?

This isn’t actually something that existing blockchain platforms provide much help for. And yet it is a real need even for some simple cryptocurrency transactions such as implementing an escrow arrangement or telling somebody how to pay you! In essence, for every meaningful transaction on a blockchain platform, there is usually an out-of-band negotiation flow that needs to take place.  Just like in real business.

And as we worked through countless use-cases during our research, what emerged was a need to make it easy for developers to write this back-and-forth business logic for the collaborative negotiation of a deal or construction of a transaction.

We discovered a need for an inter-firm workflow automation technology where developers don’t have to worry about any of the complex technical work necessary to orchestrate such co-operation at scale across a global network. In particular, it needs to be possible to define business processes that flow between, within and across firms, yet which can be coded in a simple and natural programming style, where you can express what each party in a transaction should do and have that work automated within and across your firms.

Unfortunately, developing something so ambitious would be hard if implemented from scratch. So it provided further weight to our emerging belief back then, a certainty today, that a successful blockchain platform for the enterprise has to build on the work of an existing, massive ecosystem, where it can reuse as much existing infrastructure and code as possible.

If we now turn to the specific case of Corda, we were able to take cutting edge technologies from the Java ecosystem and add some seriously clever engineering by the R3 engineering team to deliver something unique called the Flow Framework. In particular, recent advances in automatic checkpointing of business logic, with restore across system restarts, means developers can write what looks like entirely normal straight-line logic and yet, behind the scenes, a complex international workflow is being orchestrated. The first time you use it, it feels a bit like magic.  This is an example of why it’s so important to build on the foundations of an existing and robust ecosystem, such as that provided by Java, whilst at the same time bringing the programming model and tools right up to the contemporary cutting edge.

The ideal enterprise blockchain is also a next-generation asset ledger…

A repeated theme from our work with members was that the real wins come when the new enterprise blockchain applications become systems of record for real transactions.

This means they need to provide legal certainty.  Application developers need to link their contract code to associated contract legal prose to give certainty about how disputes will be managed.  But, for this to work, one must go further. Just as a successful enterprise blockchain identity layer needs to be based on legal names, the actual nodes – the machines running the business logic – also need to be clearly associated with a specific legal entity.

Such a framework, provided by the legal prose, identity-driven messaging and legal entity association with nodes, means that contracts signed on platforms that support it can be legally binding if you choose. It’s a prerequisite to being safely able, at scale, to support direct issuance of assets and the direct formation of contracts.

It means that a bank or other firm can issue assets (cash, equities, even commercial paper or syndicated loans) on to the ledger and have them be directly owned and seamlessly transferred to other members of an appropriate business network in near-real-time and with settlement finality.

And make no mistake: that is a hard trick to pull off.  For dematerialised assets to have full utility, they must be easily transferrable to any and all legitimate potential owners: isolated islands won’t do. So it turns out that this also has strong implications for the most fundamental aspects of the design: how data on the blockchain is managed, distributed and evolved, which I describe more below.

The issue we found during our research was that if you get this wrong, you risk trapping assets in silos, where they can only be transferred amongst members of a preconfigured list or through the involvement of third-party market-makers, or by asking the issuer to impose friction by cancelling and reissuing them.

It’s a tough problem to crack, as the link above makes clear in its analysis of another platform. The criticism in that link also applies to Quorum: they both suffer from this stranded asset problem and it means apps you write for those platforms will almost certainly have to be extensively rewritten in the future when those platforms are redesigned to correct these shortcomings. A costly prospect.

It’s one of the reasons we were so focused on delivering a version of Corda with “API Stability”: members and other developers told us loudly and clearly that they highly valued knowing they wouldn’t have to redesign their apps in the future

And the ideal enterprise blockchain is also first-of-a-kind decentralised database…

At the heart of the enterprise blockchain vision is the idea that, if two or more parties are transacting with each other, then they should all have an identical copy of the associated data.

“I know that what I see is what you see”.

To achieve this, we have to ensure that all parties have fully validated that their shared state was indeed calculated correctly.  At heart, it is this that distinguishes blockchains from what went before: we don’t take things on trust; we verify.  And to verify we need to run the logic for ourselves. We have to check the provenance of data.

What we want is, in effect, a globally shared database, but where each party only has the rows of the database that pertain to transactions they’re part of.  So we need to make sure that you receive the rows you need and if anybody tries to change any of them, they can only do so if the rules are followed and that you get to sign off on it and/or get notified afterwards as appropriate. To get an idea of how this works under one architecture, see this simple analogy.

The design we concluded that best achieves this is one that builds on and heavily extends and generalises ideas that originated in Bitcoin rather than Ethereum. This model, which Corda adopts, encourages developers to think of the fundamental units of information in their business problem and how they can evolve over time in isolation. Corda then adds the powerful flows I described earlier to orchestrate their updates via transactions that specify precisely which data units – “state objects” – are being referenced, created or replaced.

This approach allows for arbitrarily rich scenarios and sophisticated negotiations and business processes to be modelled with ease, whilst allowing the system to tell precisely who needs to receive what data, when, whilst revealing nothing more than is needed to prove provenance.

Other platforms take a different approach, based on an Ethereum-like model.  They typically start with the idea of an “everybody sees everything” full broadcast blockchain, representing a globally shared computer, and go fractal in order to correct for the privacy issues that result: they spin up tens or hundreds or thousands of separate mini shared-computers, one for each group that wants to transact in privacy. This approach can work but our analysis, confirmed by later experience, was that, for real world scenarios, it gets very complex, very quickly.

In the case of Quorum, each “confidential contract” can be thought of as a mini Ethereum universe that is visible only to the participants in that contract. This idea of millions of mini-blockchains works for some scenarios, but as I described in this post the approach, used by some other platforms too, fails completely if you ever need to support asset mobility. As soon as you need to prove provenance of one piece of data in a confidential contract to somebody else, you have to show them everything in that contract. Game over: you either lose privacy in the hunt for provenance or your assets are stranded, with their provenance impossible to be demonstrated to outsiders.

And… the ideal enterprise blockchain must also be designed for the cloud

The world doesn’t stand still. The enterprise software market in 2017 is different to the market in 2000…  In particular, most new applications that get deployed in the future will run in the cloud, increasingly in a public cloud.

So this was inevitably a hot topic of debate throughout the consortium’s deliberations over the last couple of years: how can we support application developers who need to write applications that will form legally binding contracts and be the basis of their audited books and records yet which will run on other people’s computers in a potentially hostile environment?

We considered the obvious options: zero-knowledge proofs, homomorphic encryption, secure hardware and more, and combinations of them all.

Our first conclusion was that the answer needs to be multi-layered: deterministic sandboxes, signing of all code and transactions, everything underpinned by legal-entity-level addressing, support only for a well-understood and tested managed runtime (in Corda’s case, the JVM, which we heavily lock down) and so forth.

But we also made a decisive choice: when weighing up risk, availability of skills, time to market and a collection of other factor, we concluded that the right initial approach is first to support hardware security technologies and, Intel’s SGX technology in particular.

This allows users to deploy Corda applications to other people’s computers (ie on public clouds) in a way that prevents those operators from interfering in transaction verification or accessing historical transaction information yet with no material limitation on the business logic that can be run. And we found that making this work well required it to be designed in from the ground up, which we did.

We concluded that this approach contrasted favourably with a higher-risk one being taken by platforms such as Quorum, who are recommending today the use of zero-knowledge proofs. Zero-knowledge proofs are a very fine piece of engineering and mathematics. They’re almost certainly part of the future of privacy in the long-term on blockchains – and Corda is designed to adopt them when they have matured.  But, today, the story is different.

First, and as Mike and Kostas in our engineering team discussed at CordaCon in September, the world of Zero Knowledge Proofs is young, very few people have skills, it is almost impossible to know what one of these things actually does when written and the security assumptions they rely on are unproven. Indeed, the “zk-SNARK” technology being promoted in February this year by the inventor of Ethereum is now being challenged by something new, called zk-STARKs This is a promising sign of a vital research community, rapidly advancing a young field.  But it’s a risky foundation upon which to build enterprise solutions before it has matured more.

Secondly, the integrity of the financial system rests on the idea of atomicity: related activities should either all happen or not happen at all. Inconsistent, half-complete exchanges just won’t do.

This requirement was hammered home to us by our members during the design process for Corda and so the ability to do complex atomic “payment versus payment” and “delivery versus payment” transactions is a fundamental part of the architecture.

Unfortunately, Quorum, which needs to use zero-knowledge proofs to achieve acceptable privacy in their system cannot, according to their own documentation, achieve delivery-versus-payment:

… the POC solution will not support cryptographically-assured DvP (i.e. atomic exchange of assets). This is because ZSL currently has no means of supporting shielded DvP-style functionality.”

This is why it’s so important to distinguish between the generic promise of enterprise blockchain technology and the practical reality of specific implementations.

Thirdly, we need to remember at all times that this whole emerging industry is about consensus, about being sure your counterparts really do see the same things you do and that you agree on all important data, when it happened and in what order.

But we learned from our members that how you reach this consensus may need to differ based on the business context: maybe you’re trading amongst a group of peers who you know well: you may each need to participate in the decentralised consensus forming process by running a node as part of a consensus cluster and reach consensus quickly and with finality amongst yourselves.  But, perhaps for some other line of business, you’re transacting with people all over the world and you want an independent, decentralised group of impartial observers to timestamp the transactions and help you reach agreement about ordering. That must also be supported: you need to be able to use a fully byzantine-fault tolerance decentralised cluster of consensus nodes, operated by mutually distrusting entities. And a successful platform needs to support all these modes – and more – and, here’s the key part: on the same network, at the same time!

So when we designed Corda, we engineered it so that you’re not forced to pick one consensus model and expect everybody to accept a one-size-fits-all for every transaction they perform on their blockchain network. That would be the road to disconnected, isolated blockchain networks… and the fast lane to stranded assets.

Why I believe Corda is the future of enterprise software

This piece began by outlining what I consider to be some of the critical findings of our journey so far. And we’ve embedded as many of them as possible into the design of the platform we jointly defined with this unprecedented consortium of institutions and the hundreds of technologists who so willingly gave their time to help us get it right.

It’s why I am so careful to stress that all enterprise blockchain platforms are not the same.

There is a reason Corda looks different to platforms like Fabric and Ethereum.  It’s because we’re not merely trying to take public blockchain technology – designed for a completely different purpose – and force it inappropriately into the enterprise. 

Instead, we’re doing something altogether more exciting and ambitious: we’re building the future of enterprise software. We’re tapping into one of the largest technology ecosystems ever seen, the Java ecosystem, and building on the work of countless thousands of skilled engineers to deliver an enterprise blockchain that’s designed directly to solve real problems faced by businesses today.

It’s why I believe the Corda enterprise blockchain is the future of enterprise software.

Indeed, firms like FinastraCalypsoHPE and others discovered Corda for themselves after reaching similar conclusions as us about what the right design for a consensus system between identifiable parties in a regulated environment needs to look like and our large and growing roster of partners, including Microsoft and Intel, is a testament to the momentum.

Perhaps I shouldn’t be surprised: we were immensely fortunate to have the deep and insightful input of literally hundreds of senior technologists from across dozens of the world’s largest companies help with the design. I hope one day to be able to list each and every one of them.

Indeed, I hope we will look back on Corda in a few decades as one of the largest and most successful collaborative design efforts in the history of enterprise technology!

Finally: a note on terminology. In this post I used the word ‘blockchain’ to describe Corda.  I tried for a long time to retain engineering purity (“Corda is very much like a blockchain and has chains of transactions but, strictly speaking, it doesn’t batch them into blocks – it’s real-time – so we probably shouldn’t call it a blockchain!”) But the reality is that the market uses the term Blockchain to describe all distributed ledger technologies, including ours. So I’m not going to fight it any more…!

Introducing Corda 1.0

Corda’s “API Stability” promise is an industry-first and means you can build on Corda with confidence. The roadmap for large-scale DLT deployment is now clear: it’s time to make your choice.

Corda 1.0 is here!

Corda – the world’s only distributed ledger platform designed and built from the ground up to record, manage and synchronise contracts and other shared data between trading partners – reached a critical milestone today: version 1.0 and, with it, core API stability.

API stability marks the point at which Corda’s contract with application developers is made firm: you can develop your applications on Corda 1.0 and, as we continue to enhance and mature the platform, you can upgrade with confidence, safe in the knowledge that the core APIs won’t change underneath you.

This is a critical landmark on the industry’s path to widespread adoption of enterprise blockchain technology and DLT and it is a promise that no other competing platform has made.

Build on Corda with confidence.

But why does this matter?

It matters because Corda solves a hitherto intractable problem in commerce.

The problem is easy to state and hard to solve. Every major firm around the world has built systems to manage their relationships and contracts with their trading partners. And each of their trading partners has done the same. Information is recorded by each of them multiple times, in multiple places, in multiple different ways and the information just never lines up properly. Each system is different. Each system has different bugs. And it costs an immense amount of money to keep them in sync and deal with the problems that arise when they’re not.

And this means that business leaders simply can’t move as fast as they need to in today’s world.

Here are some examples of the problem:

Corda is the only distributed ledger platform designed from the ground up to solve these problem by addressing the root cause.  With today’s version 1.0 release, we tell the world’s developers that it’s time to make your choice: build on Corda’s stable core API and create the next generation of distributed applications!

I wrote in 2015 that the wave of innovation that had been catalysed by Bitcoin was actually two phenomena: the emergence of decentralised crypto-assets and an entirely new way of solving a hitherto intractable problem in finance and commerce more generally. With Corda 1.0, this second blockchain revolution is now upon us.

With Corda, multiple parties who don’t fully trust each other can nevertheless collaborate to manage their shared data and this can have big implications for commerce.   It means we can envisage a future where I can look at my books and records and know, for sure, that what I see is what you see.

And if I can do that then we can transact in confidence, making business decisions in real-time, automating our joint activities with certainty: the promise of smart contracts. Facts recorded by the ledger can be regarded as authoritative rather than “shadows” of authoritative data held elsewhere, enabling settlement to take place directly across the platform.

Commerce without friction.

This is the opportunity Corda was designed for and, with Corda 1.0, the world’s developer community now has access to an open-source platform with a solid foundation that will take you with it as it continues to mature.

Corda’s unique design is the result of an intense period of research, development and design that included hundreds of senior technologists from across the global financial system, and the open source community, who have been actively engaged with Corda since we open-sourced it in November 2016.  Indeed, it was through our open source community that we discovered that Corda is applicable to far more than just finance! We’re seeing use-cases in government, insurance and beyond.

And, by working with experts and leaders at our extensive list of partners at firms such as Microsoft, HPE, Cognizant, Calypso, our community gains from the collective wisdom and shared learning that comes from a true collaborative community.

So why did the initiatives above (and so many more) choose to build on Corda, in many cases, switching to Corda after evaluating other technologies? We hear several reasons repeatedly given:

  • “My business problem is all about keeping records in sync with my trading partners and automating the activity that surrounds them.”
  • “My business dealings are complex; agreeing new terms requires negotiation… I need to be able to communicate back and forth with my trading partners.”
    • Corda’s unique flow framework makes it really simple to automate workflows between parties without writing complex event-driven code or dealing with asynchronous callbacks. No other platform has anything like Corda’s flow framework. The flow framework is what makes this decentralised, confidential netting solution so powerful.
  • “I need to integrate with existing systems easily.”
    • Corda writes its data directly into a relational database for you to query and uses well-understood and time-tested integration tools such as message queues (MQ) to move information around. Corda is built to integrate..!
  • “I need to deliver my solution quickly.”
    • Corda is designed for developer productivity. Developers can write their apps in Java – which over seven million developers know and the platform has been engineered for a thoughtful and delightful developer experience. This is enterprise software that developers actually like to use!
  • I don’t want to have to build the solution myself.”
    • Corda has a thriving ecosystem of software vendors and consultancies who have independently discovered the platform and are choosing to build their applications and delivery practices around it. Our partner team can introduce you to a firm who can meet your needs.
  • “I don’t want to be left on an evolutionary dead-end if I adopt DLT early.”
    • With Corda 1.0, you know your future is protected; you can upgrade to new versions of Corda and your applications won’t need to be extensively rewritten. What’s more, with 1.0, R3’s successful funding round, Corda’s large and growing open-source community and our extensive network of partners, Corda is now established as one of the few general-purpose DLT platforms that will still be standing when the market consolidates.
  • “I need my business dealings to be private.”
    • Corda is designed only to share data with those with a legitimate need to know – just the information needed to allow them to validate the provenance of facts on the ledger and no more: provenance with privacy. And Corda is designed from day one to work with Intel’s SGX privacy technology as it rolls out.

Get started with Corda today

So, if you haven’t already, now is the time to jump in to the Corda community and on LinkedIn.  The whole team is on slack.corda.net and you can get started here!

What Slack Can Teach Us About Privacy In Enterprise Blockchains

“Channels” in Hyperledger Fabric don’t work the way you think they do…

Corda and Fabric have very different approaches to delivering privacy. In this post, I compare the models, explain why Corda works the way it does and why I think the Fabric privacy model is flawed. It turns out this can have real-world costly business implications.

But first… let’s set up some intuition.

If you’ve ever used the popular messaging tool, Slack, you might recognise this message…

Slack

If you add a new member to a private channel in Slack, you have two choices: share your entire history or start a fresh, empty, channel. This works for interpersonal comms but it turns out it doesn’t work nearly as well for the “trust but verify” world of enterprise blockchains. 

This message reveals a fundamental truth: if you’ve shared lots of information in private with some people – on Slack or in an email thread, perhaps – then you have to be very careful about adding somebody new to that group, especially if you care about controlling what they can and can’t see. Remember that time you added somebody to a long “reply-to” chain, only to realise there was something at the bottom that you really didn’t want them to see?!  Undo! Undo!! Undo!!!

Famously, there’s no “undo” button on a blockchain, so we have to get things right first time.

In this piece I’ll explain how Fabric’s privacy design turns out to be very similar to Slack channels. However, it also turns out that a model that works superbly for Slack doesn’t work as well in the world of enterprise blockchains for some very common use-cases.

But first, some history.

When we began our architectural journey at R3, we examined a large number of platforms. We concluded that none met our needs and we embarked on the project that culminated in Corda, the industry’s only finance-grade enterprise blockchain platform.

One of the platforms we included in that initial evaluation and rejected as unsuitable for the broad range of needs of sophisticated financial institutions was the first version of the Fabric platform.

But that was then… and a lot of time has passed. It’s always valuable to revisit past decisions in the light of new evidence and, since Fabric has just reached an important milestone, now is a good time to look again.

One of the key changes since 2015 is the introduction of something called “channels”, intended to address the severe privacy shortcomings in the initial design. It turns out that Fabric channels are very similar to an idea we had considered and rejected as too limiting at the start of the design process for Corda.

In this post, I explain why we rejected this design for Corda and what I think some of the key problems will prove to be.

As we know, early blockchain designs sprayed data around the network and everybody received and processed every transaction. This is how Bitcoin and Ethereum work and it is, of course, a fundamental part of how they work.   It’s the right design for those public blockchain platforms.

But that design, which is perfect for those platforms, is just not appropriate for most problems in today’s enterprise world. So the first version of Fabric, which broadcast data globally like many other platforms of the time, had to be extensively redesigned.

The new solution adopted by Fabric is called “channels”. The idea is effectively to let you set up many, many “mini blockchains” – each of which is called a “channel”.  So you and I may share a channel. Perhaps Alice, Bob and Charley share another one. And maybe Alice, me and Ivor share another one.  It’s as if there are many little private blockchains where members of a channel can see everything in that channel but nobody else can.

Simple, right? Elegant, right?

Unfortunately, no.

My biggest worry about the design when we first considered – and rejected – it for Corda is that assets will get stranded.

Imagine you issue a bond to an investor in a private channel between you and them. Remember: the whole point is that it’s private so you wouldn’t want anybody else in that channel. Why would you want them to know about your private deal?

And now you have that channel with your investor, you can use it to engage in some other transactions with them too.  Perhaps you use this bilateral channel to manage some records pertaining to some other deal you’re working on together.  Or maybe they’re also a customer of yours and you want to manage a complex order.  There will invariably be lots of different pieces of information in a channel – different deals, trades, records – all being managed together and commingled.  And this will be repeated across all the other bilateral and multilateral channels in which you participate.

And, because of the nature of the programming and consensus model used by Fabric-style blockchains, all information in a channel inevitably gets commingled with all other information in that channel.  There’s no easy way to extract just some pieces, with history and provenance… a channel is an all or nothing proposition. This is intrinsic to how these sorts of blockchains work and is a reason Corda uses a totally different architecture based on individual “states” representing specific shared facts, each of which can evolve independently.

A good way to think about this problem is as if a channel is like a break-out room at a conference… filled with whiteboards and sticky-notes on the wall. If you’re in the room, you can see and understand everything… but if you were to just take one piece of paper out of the room, it would make no sense to anybody else because they’d need the full history of everything that happened in the room and all the other papers to understand it.

Or, rather, I thought that was a good way to think about it until I sent an earlier draft of this article to some colleagues for review and one of them pointed out that this is precisely what happens when you manage private conversations in Slack!

If you set up a private channel in Slack and then try to add somebody else, they get to see everything that went before.  Or… you have to set up a brand new channel where they get no history, no context, no provenance.  It turns out the Slack app even has a perfect error message that describes this issue:

Slack

Slack knows why a channel architecture is problematic for a distributed ledger

So imagine you want to take a bond you’ve bought from the issuer in that channel and sell it to somebody else.  How would you do it?

  • Well…. you can’t invite them into the channel, because then they’d see all your other private information. A non-starter.  It would be like inviting them into your secret breakout room and hoping they didn’t look at things they weren’t supposed to.
  • And you can’t easily just extract the pieces of history needed to prove the history of that bond, because everything is commingled.
  • And you can’t simply tell them you own the bond. Why would they believe you? The whole point of enterprise blockchains is that each party verifies the information it is given. This is what distinguishes enterprise blockchains from databases, after all.
  • I suppose you could ask the issuer to cancel the issuance in your channel and reissue it in the new buyer’s channel.  But now we’re getting a little bit silly. This would be indistinguishable from simply managing the assets on the books of the issuer. It would defeat the point.

This is not just theoretical: it could have real-world impact.

If it is difficult to move assets between channels with provenance, one has to resort to cumbersome workarounds. Workarounds such as introducing “market makers” who sit between channels and maintain liquidity in both. But this has real costs: additional people to trust, additional fees, additional liquidity needs…

How is Corda different?

As I’ve written in other pieces, we spent a TON of time on Corda’s design: the data model, the fundamental conceptual framework and, critically, our solution to the thorny problem of how to assure privacy whilst allowing parties independently to validate chains of custody and other shared data… the essence of what makes a blockchain a blockchain and not just an expensive distributed database!

Our design addresses the problems in this article head on: data is shared at the level of individual deals or agreements or trades or contracts, with only the transactions needed to verify provenance being shared and no more.  On top of this we layer anonymization and other privacy-enhancing techniques. These techniques build on top of each other. The need to prove provenance never goes away but we do our absolute best only to share the data that is needed to satisfy the recipient.

What’s more, we also built Corda to be able to use Intel’s game-changing SGX technology – without any changes to apps and with Corda’s famous developer-friendly programming model.  So I was delighted that we could announce our partnership with Intel earlier this month.

I’m massively optimistic about the potential of blockchain technology to solve real problems in business. Just make sure you fully understand the pros and cons and different tradeoffs of each before making your selection: as always, one size never fits all.

Post-Script

I should stress that I have a boatload of respect for our friends at IBM – and elsewhere. I think channels are a poor architectural solution but I value immensely the collaboration we have via the Hyperledger Project (where we are both premier members) and beyond.  And I look forward to deepening this collaboration further.   There is more that unites projects such as ours than divides us!

And I should also point out that the Fabric team do know about these issues. For example, see this recent Stack Overflow question:

“How do we enforce privacy while providing tracing of provenance using multiple channels in Hyperledger v1.0?”

The answer is: “At the moment there is no straight forward way of providing provenance across two different channels within Hyperledger [Fabric] 1.0”.

And the answer goes on to reference a design document for a fix. That link is to a very long and complex design document. That tells me that the design problem may be pretty fundamental and can’t be fixed easily. But it’s good news that it is being looked at. We all benefit when platforms develop and evolve and I hope to see significant improvements in this area over time.

 

Update

2017-07-20

IBM’s Dan Selman has taken me to task about this post!  He correctly points out that I didn’t say too much about Corda’s design:

This is because I’ve written about it extensively elsewhere but he’s right: I should have linked.  This video from our Developer Relations team gives a pretty good overview:

And the other videos are pretty good too!

What that video doesn’t say (but should!) is the key point: in real-life scenarios, the dependency tree for any given transaction is invariably a very small subset of the overall set of transactions and so this technique (lazy on-demand provision of just the directly-required dependency tree and no more) gives us the optimal balance.  It is enabled by the transaction design, where each transaction specifically specifies which previous state objects (“shared facts”, if you like) are being superseded.   Put another way: we explicitly declare which parts of the shared state are being updated (actually, replaced) and so we know precisely which proof needs to be provided by one party to another.

In the Slack analogy, it would be equivalent to being able to automatically “lift out” just the pieces of a shared conversation that were directly relevant to the new person you wanted to add to the chat, without also showing them parts of the shared conversation that they have no need to see.

A Simple Explanation of Enterprise Blockchains for Cryptocurrency Experts

And why R3’s Open-Source Corda platform is the one to watch…

We’re doing some really interesting engineering at R3 right now… We have Java running in Intel SGX… We’re hacking a JVM to make it deterministic… We’ve proved you can suspend threads of execution to a database and bring them back to life across restarts as if nothing happened… (We even got emojis to display cleanly on multiple different terminals….)

“It’s just so nice to finally see people doing software engineering in this space” (unsolicited comment at a recent conference I attended)

But why are we doing all this work? After all, public blockchains are red hot right now: prices reaching new highs, “Initial Coin Offerings” showing no sign of slowing and new innovations announced daily. Why is a strange firm called R3, which recently raised $107m to complete the build out of our open source Corda platform, heading off in what seems like a different direction? Aren’t we just building over-complicated centralized databases? Or solving a problem that nobody has?

To be honest, I thought the public blockchain community didn’t have much interest in our work… until Joel from our developer relations team visited San Francisco to deliver our Corda technical training. He had been nervous: many cryptocurrency ‘maximalists’ are West-Coast based… and he thought he would be in for a hard time.

Anyway… he needn’t have worried. The audience was the best he’d ever presented Corda to: uniformly respectful, engaged, questioning and inquisitive. And it made me realise: I’ve done a terrible job of explaining what we’re up to and why we’re taking the route we are.

What problem do enterprise blockchains solve?

I wrote about this in more depth when we first announced Corda but, in short, the story is simple:

  • In the beginning there was Bitcoin…
  • … and it was a revelation.
  • Not only because, for the first time ever, we had a censorship-resistant, confiscation-proof, scarce, digital bearer asset…
  • … but because the architecture that Satoshi built to give us this amazing gift taught us something we didn’t previously know. It taught us that:
  • It is now possible to build systems that are operated by multiple parties, none of whom fully trust each other, that nevertheless come into and remain in consensus as to the nature and evolution of a set of shared facts.
  • In Bitcoin, the set of shared facts are: how many bitcoins have been mined and what conditions govern how they can be spent?
  • Newer platforms, such as Ethereum, build on these ideas and expand the set of facts over which we’re coming into consensus; in Ethereum’s case: what is the state of a shared world computer?

As we know, Satoshi set a very high bar for Bitcoin. It works when you don’t know who most of the participants are and it lets miners come and go at will without anybody even knowing who they are.  Like I’ve long said, Bitcoin is a work of genius.

But a key point to stress is that those of us building enterprise blockchain platforms aren’t trying to build a better Bitcoin or even build a better Ethereum. (Why bother? They already exist!) Instead, the thing that interests us is the sentence above that I wrote in italics:

It is now possible to build systems that are operated by multiple parties, none of whom fully trust each other, that nevertheless come into and remain in consensus as to the nature and evolution of a set of shared facts.

This is tantalizing… because it suggests we could completely transform the economics and structure of entire industries. Not by introducing a new currency or decentralized governance model (that’s already being built out by the public blockchains, after all).  But by also massively improving the efficiency of what already exists.

If we knew that all parties were in sync, we could accelerate securities settlement, optimize supply chains, liberate assets stranded in one silo for productive use elsewhere and more.  Anywhere where trade is hampered because of inconsistent systems could be in scope for improvement.  In short, we now have at our hands a new approach to solving one of the trickiest problems in transaction processing: the reconciliation problem.

If we could be sure that one firm’s IT systems were in perfect sync with their counterparts’ systems, it is mindblowing to imagine how much error, risk, duplication, complexity and cost could be eliminated… and how many hitherto impossible transactions became feasible.

In other words: quite separate to the well-understood revolution ushered in by the advent of Bitcoin, an entirely different field also got a massive kickstart. Two revolutions for the price of one.

It doesn’t sound as exciting as a new world currency, to be sure. But it could, in its own way, be utterly transformational. And note: the solution I’m talking about is not the same as a distributed database. And it’s not the same as a fully centralized solution. And it’s not another cryptocurrency or public blockchain. It is something new entirely.

In short, there’s a reason why enterprise blockchain firms like ours look, talk and act differently to cryptocurrency firms and communities: we’re building on some of the same technology, but solving different problems. Problems like managing trade finance relationships, confirming trades and issuing and trading Commercial Paper.

But don’t fall into the trap of thinking all enterprise blockchains are the same. Because they’re not. The Corda introductory whitepaper and Corda technical whitepaper go into this in more depth. But I know you’re busy. So let’s look at just three aspects.

Architecture

First, architecture.  We set ourselves the challenge of building an enterprise blockchain that met the needs for the most demanding clients in business: the financial services industry. It’s why it’s the only enterprise blockchain designed to support interoperable business networks: we want ecosystems to be able to transact and to trade. And it’s why we built a platform that could easily talk to existing business applications and which businesses could actually deploy.

That’s why Corda runs on the Java platform, stores its data, immediately queryable, in a relational database and moves data using message queues.  This facilitates integration, keeps operations people happy in big companies and massively simplifies the design.  Mike Hearn has spoken about this at length.  It sounds so simple – so traditional – yet it’s unbelievably valuable. We scratch our heads every day asking ourselves why everybody else in the enterprise blockchain world left this opportunity wide open for Corda to take!

Contrast that with Fabric, part of the Hyperledger Project: written in Go, it uses a gossip network to spray data indiscriminately around the network, with a cumbersome “channels” abstraction bolted on to fix that problem. And rather than default to a SQL database, it offers a strange choice between a key-value store or a NoSQL document database.

JPMorgan’s Ethereum-fork, Quorum, is still a work in progress so it’s hard to comment but it has a talented team behind it. So I’ll just note in passing that the problem that Ethereum solves so well in the public blockchain world is very different to the problem that large institutions have. So it’s not immediately obvious why you’d use the solution to one as the foundation of the solution to the other.

Privacy

Secondly, privacy.  Corda’s design, from day one, was based on an atomic, need-to-know privacy model that enables multiple different business networks and transaction types to co-exist and interoperate at the same time. Truly an interoperable network of networks is being built out.

Fabric’s design is very different. It has had at least two entirely different privacy designs in its short life and the latest, “channels”, suffers from all the same problems as other coarse-grained approaches to privacy: you end up with lots of mini-global-broadcast blockchains that don’t talk to each other and in which assets will get stranded; the opposite of the vision to which we’re building. You might get away with this design in some simple cases but will then come unstuck when you try to extend the solution.

Quorum’s approach is more innovative but I think it still fundamentally suffers from this problem.

Corda is built for the future

Finally, Corda is built for the future. When we designed Corda, we looked at where the broader industry was going and tried to anticipate those trends.  That’s why Corda is designed to work naturally and seamlessly with Intel’s SGX security technology. It’s why you can write your apps in any JVM language and why we chose to write the base platform in Kotlin, one of the most exciting new languages around – a decision which was vindicated when Google made it a first-class language for its Android platform – and it’s why Corda is jam-packed with cool little developer-friendly features that other platforms simply don’t have, such as our support for reactive programming and support for continuations, which we can automatically persist across JVM restarts.

To be fair to IBM, Fabric is also built for the future. If you assume the future runs on a Mainframe 🙂

Learning More about Corda

Corda is currently in public beta, we’ve just shipped our latest milestone release and you can download our one-click live DemoBench tool to get started in no time.

Or if you’d prefer to look at or contribute to the code, head over to our GitHub repo: https://github.com/corda/corda.

The team does its work in public… join the conversation at slack.corda.net!

 

Finally: a note on terminology. In this post I used Blockchain and Distributed Ledger interchangeably.  I tried for a long time to retain engineering purity (“Corda has chains of transactions but it doesn’t batch them into blocks so we probably shouldn’t call it a blockchain!”) But the reality is that the market uses the term Blockchain to describe all distributed ledger technologies, including ours. So I’m not going to fight it any more…!

 

Announcing The Corda Public Beta

Corda enters Beta this week!

As reported last month, R3 has just completed the first two of three tranches of our Series A funding round, raising $107m to fund the development of Corda, our distributed ledger platform designed from the ground up for the needs of commerce. It means all those people waiting for a technology that is fit for purpose now have certainty: Corda is here for the long-term and you can adopt it with confidence.

Corda is an open-source, financial-grade, distributed ledger platform with a difference: it applies the concepts of blockchain technology, while enabling strict privacy in an open, global network.

In this post, I announce our first public beta of Corda – and share some thoughts on an exciting announcement from our newest partner, HP Enterprise. Bottom line: if you haven’t already started building on Corda, now is the time to start.

Corda running on HPE NonStop

News on the beta in a moment but first some exciting news from our partners at HPE, who this week demonstrated Corda running on their HPE Integrity NonStop Platform. You can read more about the announcement here. This is a big deal: our vision for Corda is that companies use it to record, manage and automate the contracts and other agreements they have with their trading partners and counterparts.  That is a broad vision and entails deep integration with core systems. Proving deployment of Corda on HPE’s NonStop platform, on which so many of the world’s most trusted companies run their most important applications, is a major step forward.

Corda First Public Beta Releases This Week

Now back to the beta…

In the six months since the open sourcing of Corda in November 2016, our development team and extended community have been hard at work enhancing the codebase. This public beta represents another step forward in our path towards API stabilisation for production applications.  Our new docsite is here, the code is here and you can download the DemoBench here.

The beta is also a good point to reflect on the big steps forward since the open-sourcing of Corda:

  • An industry-leading vault – supporting soft-locking, direct SQL queries, automated population of relational tables and streaming updates in order to deliver the industry’s most productive DLT programming model.
  • BFT Notary Clusters – delivery of a Byzantine fault tolerant (BFT) decentralised notary, based on the BFT-SMaRT protocol and demo. Contrary to some mistaken press reports, Corda’s consensus mechanism is fully distributed…!
  • Corda Shell – enables developers and node administrators to easily command the node by running flows, RPCs, and SQL queries, and monitor the node’s activities. Type “flow watch” at the Corda shell and watch your node at work…
  • Inter-node workflows (the flow framework) – the industry’s only framework for automating communication and negotiation between nodes across the network. Flows now also support versioning, laying the ground-work for nodes to run multiple versions of the same flow at the same time, increasing support for backwards-compatibility. This is all part of making Corda the easiest DLT platform to manage in real production settings.
  • Identity – an implementation of X.509 identity standards, to enable interoperability with other systems, and support for confidential identities.
  • Increased performance – multiple performance upgrades, including higher memory efficiency, better RPC performance and ability to send larger messages.
  • The Corda Explorer and DemoBench – watch this video to learn how to run real Corda nodes on your own machine, with a single click, and see the power of the Corda architecture for yourself.
  • Security – readying the key management service to support the use of Hardware Security Modules for key storage.

We’ve also responded to feedback from the community to simplify how flows are registered and how developers achieve multiple signatures for a transaction – eliminating the need for 150 lines of code per flow! Keep the feedback coming and together we’ll make Corda even better.

We’ve been proving Corda with our members on our TestNet, the world’s first long-running, permissioned, general purpose distributed ledger network.  And we plan to open this up to everybody in the coming months. Join our Slack to join the conversation and watch our blog for announcements.

Corda has proven to be a hugely popular platform for building distributed ledger applications, with the Corda developer community growing extensively since the codebase was first open sourced – dedicated Corda Meetups all over the world, thousands of active members on our Slack, and developers trained all over the world through our oversubscribed training programmes.

If you haven’t seen Corda for yourself, this short video shows you what you’re missing. You can also watch our training videos, read our extensive ‘getting started’ documentation, and use the ground-breaking Corda DemoBench to help your colleagues and clients understand the value. All this and more is accessible via corda.net.

What’s next?

This beta is an important milestone. It represents the last major checkpoint before we stabilise our APIs and announce full version 1.0. Unlike some other codebases, “1.0” means something very significant to us: it will be the point at which we commit to API compatibility. You will be able to take apps you deploy on 1.0 and migrate them to run on future versions of Corda unchanged. That is a massively important and bar-raising promise to make and we take it seriously.  The team doesn’t get to say Corda is at 1.0 until it hits that bar.

But we also know that few other platforms hold themselves to that standard and Corda is already more mature, stable and productive than many other distributed ledger platforms out there.

And that’s why today’s Beta announcement is so important.  If you haven’t already started building your production Corda applications, now is the time to join all the other firms who are already building sophisticated offerings on Corda.

Start at corda.net, access the docs and sign up for training.

We love our partners

As should be clear by now, R3 is an enterprise software company, with the Corda platform at our heart.  And that means something important: it means opportunity for you.

If you’re a blockchain startup or a FinTech startup more generally or an independent software vendor or consulting firm or systems integrator, the Corda platform is the enterprise DLT solution for you and your clients.

  • We’re not building applications or “top of stack” solutions; that’s where you come in.
  • We’re not delivering complex solutions for clients; that’s where you come in.

So it’s no surprise that so many of the world’s startups, ISVs and SIs also see the power of a truly open platform and why they’re flocking to build on Corda.  We would love to partner with you too.

Email us at partner@r3.com.

Corda: Designed For Commerce, Engineered for Deployment

When we say Corda is designed specifically for enterprises, we mean it!

I’ve spent a lot of time with clients recently and it’s been thrilling to hear how so many of the unique design decisions we’ve made with Corda really resonate.

R3 has been building this new open-source distributed ledger platform in close collaboration with hundreds of senior technologists from across our global financial services membership. And that’s why Corda resonates with business people, because it was designed from the ground up to solve a real business problem: helping firms automate and manage their dealings with each other with legal certainty and without duplication, error and unnecessary reconciliation. Applying the essential insight of blockchain intelligently to the world of commerce.

Corda: inspired by blockchain systems but built from the ground up with the needs of today’s businesses in mind.

But Corda also resonates with technologists in these firms.  This is because we designed Corda to be deployable and manageable in the complex reality of today’s IT landscape. This sounds mundane but turns out to be critical, as I’ll explain in this article.

Corda: the only DLT platform that has been designed to make your IT department smile!

The core reason that Corda appeals to IT departments is simple: we’ve designed it so they can understand it, deploy it and manage it without having to unnecessarily rethink everything about how they operate. For example, Corda runs on the Java platform, it uses existing enterprise database technology for its storage, and it uses regular message queue technology to move data around.

These details seem small, but they turn out to be absolutely crucial: they mean enterprise IT departments already know how to deploy and manage Corda! It means that firms who select Corda will be able to get their solutions live so much quicker than those who mistakenly choose a different blockchain fabric.

No other DLT platform is as standards-compliant, interoperable or designed from the ground up to be deployed successfully into enterprise IT departments. And we’re not just talking about finance, by the way. Corda is applicable to every industry where needless duplication of data and process is prevalent: it turns out that if you can make it in finance, you can make it anywhere…

But this also leads us to another key point that explains why Corda is gaining so much interest: to get DLT projects live, multiple firms will have to move as one.

They will have to collaborate.

Corda is the product of R3, the largest-scale such collaboration the financial world has ever seen and this need for collaboration is hard-wired into its design. We’ve already discussed how Corda reuses existing standards wherever possible – massively simplifying the steps each firm seeking to deploy it needs to go through. But these insights go deeper. For example, there is usually a need to manage complex interfirm negotiations prior to committing a transaction, something enabled by Corda’s unique “flows” feature and entirely missing from other plaforms.

This need for collaboration is not restricted to large institutions themselves, of course. Getting complex DLT applications live requires partnership with implementation firms and software vendors.  Our obsession with collaboration is why Corda is so attractive to so many of these firms – our partners: they see that Corda is the right platform for business and in R3 they see a partner with collaboration in its DNA.

The reality is that there are actually very few fully open-source, credible, enterprise blockchain and DLT platforms, so when systems integrators respond to client requests for proposals, Corda is the one that many of them choose to bid.  This is not only because it is perfectly tailored for commerce but because it is the result of a genuine collaborative effort over which no one technology vendor, who may also be a competitor to them, has outsize influence: they can compete on a level playing field to serve their clients.

When you bring these strands together, it quickly becomes clear why Corda is appearing on everybody’s shortlist for projects right now.

Corda is the only enterprise DLT…

  • … designed from the ground-up to solve real business problems with privacy, scalability and legal certainty engineered in from day one.
  • … built to make the IT department smile
  • … with collaboration in its DNA: engineered to be deployed between firms in a practical and realistic way
  • … with a true ecosystem of partners competing to serve clients on a level playing field: no conflicts of interest, no fear of vendor lock-in.

As always, you can learn more about Corda at corda.net.  You can join our thriving community at slack.corda.net.  Our code is open-source and available at github.com/corda. And you can email us at partner@r3.com if you want to grow your business by building or deploying Corda solutions for your clients as one of our growing community of partners!

 

The Corda Way of Thinking

The Corda Way of Thinking

Corda is a revolutionary new Distributed Ledger Platform, the only DLT specifically designed for the needs of financial services. This article introduces the “Corda Way of Thinking”: understand this article and you’ll be well on your way to being a Corda Expert Solution Designer…!

What problem are we trying to solve with Distributed Ledgers? 

The world is full of people who need to collaborate, trade and transact.  And, to do that, we need to know that the agreements – the contracts – that underpin their relationships are clearly documented, clearly understood and consistently recorded.

The promise of DLT is that we can all have our own records and yet somehow, as if by magic, they all stay in sync whenever somebody legitimately updates any of them.

Corda achieves this in a unique and massively powerful way. And this article explains it. And to prove how easy it is, we’re going to do it without computers…

Imagine we live in a world where all we have is paper, photocopiers and the postal service…  How could you keep a network of trading partners around the world in perfect sync with each other?

Building a Distributed Ledger with paper, photocopiers and the postal service

First, let’s imagine I have a filing cabinet filled with papers… each sheet represents a specific piece of information that I share in common with at least one other party… maybe it records a deal or a loan between us or is my healthcare history and you’re my doctor. Each piece of paper could record anything.

And you have a filing cabinet full of your papers … In fact, everybody has their own filing cabinet with their own papers.

And each paper is numbered – so it’s easy to find.

If you and I share a piece of information in common then we’ll both have a copy of that record: it will have the same number and contain precisely the same information.

picture1

In the diagram, we see that Richard and Albert both have a copy of record #128, Albert and Harrison both have a copy of record #140 and all three of them have a copy of record #132.

Each person’s filing cabinet only has papers that relate to their own business dealings with some other person or people. For example, Harrison doesn’t have a copy of record #128 because he is not involved in that deal.

OK – so that’s the set-up. I have a filing cabinet full of numbered papers, you have a filing cabinet of numbered papers, everybody has a filing cabinet of numbered papers.  Each sheet of paper represents a contract or agreement or deal or other interesting fact… and anybody who needs to have one has an identical copy.

Let’s look a little closer at record #128.   This is a record that I (Richard) and Albert both have.

picture2

Record #128 is a piece of paper that both Richard and Albert have.  It records a bet they have entered into.

It turns out that this piece of paper records a bet between me and Albert: if it rains in London on Wednesday, he owes me $10; if it doesn’t, I owe him $10. A weather report from a reputable newspaper will serve as proof. Pretty Simple. But note that the ideas we’ll be talking about work for more complex, multi-party situations, too.

So, as of right now, we’re in consensus. We both have the same details of the bet and are in agreement about it.

I know that what I see is what Albert sees.

picture3

Both Richard and Albert have a perfect copy of the record which records the existence of the bet and its current status (that we’re waiting to find out if it rained or not on Wednesday)

As you can tell, we’re starting in the middle of this story and you’re probably asking yourself how we came into consensus in the first place!  Don’t worry – we’ll get there!  It just happens to be easier to tell the story if we jump right into the middle.

Time passes…. And now it’s Thursday.  It’s time to resolve the outcome of the bet: did it rain yesterday or was it dry? We need to come to consensus on this and agree who owes the $10 to the other. And we need to update our records to record this updated information perfectly and consistently.

I can reveal that it… RAINED!  This means I win!!

And I have a report from the Wall Street Journal to confirm how wet it was. As I know you weren’t simply going to take my word for it…  And, as it happens, the terms of the bet demand that the winner provides proof to the loser for their records.

picture4

My newspaper confirms that it rained yesterday.  I will need to send a copy of this proof to Albert soon.

So we now have to update our shared record. We need to record that an external observation has been made, that it confirms that it rained and that you therefore owe me $10.

This is the key problem we’re trying to solve: bringing parties into consensus about the evolution of shared facts.

Our shared fact, of course, is the existence, nature and detail of a bet we’ve entered in to. And the evolution is that I now know that it rained and need to make sure that you know this and that you owe me the money.

Now – and this may seem a bit strange – we’re not actually going to edit or amend our pieces of paper; we’re going to create new records that completely replace the old ones.

That’s because in complex scenarios with huge numbers of events and updates, we’d be crossing things out all the time, making amendments and gluing new pieces of information at the bottom. It would be a total mess.  And we’d lose all ability to look back at history to remind ourselves what had happened in the past, with certainty that the historical record was completely tamper-free.

But that’s OK… paper is cheap… so it’s really no hardship to fill out a new blank sheet to record the new status of any agreement in its entirety from scratch each time something changes and replace the old version with this updated, newer version.

So what I’m going to do is fill in a brand new piece of paper with all the updated information of the bet.  And our challenge is to figure out a way to make this piece of paper replace the previous one as the dominant, current record of our deal for both me, Albert and anybody else who has a copy (perhaps a regulator or his accountant).

picture5

I know it rained and that I won the bet so I want our shared record (#128) to be replaced with an updated record (#156 in this case) that contains the latest information about the bet

So I pull out the original record, #128, from my filing cabinet – I’ll be putting a big red line through it later, and sending it away for archiving – and I start writing a letter to you.

In that letter, I reference this piece of paper – each piece of paper has a unique number at the top, remember. The letter goes something like this:

“Hello Albert!  Richard here.  Remember that bet we entered in to? You should find it in your filing cabinet under reference #128.  As you know, the bet related to yesterday and guess what? It rained in London! I’m attaching a copy of the newspaper report as proof. Sorry old chap, but that means you lost. So you owe me $10.  I’m attaching a new piece of paper that fully summarises the terms of our bet and memorialises that it did indeed rain and that this means you owe me $10.  I think you’ll find per the terms of the gambling rulebook we agreed to abide by mean you’re obliged to update your records with the attached sheet of paper.  I have done likewise.  You can pay me next time we meet.  Cheerio! Signed Richard”

picture6

I write a letter to Albert explaining that he needs to remove record #128 from his filing cabinet and replace it with record #156, which I’ve attached and which records that he now owes me $10. I include a copy of the newspaper report and put it all in a letter, which I post to Albert.

I send Albert a copy of this letter, with attached piece of paper recording the new state of the deal, and a copy of the newspaper report.  I might also send a full copy to a regulator if betting is regulated in my country and to Albert’s accountant if he also had a copy of #128… this doesn’t have to just be a bilateral conversation after all.  But let’s just focus on Albert for now.

When Albert receives the letter, he digs out his copy of the old record from his filing cabinet (it will have the same reference number) to remind himself about the details of the bet. He compares this record (#128) with the new one I’ve sent (#156), checks that the updates comply with the rules in the rulebook we’d agreed to use and, because it does, he puts a big red line through the old piece of paper and sends it away for archiving.  And he then adds the new piece of paper to his filing cabinet.

picture7

Record #128 has now been superseded by record #156

We both now have identical, updated versions of the contract in our filing cabinets.  And because we’d pre-agreed to use the same rulebook there can be no ambiguity: I had provided the proof necessary to move us to an updated version of the agreement. He makes a mental note to pay me next time we meet and we’re done.

Now, in this case, it was pretty simple. This example only required one letter to be sent:  mine to Albert.  The newspaper proof left him with little choice but to accept the updated record.   But you can imagine scenarios where Albert needs to reply back or maybe even write to somebody else before we can conclude that the new record has superseded the old one.  It’s these complexities that Corda handles for us – but which aren’t necessary for understanding the core concepts.

Now… if you’ve got this far and understood all the concepts then you know pretty much everything you need to know to build solutions on Corda.

Corda is deliberately and powerfully simple 🙂

So let’s map this example to how Corda actually works

What are the building blocks of a Corda solution?

picture2

  • All those pieces of paper representing deals, contracts, trades, balances, IOUs, loans?
    • These are Corda State Objects
    • Mental model: think of documents recording all the details pertaining to a single trade, balance, trade, agreement and so forth.
  • The filing cabinet?picture8
    • That is the Corda Vault
    • Mental model: think of the place where the most current versions of all
      your contracts and deals and trades and IOUs and bets are stored
  • Those covering letters telling your peers that you’d like them to put red lines through some pieces of paper and replace them with some new ones that you’ve attached with a paperclip and included in the envelope?
    • These are Corda Transactionspicture6
    • Mental model: think of them as letters from one party to all other interested parties suggesting it’s time to remove some papers from their filing cabinet and replace them with some new ones, which are attached to the letter, provided the recipients agree this complies with the rules we’d previously agreed.
  • The postal service?
    • That’s the Corda point-to-point messaging network
    • Mental model: think of it as being how information flows around the Corda network. We don’t send everything to everybody; just those with a need to know.
  • The rulebook that Albert used to check we’d done everything in accordance with the gambling rules we agreed to be subject to?
    • That’s Corda Contract Code
    • Mental model: think of this as being the test you apply whenever you get a new letter asking you to cross-out some pieces of paper and replace them with some new ones: is this letter asking me to do something that’s consistent with the rules I signed up to when I first entered into this agreement?  This is a fundamental component of Corda’s consensus architecture (the other being Consensus Clusters, discussed briefly below)

 

So that’s pretty much it:  pieces of paper, letters, photocopiers, the postal service. And yet it turns out to allow us to model all sorts of business problems and it lets us maximise privacy, scalability and performance: only the parties to deal need to process them and store them, for example.

Once these ideas click in your head, you’ll find yourself mentally trying to model all kinds of problems as Corda state objects, transactions and contract code. And you’ll start introducing some additional questions to your business analysis sessions.  You’ll ask questions like:

  • What’s written on the pieces of paper?
  • Whose responsibility is it to write the letters?
  • What business logic do they use to fill out the new pieces of paper and figure out which ones from the filing cabinet they’re going to replace?
  • To whom do they send them?
  • Can we do this with a single letter or do you need to provide additional information and reply to me? Do some steps require you explicitly to agree (by countersigning the letter?)
  • What rules do we need to have pre-agreed to use to check that the letters are asking us to do something valid?

And this, at heart, is the Corda Way of Thinking: a way of ensuring you’re in perfect synchrony with your trading partners that naturally and obviously maps to real-world ideas and which anybody can understand.

So, if you’ve tried to use other distributed ledger platforms and got frustrated at how quickly they got complex whenever you tried to do something that didn’t spray data to everybody or how you were expected to be a crypto expert, fear not:

Corda is different.  Corda is simpler.  Corda is better.

And now you know the secret of the “Corda Way of Thinking”.

Hang on… what about consensus algorithms??????

OK… 🙂  if you’ve studied other distributed ledger platforms, you may still have a couple of questions.  If you fall into that category, continue reading…

Consensus: How do we deal with the problem of two different letters crossing in the post, both referring to the same piece of paper in the filing cabinet? We can’t cross it out twice! So we need a way to choose which letter trumps the others.

  • Answer: Corda Consensus Services (aka “notary clusters”.)    In Corda, the question of whether an update is valid is purely a matter for those processing or verifying the transactions but we need to resolve conflicts if two transactions try to update the same record at the same time.  This is where Corda Consensus Services come in. Corda has a really innovative design here, too, allowing multiple Consensus Services on the same network, including consensus service clusters running  different consensus algorithms.   (Note: the Corda documentation also refers to notary clusters: this is the same concept, but consensus services is a more understandable name!)

Workflows: What do we do if I need a response from you before I can finish updating my filing cabinet? Do I just sit there, motionless, waiting?

  • Answer: Corda flows. These are also what we would have used to establish our first record of the deal, #128, where both of us would have needed to agree that the details of the bet were correct.

You can read more about both of these in our documentation!

Summary

I’m grateful to my colleague Chris Khan, for the awesome diagrams and Clemens Wan, for creating this table, which pulls it all together:

Analogy Component Responsibility Corda Component DLT Role
Filing Cabinet Keeps track of papers Vault Stores State Objects
Sheet of paper describing a contract or agreement or deal Represents specific facts or document State Object Stores data model and references to legal prose and contract code (the “rulebook”)
Newspaper Weather Report Provides weather at time of bet Oracle Third party trusted data source for the specific deal
Letter with cover note Tells other parties that somebody has calculated some updated papers, with evidence. Transactions Method of evolving the state objects as governed by contract code
Rulebook Provides rules that govern the bet Contract Code Provides verifications and rules that govern the state object’s evolution
Pinned paper to noticeboard Provides a copy of the cover note for others interested in the party to review and sign Corda Flow Manager Specifies transaction details for multi-party signature
Signature Proof that letter really did come from who it claims to be from Signature (digital) Proof that a transaction really did come from who it claims to be from – prevents repudiation
Postal Service Ensures letters are sent to the correct parties and delivered reliably Network Map Service & point-to-point messaging network Provides a reliable way of ensuring transactions get delivered to precisely the right parties and nobody else

 

 

Towards Deeper Collaboration in Distributed Ledgers: Thoughts on Digital Asset’s Global Synchronisation Log

It’s now almost two months since we open-sourced Corda and I’m delighted by the reception it has received. In our rapidly growing community, we’re already seeing new users grow into leaders who help other newcomers get to grips with the platform.

And I am amazed by the number of inbound messages from users who have been impressed by the quality of Corda’s design and codebase – and who are already building significant applications and products on top of it. Indeed, as I write this, one of our member banks is running a global hackathon, with over 150 of their developers building on Corda and I’ve just returned from our Asia Members’ Conference in Hong Kong, where I sat through so many presentations about Corda projects I didn’t even know were happening…

If you’re not already on our Slack or participating in our discussion forum, you’re missing out!

But one additional benefit from delivering on our commitment to make Corda Open Source is that it means we can explore opportunities to collaborate with peers, competitors and partners across the ecosystem: identifying areas where our visions are aligned, where we see things the same way and where we might be able to reuse rather than needlessly reinvent.

A good example of the potential for firms who some might see as competitors (but who actually aren’t…) to collaborate was provided late last year in the form of an excellent whitepaper from Digital Asset: The Global Synchronization Log.  The paper helps clarify some really important aspects of distributed ledger design and shows a really deep understanding of the tradeoffs that are inherent in the design of these platforms.

The first time I read the paper, I was struck by how closely our two firms’ visions for DLT are aligned. As Mike Hearn has written, there are two fundamentally different ways to design a DLT –“UTXO” or “replicated virtual machine” – and it was very encouraging when I realised that our two firms, completely independently, had both concluded that the correct architecture for a significant range of important financial services use-cases is the UTXO model.

This bears repeating: two firms who, in R3’s case, had worked with a huge consortium of financial institutions on a groundbreaking year-long Architecture Working Group and, in the case of Digital Asset, had begun delivery of implementations for clients, had reached extremely similar conclusions about what the “correct” architecture should look like.

But, in reading the paper, it was also clear that we had made some different decisions, too.  And the interesting thing is that the differences are almost all related to choices we’d made about acceptable tradeoffs.  As I’ve often written, there are no perfect solutions in DLT; just tradeoffs. But I will also freely admit that we made some additions to Corda’s technical vision in the light of the paper!

So it’s time, I thought, to share my thoughts on what I think are the key points in the paper and outline how I think Corda could be a perfect way to implement the concept.

What is the Global Synchronisation Log?

At the heart of this space is a beguilingly simple vision:

DLT allows me to build systems where “I know that what I see is what you see”

That is: if a computer system that I own and run and which exists to serve my needs tells me something about a deal you and I have done, I want to know that the system you’re looking at, that you own and run and which exists to serve your needs, is telling you the same thing.

Before Bitcoin and blockchains and Distributed Ledger Technology there were only two ways of doing this, neither of them perfect:  1) we could build a centralised infrastructure and just agree to agree that whatever they say is the truth… consensus by authority, if you like or 2) we could build our own systems and then spend all our lives checking that they had come to the same conclusion about everything… consensus by reconciliation.

Bitcoin and the systems it inspired showed us there was a third way:  we could use advances in cryptography, consensus algorithms and other technologies to give ourselves near total assurance that our systems were in sync without having to employ armies of people to check.

But there was a problem… and this problem is at the absolute heart of everything that’s going on in the DLT space today: the solution invented by Bitcoin and refined through subsequent systems depends on all data being shared with all parties.  So you gain something amazing on one hand: an end to errors, duplication, inconsistency and associated risk. But, on the other hand, you create a privacy nightmare and a system that goes slower the more things you use it for.

This is precisely the conundrum that motivated the invention and development of Corda. We decomposed the building blocks of existing blockchain platforms and reassembled them in the light of the different threat-model we have, the different use-cases and different tradeoffs we are prepared to accept.

One of the key insights in our work was that, for our scenarios, we can separate transaction verification from the question of whether two verified transactions conflict with each other. I wrote about this when we first announced Corda in April last year.

We think the question of transaction verification should be down to the transaction participants: if one of them pretends that their smart contract produced a different answer to what it actually did then we’ll deal with it out-of-band; it’s a permissioned system and we know who they are…   They gain nothing by playing games like that.

But the question of which transactions actually get confirmed is a question for an independent observer; we need somebody we all trust to choose between two equally valid but conflicting transactions. At R3, we call this observer a notary but that’s just the name we use to generalise the role performed by miners in a traditional blockchain.

In so doing, we addressed many of the privacy and scalability issues of other platforms at a stroke.

But it’s a tradeoff, of course.  Because there’s something that a full public blockchain gives you that this approach doesn’t. Both approaches assure you that only valid transactions can get confirmed, but a full public blockchain also ensures that everybody gets to know when this happens.

But, of course, a traditional blockchain does this by using full broadcast, in the clear, of pretty much everything that happens. A privacy and scalability disaster.

So we had some very heated debates when we designed Corda about which tradeoffs were acceptable and which ones were not. And the GSL paper touches on all of them really succinctly.

Two of the more important debates were as follows:

  • If I send a full transaction to a notary (think ‘miner’ in a traditional blockchain), that could be a privacy leak: the notary gets to see all the data in the transaction. But if I only send the pieces of the transaction that the notary actually needs to see in order to decide transaction ordering then I could execute a “denial of state” attack by having the notary confirm an invalid transaction that “consumes” an input and stops a valid transaction from subsequently being confirmed.
  • If I send a transaction to a notary, how does it know which other parties to inform? I could execute an attack whereby I get a transaction confirmed but the other side doesn’t learn about it… that might allow me to selectively choose not to reveal it if it so suits me.

In Corda, we made the following observations. We said:

  • The “notary privacy versus denial-of-state” question is one that should be solved on a case-by-case basis. So we support “validating notaries” that need to see all data and “non-validating notaries” that just see the subset that allows them to make a confirmation decision. But we require the non-validating notaries record who sent them the transactions they sign so we know who’s to blame if anybody does try to do something nefarious.
  • But the notification issue is more tricky: recall, the full-broadcast solution used in “traditional” blockchains just won’t cut it. Indeed, that’s why, in Corda, there is no global broadcast, by design.  So if a notary is going to inform you that something happened, it needs to know who you are and how to reach you. But that’s also a privacy issue if you implement it simplistically. So users effectively need the right to decide who they trust more: the notary or their counterparties.

So now to Digital Asset’s paper.   What they propose is very reasonable.  In essence, they say the following:

  • The Digital Asset GSL model is comfortable with the risk of a “denial of state” attack. (As are we at R3 for many scenarios, by the way, because the mitigations are robust; but Corda’s default mode is to protect against this threat).
  • So this means it’s fine if the notary only gets to see the subset of a transaction that is needed in order to determine ordering/uniqueness.
  • But GSL users are entirely not OK if a transaction can be confirmed and yet all the affected parties don’t get to hear about it at the same time as the transaction submitter.

And the paper goes on to explain how they think that last problem should be solved.

In essence, they do the following:

  • They effectively add the identities of all the parties who should know about the transaction to the outside of the transaction. This is the part that the notary sees.
    • They don’t actually put the interested parties’ identities on directly – that would be a privacy leak – but that’s the effect; you can think of them as “tagging” the transaction with the list of everybody who needs to know about it.
  • But that’s not enough, of course. The notary doesn’t get to see most of the transaction contents, remember… so the list could be wrong and the notary wouldn’t know! So they go a step further.
  • They add an additional rule to the transaction verification logic: if the transaction doesn’t “tag” the right set of intended recipients then it isn’t considered valid.
  • So now you have something pretty cool: you can get a transaction that fails to tag the right people notarised just fine (the notaries are ‘non-validating’ in the DA model, remember). But the “attacker” gains nothing because the transaction itself won’t be considered valid per the rules of the system.  So whatever nefarious scheme you were plotting fails…
  • And if you do construct a valid transaction then the act of getting it confirmed is also the irreversible act of having the notary inform all affected parties. So a bad guy doesn’t get to withhold valid, confirmed transactions.
  • This approach binds the question of transaction validity to the question of notification of affected parties.  You can’t have one without the other.

So you achieve something useful: transaction contents remain visible only to those who need to see them, transaction verification is in the hands of those to whom they pertain, notaries don’t see what they shouldn’t and if a transaction gets committed all relevant parties get to hear about it.  For a good number of use-cases, that’s a decent set of tradeoffs.

So can Corda provide a solution for the GSL?

(Spoiler: YES!)

It turns out that Corda’s design already has every single one of the features needed to implement the GSL – apart from one, which we added specifically to address this requirement.

  • Corda’s notaries already log the transaction submitters when operating in non-validating mode so we already solve the “denial of state issue” just fine.
  • Corda already supports “transaction tear-offs”, the mechanism whereby only the relevant information is shared with third parties such as notaries, using Merkle trees.
  • Corda already supports the concept of “participants” – aka“tags” – a list attached to each transaction that identifies interested parties
  • Corda’s transaction verification engine already allows contracts to verify that the participant list is correctly populated.

So we already have the mechanism to bind the verification to the population of the notification list.  But the “out of the box” design does not then ensure the notification actually happens…  This was a deliberate choice based on prioritisation of requirements and (yet another!) tradeoff around privacy.

In other words, there was one missing piece, albeit a deliberate one.  But reading this paper made us convinced adding that feature made sense and so we’ve added it to our design and will be added to the codebase in a future milestone release. The thinking is captured in section 7.5 of our technical whitepaper, starting page 33.

Note that our proposed implementation is slightly different to the design sketched in the Digital Asset paper because we deliberately and famously don’t have a blockchain: so there is no data structure that participants can passively browse to look for transactions of interest. Instead, we use a push point-to-point messaging network.  So the notary will directly inform affected parties.

Open Innovation: 2017 is the Year Corda Goes Mainstream

One of the many benefits of working on an open source project is that it becomes so easy and natural to explore these sorts of concepts with other firms, through initiatives such as the Hyperledger Project; through discussion of each other’s papers, like here; and through coding and direct collaboration between developers: we’re very much enjoying working with one of Digital Asset’s developers in our public Slack group, for example.

We think Corda is shaping up to be a perfect architecture for implementing the Global Synchronization Log concept and I am grateful to the team at Digital Asset for sharing their thinking – and their list of requirements – so openly and clearly.

Here’s to open innovation!

Countdown to Corda Open Source

R3 will soon be open-sourcing Corda. Here’s what to expect.

As I confirmed a few months back, R3’s Corda platform will be open-sourced, under the Apache 2 licence, on November 30.

Corda is a distributed ledger platform designed and built from the ground up for the recording and automation of legal agreements between identifiable parties. It is heavily influenced by the requirements of the financial industry but we believe the community will find the underlying architecture will lend itself to a broad range of applications.

We’ve built Corda because we see requirements – especially in finance – that need a distributed ledger but which cannot be met by existing platforms.

  • Corda is the only Distributed Ledger platform designed by the world’s largest financial institutions to manage legal agreements on an automatable and enforceable basis.
  • Corda only shares data with those with a need to view or validate it; there is no global broadcasting of data across the network.
  • Corda is the only Distributed Ledger platform to support multiple consensus providers employing different consensus algorithms on the same network, enabling compliance with local regulations.
  • Corda is designed to provide a great developer experience and to make integration and interoperability easy: query the ledger with SQL, join to external databases, perform bulk imports, and code contracts in a range of modern, standard languages.

We designed it with the members of R3, the world’s largest financial services DLT consortium, but we think its applicability is far broader.  You can find out more in our introductory whitepaper and my blog post on why we’re building Corda and what makes it different. If you prefer videos, here’s a short interview I did with Simon Taylor of 11:FS that explains the thought process behind Corda.

What we’ll release on November 30 is pretty much the full codebase as it exists today and we will be improving it actively and openly from then on. In fact, the only code we’ve held back pertains to laboratory projects we’re working on with our members and work on our own commercial business products that will run on top of Corda.

So do take a look around when the code is released: there’s a lot in there that is still work-in-progress and not yet integrated. For example, you’ll find a fascinating approach to writing financial contracts in the experimental branch and ongoing work on our deterministic sandbox for the JVM.   We will, of course, also be developing a commercial version of Corda for those who need specific enterprise features and support, but the open source codebase is the foundation of everything we do.

This is a really important point: distributed ledger technologies will have such phenomenally powerful network effects that it is unthinkable that serious institutions would deploy base-layer ledger software that is anything other than fully and wholeheartedly open. And it’s why we’ve been committed all along to releasing Corda just as soon as we were sure it was heading in the right direction.  It is and so we are.

We will also be publishing a draft of our technical whitepaper.  This whitepaper outlines our roadmap to version 1.0 of Corda and production-readiness.

What to expect on November 30

We’re really proud of Corda and its progress to date. But, that said, Corda is far from finished. Mike Hearn will soon be publishing a “warts and all” description of quite how much work we still have to do. This is true for all other platforms in this space, of course, but I feel a particular responsibility to be transparent given the ambitions we have for Corda and the uses to which it will be put.

By way of example, perhaps a good way to help you figure out what we still have to do is to look at some items on the list of work we’ve set for the months ahead of us:

  • Functional completeness: Corda still has gaps in its functional capabilities. The technical whitepaper outlines the full vision and you’ll see us working on and merging a lot of functional enhancements in the coming months to implement the full vision in the paper.
  • Non-functional characteristics: We focused first on design and then on implementation of Corda’s core functionality. The work to ensure we meet our non-functional requirements, such as performance, is still ahead of us but we have a clear roadmap and have designed the platform with these needs firmly in mind.
  • Security hardening: There are lots of areas where we need to tighten up security. Much of this we know about and we have called it out in the code or associated docs. But there will, of course, be others. So just as you shouldn’t be using other enterprise DLT platforms in production just yet, please don’t download Corda and put it straight into production just yet either!
  • API Stability: Corda’s development is iterative and organic – and it is heavily influenced by the range of projects and applications to which our members are choosing to put it. As we learn about common patterns and discover assumptions that prove to be wrong, we adapt. In particular, this means that we do not commit to API stability or backwards-compatibility until version 1.0.  Expect parts of the implementation to change in the coming months, perhaps quite significantly!

But these things are transient: we know how to fix them and we’ll knock the issues off one-by-one in the coming months as we head towards version 1.0.  But we want you to be fully aware of them.

Why are we open-sourcing Corda now?

We had a vigorous internal debate about when was the right time to release Corda: wait until it was more mature, when we were confident we’d ironed out the bugs and made it fly?  Or wait only until the design roadmap was clear and then share it immediately with the world for comment, criticism, contribution and collaboration?

We’ve wholeheartedly chosen the latter path: to release early and to work openly.

We’re serious about inviting the community to critique, collaborate and contribute. To take one example, our friends at Digital Asset recently published an excellent paper describing a set of requirements for what they call a “Global Synchronisation Log” (GSL), encouraging those in the community to incorporate these requirements into their platforms. We think that Corda’s vision is extremely well aligned to the GSL concept and by open-sourcing our work whilst there is still time to tweak our design it means we maximise the opportunities for firms such as ours to collaborate.

But open-sourcing Corda when it is still fairly young is not without its risks!  In fact, I’m a little apprehensive. I’m a completer-finisher and I obsess over every detail. So the idea of releasing something before it’s perfect makes me feel uncomfortable.  You will find gaps, issues, problems. But that’s fine: please do share what you find.  Even better, submit a fix…!

In fact, I also have a hope that some of those who come to critique will find that they nonetheless like much of what they see, and may even join the community.

What happens next?

I performed a thought experiment a while back… I asked: what will the enterprise distributed ledger world look like when everything settles down in a few years? How many independent enterprise DLT platforms will the world need and which ones will they be?

My conclusion was that there will probably be at most three such platforms, each carefully designed and adapted for a specific set of requirements. They will all be fully open source. And they will be surrounded by thriving, inclusive communities.

And we firmly intend to ensure Corda is one of them.

Our open-source release next week is a key step on that journey.

How to get Corda on November 30

Corda’s home will be corda.net.

Head over to corda.net on November 30 for links to the codebase, simple sample applications and a tutorial to get started writing your own CorDapps.

 

 

On Distributed Databases and Distributed Ledgers

Why can’t companies wanting to share business logic and data just install a distributed database? What is the essential difference between a distributed database and a distributed ledger?

Last month, I shared the thinking that led to the design of Corda, which we at R3 will be open sourcing on November 30; and Mike Hearn and I were interviewed by Brian and Meher of Epicenter last week. We’ve been delighted by the response and are looking forward to working with those seek to build on Corda, help influence its direction or contribute to its development and maturation;  there’s a lot of work ahead of us!

But one or two observers have asked a really good question. They asked me: “Aren’t you just reimplementing a distributed database?!”

The question is legitimate: if you strip away the key assumptions underpinning systems like Bitcoin and Ethereum, are you actually left with anything? What is actually different between a distributed ledger platform such as Corda and a traditional distributed database?

The answer lies in the definition I gave in my last blogpost and it is utterly crucial since it defines an entire new category of data management system:

“Distributed ledgers – or decentralised databases – are systems that enable parties who don’t fully trust each other to form and maintain consensus about the existence, status and evolution of a set of shared facts”

“Parties who don’t fully trust each other” is at the heart of this. To see why, let’s compare distributed databases and Corda.

Comparing Corda to a distributed database

In a distributed database, we often have multiple nodes that cooperate to maintain a consistent view for their users.   The nodes may cooperate to maintain partitions of the overall dataset or they may cooperate to maintain consistent replicas but the principle is the same:  a group of computers, invariably under the control of a single organisation, cooperate to maintain their state.  These nodes trust each other.   The trust boundary is between the distributed database system as a whole and its users.    Each node in the system trusts the data that it receives from its peers and nodes are trusted to look after the data they have received from their peers.  You can think of the threat model as all the nodes shouting in unison: “it’s us against the world!”

This diagram is a stylised representation of a distributed database:

 distributed-database

In a distributed database, nodes cooperate to maintain a consistent view that they present to the outside world; they cooperate to maintain rigorous access control and they validate information they receive from the outside world.

So it’s no surprise that distributed databases are invariably operated by a single entity: the nodes of the system assume the other nodes are “just as diligent” as them: they freely share information with each other and take information from each other on trust. A distributed database operated by mutually distrusting entities is almost a contradiction in terms.

And, of course, if you have a business problem where you are happy to rely on a central operator to maintain your records – as you sometimes can in finance it should be said – then a distributed database will do just fine: let the central operator run it for you.  But if you need to maintain your own records, in synchrony with your peers, this architecture simply won’t do.

And there are huge numbers of situations where we need to maintain accurate, shared records with our counterparts. Indeed, a vast amount of the cost and inefficiency in today’s financial markets stems from the fact that it has been so difficult to achieve this. Until now.

Corda helps parties collaborate to maintain shared data without fully trusting each other

Corda is designed to allow parties to collaborate with their peers to maintain shared records, without having to trust each other fully. So Corda faces a very different world to a distributed database.

A Corda node can not assume the data it receives from a peer is valid: the peer is probably operated by a completely different entity and even if they know who that entity is, it’s still extremely prudent to verify the information.   Moreover, if a Corda node sends data to another node, it must assume that node might print it all in an advert on the front page of the New York Times.

The trust boundaries – the red curves in the diagram- are drawn in a completely different place!

decentralised-database

In Corda, nodes are operated by different organisations and do NOT trust each other; but the outcome is still a consistent view of data.

To repeat, because this distinction is utterly fundamental:  nodes of a distributed database trust each other and collaborate with each other to present a consistent, secure face to the rest of the world.   By contrast, Corda nodes can not trust each other and so must independently verify data they receive from each other and only share data they are happy to be broadly shared.

And so we call Corda a distributed ledger, to distinguish it from distributed databases. A distributed ledger that is designed painstakingly for the needs of commercial entities.

Put more simply: you simply can’t build the applications we envisage for Corda with traditional database technology.  And that’s what makes this new field so exciting.