What Slack Can Teach Us About Privacy In Enterprise Blockchains

“Channels” in Hyperledger Fabric don’t work the way you think they do…

Corda and Fabric have very different approaches to delivering privacy. In this post, I compare the models, explain why Corda works the way it does and why I think the Fabric privacy model is flawed. It turns out this can have real-world costly business implications.

But first… let’s set up some intuition.

If you’ve ever used the popular messaging tool, Slack, you might recognise this message…

Slack

If you add a new member to a private channel in Slack, you have two choices: share your entire history or start a fresh, empty, channel. This works for interpersonal comms but it turns out it doesn’t work nearly as well for the “trust but verify” world of enterprise blockchains. 

This message reveals a fundamental truth: if you’ve shared lots of information in private with some people – on Slack or in an email thread, perhaps – then you have to be very careful about adding somebody new to that group, especially if you care about controlling what they can and can’t see. Remember that time you added somebody to a long “reply-to” chain, only to realise there was something at the bottom that you really didn’t want them to see?!  Undo! Undo!! Undo!!!

Famously, there’s no “undo” button on a blockchain, so we have to get things right first time.

In this piece I’ll explain how Fabric’s privacy design turns out to be very similar to Slack channels. However, it also turns out that a model that works superbly for Slack doesn’t work as well in the world of enterprise blockchains for some very common use-cases.

But first, some history.

When we began our architectural journey at R3, we examined a large number of platforms. We concluded that none met our needs and we embarked on the project that culminated in Corda, the industry’s only finance-grade enterprise blockchain platform.

One of the platforms we included in that initial evaluation and rejected as unsuitable for the broad range of needs of sophisticated financial institutions was the first version of the Fabric platform.

But that was then… and a lot of time has passed. It’s always valuable to revisit past decisions in the light of new evidence and, since Fabric has just reached an important milestone, now is a good time to look again.

One of the key changes since 2015 is the introduction of something called “channels”, intended to address the severe privacy shortcomings in the initial design. It turns out that Fabric channels are very similar to an idea we had considered and rejected as too limiting at the start of the design process for Corda.

In this post, I explain why we rejected this design for Corda and what I think some of the key problems will prove to be.

As we know, early blockchain designs sprayed data around the network and everybody received and processed every transaction. This is how Bitcoin and Ethereum work and it is, of course, a fundamental part of how they work.   It’s the right design for those public blockchain platforms.

But that design, which is perfect for those platforms, is just not appropriate for most problems in today’s enterprise world. So the first version of Fabric, which broadcast data globally like many other platforms of the time, had to be extensively redesigned.

The new solution adopted by Fabric is called “channels”. The idea is effectively to let you set up many, many “mini blockchains” – each of which is called a “channel”.  So you and I may share a channel. Perhaps Alice, Bob and Charley share another one. And maybe Alice, me and Ivor share another one.  It’s as if there are many little private blockchains where members of a channel can see everything in that channel but nobody else can.

Simple, right? Elegant, right?

Unfortunately, no.

My biggest worry about the design when we first considered – and rejected – it for Corda is that assets will get stranded.

Imagine you issue a bond to an investor in a private channel between you and them. Remember: the whole point is that it’s private so you wouldn’t want anybody else in that channel. Why would you want them to know about your private deal?

And now you have that channel with your investor, you can use it to engage in some other transactions with them too.  Perhaps you use this bilateral channel to manage some records pertaining to some other deal you’re working on together.  Or maybe they’re also a customer of yours and you want to manage a complex order.  There will invariably be lots of different pieces of information in a channel – different deals, trades, records – all being managed together and commingled.  And this will be repeated across all the other bilateral and multilateral channels in which you participate.

And, because of the nature of the programming and consensus model used by Fabric-style blockchains, all information in a channel inevitably gets commingled with all other information in that channel.  There’s no easy way to extract just some pieces, with history and provenance… a channel is an all or nothing proposition. This is intrinsic to how these sorts of blockchains work and is a reason Corda uses a totally different architecture based on individual “states” representing specific shared facts, each of which can evolve independently.

A good way to think about this problem is as if a channel is like a break-out room at a conference… filled with whiteboards and sticky-notes on the wall. If you’re in the room, you can see and understand everything… but if you were to just take one piece of paper out of the room, it would make no sense to anybody else because they’d need the full history of everything that happened in the room and all the other papers to understand it.

Or, rather, I thought that was a good way to think about it until I sent an earlier draft of this article to some colleagues for review and one of them pointed out that this is precisely what happens when you manage private conversations in Slack!

If you set up a private channel in Slack and then try to add somebody else, they get to see everything that went before.  Or… you have to set up a brand new channel where they get no history, no context, no provenance.  It turns out the Slack app even has a perfect error message that describes this issue:

Slack

Slack knows why a channel architecture is problematic for a distributed ledger

So imagine you want to take a bond you’ve bought from the issuer in that channel and sell it to somebody else.  How would you do it?

  • Well…. you can’t invite them into the channel, because then they’d see all your other private information. A non-starter.  It would be like inviting them into your secret breakout room and hoping they didn’t look at things they weren’t supposed to.
  • And you can’t easily just extract the pieces of history needed to prove the history of that bond, because everything is commingled.
  • And you can’t simply tell them you own the bond. Why would they believe you? The whole point of enterprise blockchains is that each party verifies the information it is given. This is what distinguishes enterprise blockchains from databases, after all.
  • I suppose you could ask the issuer to cancel the issuance in your channel and reissue it in the new buyer’s channel.  But now we’re getting a little bit silly. This would be indistinguishable from simply managing the assets on the books of the issuer. It would defeat the point.

This is not just theoretical: it could have real-world impact.

If it is difficult to move assets between channels with provenance, one has to resort to cumbersome workarounds. Workarounds such as introducing “market makers” who sit between channels and maintain liquidity in both. But this has real costs: additional people to trust, additional fees, additional liquidity needs…

How is Corda different?

As I’ve written in other pieces, we spent a TON of time on Corda’s design: the data model, the fundamental conceptual framework and, critically, our solution to the thorny problem of how to assure privacy whilst allowing parties independently to validate chains of custody and other shared data… the essence of what makes a blockchain a blockchain and not just an expensive distributed database!

Our design addresses the problems in this article head on: data is shared at the level of individual deals or agreements or trades or contracts, with only the transactions needed to verify provenance being shared and no more.  On top of this we layer anonymization and other privacy-enhancing techniques. These techniques build on top of each other. The need to prove provenance never goes away but we do our absolute best only to share the data that is needed to satisfy the recipient.

What’s more, we also built Corda to be able to use Intel’s game-changing SGX technology – without any changes to apps and with Corda’s famous developer-friendly programming model.  So I was delighted that we could announce our partnership with Intel earlier this month.

I’m massively optimistic about the potential of blockchain technology to solve real problems in business. Just make sure you fully understand the pros and cons and different tradeoffs of each before making your selection: as always, one size never fits all.

Post-Script

I should stress that I have a boatload of respect for our friends at IBM – and elsewhere. I think channels are a poor architectural solution but I value immensely the collaboration we have via the Hyperledger Project (where we are both premier members) and beyond.  And I look forward to deepening this collaboration further.   There is more that unites projects such as ours than divides us!

And I should also point out that the Fabric team do know about these issues. For example, see this recent Stack Overflow question:

“How do we enforce privacy while providing tracing of provenance using multiple channels in Hyperledger v1.0?”

The answer is: “At the moment there is no straight forward way of providing provenance across two different channels within Hyperledger [Fabric] 1.0”.

And the answer goes on to reference a design document for a fix. That link is to a very long and complex design document. That tells me that the design problem may be pretty fundamental and can’t be fixed easily. But it’s good news that it is being looked at. We all benefit when platforms develop and evolve and I hope to see significant improvements in this area over time.

 

Update

2017-07-20

IBM’s Dan Selman has taken me to task about this post!  He correctly points out that I didn’t say too much about Corda’s design:

This is because I’ve written about it extensively elsewhere but he’s right: I should have linked.  This video from our Developer Relations team gives a pretty good overview:

And the other videos are pretty good too!

What that video doesn’t say (but should!) is the key point: in real-life scenarios, the dependency tree for any given transaction is invariably a very small subset of the overall set of transactions and so this technique (lazy on-demand provision of just the directly-required dependency tree and no more) gives us the optimal balance.  It is enabled by the transaction design, where each transaction specifically specifies which previous state objects (“shared facts”, if you like) are being superseded.   Put another way: we explicitly declare which parts of the shared state are being updated (actually, replaced) and so we know precisely which proof needs to be provided by one party to another.

In the Slack analogy, it would be equivalent to being able to automatically “lift out” just the pieces of a shared conversation that were directly relevant to the new person you wanted to add to the chat, without also showing them parts of the shared conversation that they have no need to see.

A Simple Explanation of Enterprise Blockchains for Cryptocurrency Experts

And why R3’s Open-Source Corda platform is the one to watch…

We’re doing some really interesting engineering at R3 right now… We have Java running in Intel SGX… We’re hacking a JVM to make it deterministic… We’ve proved you can suspend threads of execution to a database and bring them back to life across restarts as if nothing happened… (We even got emojis to display cleanly on multiple different terminals….)

“It’s just so nice to finally see people doing software engineering in this space” (unsolicited comment at a recent conference I attended)

But why are we doing all this work? After all, public blockchains are red hot right now: prices reaching new highs, “Initial Coin Offerings” showing no sign of slowing and new innovations announced daily. Why is a strange firm called R3, which recently raised $107m to complete the build out of our open source Corda platform, heading off in what seems like a different direction? Aren’t we just building over-complicated centralized databases? Or solving a problem that nobody has?

To be honest, I thought the public blockchain community didn’t have much interest in our work… until Joel from our developer relations team visited San Francisco to deliver our Corda technical training. He had been nervous: many cryptocurrency ‘maximalists’ are West-Coast based… and he thought he would be in for a hard time.

Anyway… he needn’t have worried. The audience was the best he’d ever presented Corda to: uniformly respectful, engaged, questioning and inquisitive. And it made me realise: I’ve done a terrible job of explaining what we’re up to and why we’re taking the route we’re are.

What problem do enterprise blockchains solve?

I wrote about this in more depth when we first announced Corda but, in short, the story is simple:

  • In the beginning there was Bitcoin…
  • … and it was a revelation.
  • Not only because, for the first time ever, we had a censorship-resistant, confiscation-proof, scarce, digital bearer asset…
  • … but because the architecture that Satoshi built to give us this amazing gift taught us something we didn’t previously know. It taught us that:
  • It is now possible to build systems that are operated by multiple parties, none of whom fully trust each other, that nevertheless come into and remain in consensus as to the nature and evolution of a set of shared facts.
  • In Bitcoin, the set of shared facts are: how many bitcoins have been mined and what conditions govern how they can be spent?
  • Newer platforms, such as Ethereum, build on these ideas and expand the set of facts over which we’re coming into consensus; in Ethereum’s case: what is the state of a shared world computer?

As we know, Satoshi set a very high bar for Bitcoin. It works when you don’t know who most of the participants are and it lets miners come and go at will without anybody even knowing who they are.  Like I’ve long said, Bitcoin is a work of genius.

But a key point to stress is that those of us building enterprise blockchain platforms aren’t trying to build a better Bitcoin or even build a better Ethereum. (Why bother? They already exist!) Instead, the thing that interests us is the sentence above that I wrote in italics:

It is now possible to build systems that are operated by multiple parties, none of whom fully trust each other, that nevertheless come into and remain in consensus as to the nature and evolution of a set of shared facts.

This is tantalizing… because it suggests we could completely transform the economics and structure of entire industries. Not by introducing a new currency or decentralized governance model (that’s already being built out by the public blockchains, after all).  But by also massively improving the efficiency of what already exists.

If we knew that all parties were in sync, we could accelerate securities settlement, optimize supply chains, liberate assets stranded in one silo for productive use elsewhere and more.  Anywhere where trade is hampered because of inconsistent systems could be in scope for improvement.  In short, we now have at our hands a new approach to solving one of the trickiest problems in transaction processing: the reconciliation problem.

If we could be sure that one firm’s IT systems were in perfect sync with their counterparts’ systems, it is mindblowing to imagine how much error, risk, duplication, complexity and cost could be eliminated… and how many hitherto impossible transactions became feasible.

In other words: quite separate to the well-understood revolution ushered in by the advent of Bitcoin, an entirely different field also got a massive kickstart. Two revolutions for the price of one.

It doesn’t sound as exciting as a new world currency, to be sure. But it could, in its own way, be utterly transformational. And note: the solution I’m talking about is not the same as a distributed database. And it’s not the same as a fully centralized solution. And it’s not another cryptocurrency or public blockchain. It is something new entirely.

In short, there’s a reason why enterprise blockchain firms like ours look, talk and act differently to cryptocurrency firms and communities: we’re building on some of the same technology, but solving different problems. Problems like managing trade finance relationships, confirming trades and issuing and trading Commercial Paper.

But don’t fall into the trap of thinking all enterprise blockchains are the same. Because they’re not. The Corda introductory whitepaper and Corda technical whitepaper go into this in more depth. But I know you’re busy. So let’s look at just three aspects.

Architecture

First, architecture.  We set ourselves the challenge of building an enterprise blockchain that met the needs for the most demanding clients in business: the financial services industry. It’s why it’s the only enterprise blockchain designed to support interoperable business networks: we want ecosystems to be able to transact and to trade. And it’s why we built a platform that could easily talk to existing business applications and which businesses could actually deploy.

That’s why Corda runs on the Java platform, stores its data, immediately queryable, in a relational database and moves data using message queues.  This facilitates integration, keeps operations people happy in big companies and massively simplifies the design.  Mike Hearn has spoken about this at length.  It sounds so simple – so traditional – yet it’s unbelievably valuable. We scratch our heads every day asking ourselves why everybody else in the enterprise blockchain world left this opportunity wide open for Corda to take!

Contrast that with Fabric, part of the Hyperledger Project: written in Go, it uses a gossip network to spray data indiscriminately around the network, with a cumbersome “channels” abstraction bolted on to fix that problem. And rather than default to a SQL database, it offers a strange choice between a key-value store or a NoSQL document database.

JPMorgan’s Ethereum-fork, Quorum, is still a work in progress so it’s hard to comment but it has a talented team behind it. So I’ll just note in passing that the problem that Ethereum solves so well in the public blockchain world is very different to the problem that large institutions have. So it’s not immediately obvious why you’d use the solution to one as the foundation of the solution to the other.

Privacy

Secondly, privacy.  Corda’s design, from day one, was based on an atomic, need-to-know privacy model that enables multiple different business networks and transaction types to co-exist and interoperate at the same time. Truly an interoperable network of networks is being built out.

Fabric’s design is very different. It has had at least two entirely different privacy designs in its short life and the latest, “channels”, suffers from all the same problems as other coarse-grained approaches to privacy: you end up with lots of mini-global-broadcast blockchains that don’t talk to each other and in which assets will get stranded; the opposite of the vision to which we’re building. You might get away with this design in some simple cases but will then come unstuck when you try to extend the solution.

Quorum’s approach is more innovative but I think it still fundamentally suffers from this problem.

Corda is built for the future

Finally, Corda is built for the future. When we designed Corda, we looked at where the broader industry was going and tried to anticipate those trends.  That’s why Corda is designed to work naturally and seamlessly with Intel’s SGX security technology. It’s why you can write your apps in any JVM language and why we chose to write the base platform in Kotlin, one of the most exciting new languages around – a decision which was vindicated when Google made it a first-class language for its Android platform – and it’s why Corda is jam-packed with cool little developer-friendly features that other platforms simply don’t have, such as our support for reactive programming and support for continuations, which we can automatically persist across JVM restarts.

To be fair to IBM, Fabric is also built for the future. If you assume the future runs on a Mainframe 🙂

Learning More about Corda

Corda is currently in public beta, we’ve just shipped our latest milestone release and you can download our one-click live DemoBench tool to get started in no time.

Or if you’d prefer to look at or contribute to the code, head over to our GitHub repo: https://github.com/corda/corda.

The team does its work in public… join the conversation at slack.corda.net!

 

Finally: a note on terminology. In this post I used Blockchain and Distributed Ledger interchangeably.  I tried for a long time to retain engineering purity (“Corda has chains of transactions but it doesn’t batch them into blocks so we probably shouldn’t call it a blockchain!”) But the reality is that the market uses the term Blockchain to describe all distributed ledger technologies, including ours. So I’m not going to fight it any more…!

 

Announcing The Corda Public Beta

Corda enters Beta this week!

As reported last month, R3 has just completed the first two of three tranches of our Series A funding round, raising $107m to fund the development of Corda, our distributed ledger platform designed from the ground up for the needs of commerce. It means all those people waiting for a technology that is fit for purpose now have certainty: Corda is here for the long-term and you can adopt it with confidence.

Corda is an open-source, financial-grade, distributed ledger platform with a difference: it applies the concepts of blockchain technology, while enabling strict privacy in an open, global network.

In this post, I announce our first public beta of Corda – and share some thoughts on an exciting announcement from our newest partner, HP Enterprise. Bottom line: if you haven’t already started building on Corda, now is the time to start.

Corda running on HPE NonStop

News on the beta in a moment but first some exciting news from our partners at HPE, who this week demonstrated Corda running on their HPE Integrity NonStop Platform. You can read more about the announcement here. This is a big deal: our vision for Corda is that companies use it to record, manage and automate the contracts and other agreements they have with their trading partners and counterparts.  That is a broad vision and entails deep integration with core systems. Proving deployment of Corda on HPE’s NonStop platform, on which so many of the world’s most trusted companies run their most important applications, is a major step forward.

Corda First Public Beta Releases This Week

Now back to the beta…

In the six months since the open sourcing of Corda in November 2016, our development team and extended community have been hard at work enhancing the codebase. This public beta represents another step forward in our path towards API stabilisation for production applications.  Our new docsite is here, the code is here and you can download the DemoBench here.

The beta is also a good point to reflect on the big steps forward since the open-sourcing of Corda:

  • An industry-leading vault – supporting soft-locking, direct SQL queries, automated population of relational tables and streaming updates in order to deliver the industry’s most productive DLT programming model.
  • BFT Notary Clusters – delivery of a Byzantine fault tolerant (BFT) decentralised notary, based on the BFT-SMaRT protocol and demo. Contrary to some mistaken press reports, Corda’s consensus mechanism is fully distributed…!
  • Corda Shell – enables developers and node administrators to easily command the node by running flows, RPCs, and SQL queries, and monitor the node’s activities. Type “flow watch” at the Corda shell and watch your node at work…
  • Inter-node workflows (the flow framework) – the industry’s only framework for automating communication and negotiation between nodes across the network. Flows now also support versioning, laying the ground-work for nodes to run multiple versions of the same flow at the same time, increasing support for backwards-compatibility. This is all part of making Corda the easiest DLT platform to manage in real production settings.
  • Identity – an implementation of X.509 identity standards, to enable interoperability with other systems, and support for confidential identities.
  • Increased performance – multiple performance upgrades, including higher memory efficiency, better RPC performance and ability to send larger messages.
  • The Corda Explorer and DemoBench – watch this video to learn how to run real Corda nodes on your own machine, with a single click, and see the power of the Corda architecture for yourself.
  • Security – readying the key management service to support the use of Hardware Security Modules for key storage.

We’ve also responded to feedback from the community to simplify how flows are registered and how developers achieve multiple signatures for a transaction – eliminating the need for 150 lines of code per flow! Keep the feedback coming and together we’ll make Corda even better.

We’ve been proving Corda with our members on our TestNet, the world’s first long-running, permissioned, general purpose distributed ledger network.  And we plan to open this up to everybody in the coming months. Join our Slack to join the conversation and watch our blog for announcements.

Corda has proven to be a hugely popular platform for building distributed ledger applications, with the Corda developer community growing extensively since the codebase was first open sourced – dedicated Corda Meetups all over the world, thousands of active members on our Slack, and developers trained all over the world through our oversubscribed training programmes.

If you haven’t seen Corda for yourself, this short video shows you what you’re missing. You can also watch our training videos, read our extensive ‘getting started’ documentation, and use the ground-breaking Corda DemoBench to help your colleagues and clients understand the value. All this and more is accessible via corda.net.

What’s next?

This beta is an important milestone. It represents the last major checkpoint before we stabilise our APIs and announce full version 1.0. Unlike some other codebases, “1.0” means something very significant to us: it will be the point at which we commit to API compatibility. You will be able to take apps you deploy on 1.0 and migrate them to run on future versions of Corda unchanged. That is a massively important and bar-raising promise to make and we take it seriously.  The team doesn’t get to say Corda is at 1.0 until it hits that bar.

But we also know that few other platforms hold themselves to that standard and Corda is already more mature, stable and productive than many other distributed ledger platforms out there.

And that’s why today’s Beta announcement is so important.  If you haven’t already started building your production Corda applications, now is the time to join all the other firms who are already building sophisticated offerings on Corda.

Start at corda.net, access the docs and sign up for training.

We love our partners

As should be clear by now, R3 is an enterprise software company, with the Corda platform at our heart.  And that means something important: it means opportunity for you.

If you’re a blockchain startup or a FinTech startup more generally or an independent software vendor or consulting firm or systems integrator, the Corda platform is the enterprise DLT solution for you and your clients.

  • We’re not building applications or “top of stack” solutions; that’s where you come in.
  • We’re not delivering complex solutions for clients; that’s where you come in.

So it’s no surprise that so many of the world’s startups, ISVs and SIs also see the power of a truly open platform and why they’re flocking to build on Corda.  We would love to partner with you too.

Email us at partner@r3.com.

Corda: Designed For Commerce, Engineered for Deployment

When we say Corda is designed specifically for enterprises, we mean it!

I’ve spent a lot of time with clients recently and it’s been thrilling to hear how so many of the unique design decisions we’ve made with Corda really resonate.

R3 has been building this new open-source distributed ledger platform in close collaboration with hundreds of senior technologists from across our global financial services membership. And that’s why Corda resonates with business people, because it was designed from the ground up to solve a real business problem: helping firms automate and manage their dealings with each other with legal certainty and without duplication, error and unnecessary reconciliation. Applying the essential insight of blockchain intelligently to the world of commerce.

Corda: inspired by blockchain systems but built from the ground up with the needs of today’s businesses in mind.

But Corda also resonates with technologists in these firms.  This is because we designed Corda to be deployable and manageable in the complex reality of today’s IT landscape. This sounds mundane but turns out to be critical, as I’ll explain in this article.

Corda: the only DLT platform that has been designed to make your IT department smile!

The core reason that Corda appeals to IT departments is simple: we’ve designed it so they can understand it, deploy it and manage it without having to unnecessarily rethink everything about how they operate. For example, Corda runs on the Java platform, it uses existing enterprise database technology for its storage, and it uses regular message queue technology to move data around.

These details seem small, but they turn out to be absolutely crucial: they mean enterprise IT departments already know how to deploy and manage Corda! It means that firms who select Corda will be able to get their solutions live so much quicker than those who mistakenly choose a different blockchain fabric.

No other DLT platform is as standards-compliant, interoperable or designed from the ground up to be deployed successfully into enterprise IT departments. And we’re not just talking about finance, by the way. Corda is applicable to every industry where needless duplication of data and process is prevalent: it turns out that if you can make it in finance, you can make it anywhere…

But this also leads us to another key point that explains why Corda is gaining so much interest: to get DLT projects live, multiple firms will have to move as one.

They will have to collaborate.

Corda is the product of R3, the largest-scale such collaboration the financial world has ever seen and this need for collaboration is hard-wired into its design. We’ve already discussed how Corda reuses existing standards wherever possible – massively simplifying the steps each firm seeking to deploy it needs to go through. But these insights go deeper. For example, there is usually a need to manage complex interfirm negotiations prior to committing a transaction, something enabled by Corda’s unique “flows” feature and entirely missing from other plaforms.

This need for collaboration is not restricted to large institutions themselves, of course. Getting complex DLT applications live requires partnership with implementation firms and software vendors.  Our obsession with collaboration is why Corda is so attractive to so many of these firms – our partners: they see that Corda is the right platform for business and in R3 they see a partner with collaboration in its DNA.

The reality is that there are actually very few fully open-source, credible, enterprise blockchain and DLT platforms, so when systems integrators respond to client requests for proposals, Corda is the one that many of them choose to bid.  This is not only because it is perfectly tailored for commerce but because it is the result of a genuine collaborative effort over which no one technology vendor, who may also be a competitor to them, has outsize influence: they can compete on a level playing field to serve their clients.

When you bring these strands together, it quickly becomes clear why Corda is appearing on everybody’s shortlist for projects right now.

Corda is the only enterprise DLT…

  • … designed from the ground-up to solve real business problems with privacy, scalability and legal certainty engineered in from day one.
  • … built to make the IT department smile
  • … with collaboration in its DNA: engineered to be deployed between firms in a practical and realistic way
  • … with a true ecosystem of partners competing to serve clients on a level playing field: no conflicts of interest, no fear of vendor lock-in.

As always, you can learn more about Corda at corda.net.  You can join our thriving community at slack.corda.net.  Our code is open-source and available at github.com/corda. And you can email us at partner@r3.com if you want to grow your business by building or deploying Corda solutions for your clients as one of our growing community of partners!

 

Countdown to Corda Open Source

R3 will soon be open-sourcing Corda. Here’s what to expect.

As I confirmed a few months back, R3’s Corda platform will be open-sourced, under the Apache 2 licence, on November 30.

Corda is a distributed ledger platform designed and built from the ground up for the recording and automation of legal agreements between identifiable parties. It is heavily influenced by the requirements of the financial industry but we believe the community will find the underlying architecture will lend itself to a broad range of applications.

We’ve built Corda because we see requirements – especially in finance – that need a distributed ledger but which cannot be met by existing platforms.

  • Corda is the only Distributed Ledger platform designed by the world’s largest financial institutions to manage legal agreements on an automatable and enforceable basis.
  • Corda only shares data with those with a need to view or validate it; there is no global broadcasting of data across the network.
  • Corda is the only Distributed Ledger platform to support multiple consensus providers employing different consensus algorithms on the same network, enabling compliance with local regulations.
  • Corda is designed to provide a great developer experience and to make integration and interoperability easy: query the ledger with SQL, join to external databases, perform bulk imports, and code contracts in a range of modern, standard languages.

We designed it with the members of R3, the world’s largest financial services DLT consortium, but we think its applicability is far broader.  You can find out more in our introductory whitepaper and my blog post on why we’re building Corda and what makes it different. If you prefer videos, here’s a short interview I did with Simon Taylor of 11:FS that explains the thought process behind Corda.

What we’ll release on November 30 is pretty much the full codebase as it exists today and we will be improving it actively and openly from then on. In fact, the only code we’ve held back pertains to laboratory projects we’re working on with our members and work on our own commercial business products that will run on top of Corda.

So do take a look around when the code is released: there’s a lot in there that is still work-in-progress and not yet integrated. For example, you’ll find a fascinating approach to writing financial contracts in the experimental branch and ongoing work on our deterministic sandbox for the JVM.   We will, of course, also be developing a commercial version of Corda for those who need specific enterprise features and support, but the open source codebase is the foundation of everything we do.

This is a really important point: distributed ledger technologies will have such phenomenally powerful network effects that it is unthinkable that serious institutions would deploy base-layer ledger software that is anything other than fully and wholeheartedly open. And it’s why we’ve been committed all along to releasing Corda just as soon as we were sure it was heading in the right direction.  It is and so we are.

We will also be publishing a draft of our technical whitepaper.  This whitepaper outlines our roadmap to version 1.0 of Corda and production-readiness.

What to expect on November 30

We’re really proud of Corda and its progress to date. But, that said, Corda is far from finished. Mike Hearn will soon be publishing a “warts and all” description of quite how much work we still have to do. This is true for all other platforms in this space, of course, but I feel a particular responsibility to be transparent given the ambitions we have for Corda and the uses to which it will be put.

By way of example, perhaps a good way to help you figure out what we still have to do is to look at some items on the list of work we’ve set for the months ahead of us:

  • Functional completeness: Corda still has gaps in its functional capabilities. The technical whitepaper outlines the full vision and you’ll see us working on and merging a lot of functional enhancements in the coming months to implement the full vision in the paper.
  • Non-functional characteristics: We focused first on design and then on implementation of Corda’s core functionality. The work to ensure we meet our non-functional requirements, such as performance, is still ahead of us but we have a clear roadmap and have designed the platform with these needs firmly in mind.
  • Security hardening: There are lots of areas where we need to tighten up security. Much of this we know about and we have called it out in the code or associated docs. But there will, of course, be others. So just as you shouldn’t be using other enterprise DLT platforms in production just yet, please don’t download Corda and put it straight into production just yet either!
  • API Stability: Corda’s development is iterative and organic – and it is heavily influenced by the range of projects and applications to which our members are choosing to put it. As we learn about common patterns and discover assumptions that prove to be wrong, we adapt. In particular, this means that we do not commit to API stability or backwards-compatibility until version 1.0.  Expect parts of the implementation to change in the coming months, perhaps quite significantly!

But these things are transient: we know how to fix them and we’ll knock the issues off one-by-one in the coming months as we head towards version 1.0.  But we want you to be fully aware of them.

Why are we open-sourcing Corda now?

We had a vigorous internal debate about when was the right time to release Corda: wait until it was more mature, when we were confident we’d ironed out the bugs and made it fly?  Or wait only until the design roadmap was clear and then share it immediately with the world for comment, criticism, contribution and collaboration?

We’ve wholeheartedly chosen the latter path: to release early and to work openly.

We’re serious about inviting the community to critique, collaborate and contribute. To take one example, our friends at Digital Asset recently published an excellent paper describing a set of requirements for what they call a “Global Synchronisation Log” (GSL), encouraging those in the community to incorporate these requirements into their platforms. We think that Corda’s vision is extremely well aligned to the GSL concept and by open-sourcing our work whilst there is still time to tweak our design it means we maximise the opportunities for firms such as ours to collaborate.

But open-sourcing Corda when it is still fairly young is not without its risks!  In fact, I’m a little apprehensive. I’m a completer-finisher and I obsess over every detail. So the idea of releasing something before it’s perfect makes me feel uncomfortable.  You will find gaps, issues, problems. But that’s fine: please do share what you find.  Even better, submit a fix…!

In fact, I also have a hope that some of those who come to critique will find that they nonetheless like much of what they see, and may even join the community.

What happens next?

I performed a thought experiment a while back… I asked: what will the enterprise distributed ledger world look like when everything settles down in a few years? How many independent enterprise DLT platforms will the world need and which ones will they be?

My conclusion was that there will probably be at most three such platforms, each carefully designed and adapted for a specific set of requirements. They will all be fully open source. And they will be surrounded by thriving, inclusive communities.

And we firmly intend to ensure Corda is one of them.

Our open-source release next week is a key step on that journey.

How to get Corda on November 30

Corda’s home will be corda.net.

Head over to corda.net on November 30 for links to the codebase, simple sample applications and a tutorial to get started writing your own CorDapps.

 

 

Introducing R3 Corda™: A Distributed Ledger Designed for Financial Services

UPDATE: The Corda introductory whitepaper is now available! And this blog post gives more context.

As reported in Bloomberg this morning, I’m delighted to confirm that R3 and our member banks are working on a distributed ledger platform for financial services: Corda™. I explain it on our official R3 blog and reproduce it here.

For the last six months, my team and contributors from our membership have been building a distributed ledger platform prototype from the ground up, specifically designed to manage financial agreements between regulated financial institutions. I am massively excited by the progress our team, led by James Carlyle, our Chief Engineer, and Mike Hearn, our Lead Platform Engineer, are making and I think the time is right to share some details.

Corda: A Distributed Ledger for Recording and Managing Financial Agreements

Corda is a distributed ledger platform designed from the ground up to record, manage and synchronise financial agreements between regulated financial institutions. It is heavily inspired by and captures the benefits of blockchain systems, without the design choices that make blockchains inappropriate for many banking scenarios.

Corda’s key features include:

  • Corda has no unnecessary global sharing of data: only those parties with a legitimate need to know can see the data within an agreement
  • Corda choreographs workflow between firms without a central controller
  • Corda achieves consensus between firms at the level of individual deals, not the level of the system
  • Corda’s design directly enables regulatory and supervisory observer nodes
  • Corda transactions are validated by parties to the transaction rather than a broader pool of unrelated validators
  • Corda supports a variety of consensus mechanisms
  • Corda records an explicit link between human-language legal prose documents and smart contract code
  • Corda is built on industry-standard tools
  • Corda has no native cryptocurrency

Corda’s design is the result of detailed analysis and prototyping with our members and will be open sourced when the code has matured further.

In the remainder of this post, I want to share some insight into our thinking.  Why are we building Corda?  Why have we made some of the design decisions we have?  When will the code be ready for others to examine and build upon? How does this relate to other platforms and projects?

A thought experiment

When I joined R3 from IBM in September 2015, I forced myself to stop and think.  The blockchain bandwagon was running at full speed, I’d just been appointed CTO of a project intended to bring blockchains to finance but there was a nagging worry at the back of my mind…  how could I avoid falling into the trap of believing all the hype?!

I imagined myself sitting in front of the CIO of one of our member banks some time in the future.  I imagined we had naively selected a “blockchain for finance” based on what was popular at the time and widely deployed a range of products and services on top of it. And I imagined we had believed the hype, had suspended our critical faculties and had omitted any engineering.  In this imagined scenario, I now found myself facing an angry CIO, who wanted to know why the system I had built had just failed calamitously. Why on earth did I build it the way I did?!

I concluded that an entirely inappropriate answer to that question would be: “because blockchains were cool in 2015”!  No. That simply won’t do.

The reality is that solutions based on selecting the design first and then trying to apply it to arbitrary problems never work out well.  Every successful project I’ve worked on started with the requirements, not some cool piece of technology, and I was determined to bring that discipline into our work at R3.

Remind me again why a system designed to replace banks is also supposedly their saviour?

And there is a second reason for this caution: the technology and finance industries collectively “decided” some time in early 2015 that “blockchain technology” was somehow the future of financial services.

Indeed, I am one of the most active proponents of precisely that claim. But the reason for blockchain technology’s importance is extremely subtle – and this subtlety is something that most people seem to have missed.

To understand this, we need to look at Bitcoin.

Bitcoin’s architecture, as I have often written, is a marvel.  Its interlocking components are one of those rare examples of something so elegant that they seem obvious in hindsight, yet which required a rare genius to create.

But what is often missed is that the cleverest part of Bitcoin isn’t actually its architecture; I think the cleverest part was to articulate the business problem.  We don’t tend to think of Bitcoin as being the solution to a “business problem” but it can perhaps be thought of as a wonderfully neat solution to the problem of: “how do I create a system where nobody can stop me spending my own money?”   Now, I can’t claim to know the mind of Satoshi and he certainly didn’t write the whitepaper in this way but it triggers a very useful thought-experiment.

In fact, once you write this ‘business problem’ down, the design drops out almost trivially!  (Almost…) You want always to be able to spend your own money? Then you can’t have a central point of control.  It could be shut down by the authorities.  You can’t even have a collection of validators with known identities as they could also be shut down with concerted effort.  Very quickly you realise you need a massively replicated consensus system and, if you don’t want to tie actions to real-world identities, you need something like Proof of Work to make the voting work.  You work the logic through and pretty much the whole design (the blockchain, the need for mining, block rewards, maybe even the UTXO transaction model, etc., etc.) drops out.  Of course, it does push a lot of work onto the users: confiscation of somebody’s bitcoins is easy if you know their private key… but let’s leave that to one side for now.

And this way of looking at it is important because it highlights how Bitcoin’s blockchain can be thought of as the solution to a business problem.    Satoshi Nakamoto didn’t wake up one morning wanting to “apply Blockchain to finance”.  Blockchain was the tool that was invented to solve a real problem.

So we have a conundrum, right?  If that’s the case, then what on earth is the argument that says blockchain has any relevance at all to banking?!

Indeed, last time I checked, banks have the inverse of my Bitcoin problem statement!

What is the defining characteristic of blockchain systems?

So I spent most of October sitting in a dark room (really! This was our first London office… a tiny four-person room in a shared working space in the City of London) questioning some of the most fundamental assumptions about blockchains.  What is it exactly that makes them interesting to banks?

Most people had already made the mental leap that the “bitcoin package” was unacceptable as a take-it-or-leave-it deal: proof of work is unnecessary for private deployments, for example.  But, as I looked around, all I could see was firms who had accepted everything else…  It seemed strange to me that, as an industry, we could tease apart one part of the “blockchain bundle” but then stop there.

I spent several of my earlier, formative years at IBM in a role called “technical sales”.  If you’ve ever bought technology from a large IT vendor, you’ll have met somebody like me.  We’re the people who visit clients with the sales rep and act as the technical expert: we explain how the product works, make sure we’re proposing the right solution to the client and ensure there is no technical barrier to closing the deal.

A lesson I learned very early in that role was: it doesn’t matter how hard you wish or how many client meetings you schedule or how aggressive the sales rep gets, if you can’t show how your solution is going to solve the client’s business problem then the deal almost certainly won’t close.  And those that do are the ones you’ll live to regret…

Fast forward a decade, and as I surveyed the blockchain landscape in October 2015, all I could see was excitable (and vocal!) firms touting solutions that made very little sense to me for the kinds of problems I was trying to solve.  I will confess to many moments of self-doubt:  maybe they were all sane and I was the mad one..?!

But I ploughed on: even if they are right that a “take it or leave it” blockchain design is the saviour of the financial industry, I’ll be doing our members a favour if I could explain why.

So we started picking away at what can perhaps be called the “blockchain bundle”:  the collection of services that blockchains provide to those who use them.

We concluded that a blockchain such as the ones underlying Bitcoin or Ethereum or any of the private variations actually provide at least five interlocking, but distinct, services.  And the right approach is to treat them as a menu from which to select and customise… different combinations, in different flavours, for different business problems.

CONSENSUS

The first, and most important, feature of blockchains – and the thing that is probably genuinely new in terms of scale and scope – is that they create a world where parties to a shared fact know that the fact they see is the same as the fact that other stakeholders see:

“I see what you see… and I know that what I see is what you see”

And, critically:

“I know that you know that I know”! 

And:

“I know that you know that I know that you know…”

And so on…

And it makes this promise across the Internet between mutually untrusting parties.  Sure: consensus systems and replicated state machines have existed for years but consensus systems at Internet scale, between untrusting actors, that work in the face of powerful adversaries? That’s a step forward.

In Bitcoin, the shared facts are things like: “What are all the bitcoin (outputs) that have not yet been spent and what needs to happen for them to be validly spent?”.  And the facts are shared between all full node users.

In Ethereum, the shared fact is the state of an abstract virtual computer.

But notice something interesting: there isn’t some law of nature that says the set of people who have to be in consensus is the whole world.  Bitcoin just happens to work that way because of its unique business problem.   If you don’t have Bitcoin’s business problem then be very wary of those trying to sell you something that looks like a Bitcoin solution.

VALIDITY

The second feature in the “blockchain bundle” is validity. Tightly linked to consensus, this feature is the one that allows us to know whether a given proposed update to the system is valid. It is how we define the rules of the game.  What does a valid “fact” look like in the system?  What does a valid update to that fact look like?

UNIQUENESS

The third feature in the blockchain bundle is its “uniqueness service”.   I can quite easily create two perfectly valid updates to a shared fact but if they conflict with each other then we need everybody who cares about that fact to know which, if either, of those updates we should select as the one we all agree on.  The “anti-double-spend” feature of blockchains gives us precisely this service and it’s hugely important.

IMMUTABILITY

The fourth feature in the “Blockchain Bundle” is often, if misleadingly, termed “immutability”: data, once committed, cannot be changed.

This isn’t quite true: if I have a piece of data then of course I can change it.  What we actually mean is that: once committed, nobody else will accept a transaction from me if it tries to build on a modified version of some data that has already been accepted by other stakeholders.

Blockchains achieve this by having transactions commit to the outputs of previous transactions and have blocks commit to the content of previous blocks.  Each new step can only be valid if it really does build upon an unchangeable body of previous activity.

AUTHENTICATION

The final critical feature in the “Blockchain Bundle” is authentication: every action in the system is almost always associated with a private key; there is no concept of a “master key” or “administrator password” that gives God-like powers.   This is quite different to traditional enterprise systems where these super-user accounts are prevalent and petrifying from a security perspective.

So what is the financial services business problem?

So why did I take us through this analysis?  Because it gets us to the heart of the distributed ledger domain: the thing that is genuinely new is the emergence of platforms, shared across the Internet between mutually distrusting actors, that allow them to reach consensus about the existence and evolution of facts shared between them.

So if that’s what this is all about, then what are the “shared facts” that matter in finance? What business problem would we need to have for any of this work to be of any use at all?

And this is the light bulb moment and the fundamental insight driving the entire Corda project:

The important “shared facts” between financial institutions are financial agreements:

  • Bank A and Bank B agree that Bank A owes 1M USD to Bank B, repayable via RTGS on demand.
  • This is a cash demand deposit
  • Bank A and Bank B agree that they are parties to a Credit Default Swap with the following characteristics
  • This is a derivative contract
  • Bank A and Bank B agree that Bank A is obliged to deliver 1000 units of BigCo Common Stock to Bank B in three days’ time in exchange for a cash payment of 150k USD
  • This is a delivery-versus-payment agreement
  •  … and so on…

The financial industry is pretty much defined by the agreements that exist between its firms and these firms share a common problem:  the agreement is typically recorded by both parties, in different systems and very large amounts of cost are caused by the need to fix things when these different systems end up believing different things. Multiple research firms have postulated that tens of billions of dollars are spent each year on this problem.

In particular, these systems typically communicate by exchanging messages: I send an update to you and just hope you reach the same conclusion about the new state of the agreement that I did.  It’s why we have to spend so much money on reconciliation to check that we did indeed reach the same conclusions and more money again to deal with all the problems we uncover.

Now imagine we had a system for recording and managing financial agreements that was shared across firms, that recorded the agreement consistently and identically, that was visible to the appropriate regulators and which was built on industry-standard tools, with a focus on interoperability and incremental deployment and which didn’t leak confidential information to third parties.  A system where one firm could look at its set of agreements with a counterpart and know for sure that:

“What I see is what you see and we both know that we see the same thing and we both know that this is what has been reported to the regulator”

That’s Corda.

How does Corda choose from the “Blockchain Bundle” Menu?

So now we understand the financial services requirement, we can look again at the “Blockchain Bundle” menu from above and outline the choices we’ve made.

CONSENSUS

A critical piece of the Corda philosophy is that our problem is to ensure that “I know that you see the same details about a shared fact that I see”.

But this does not mean that a third party down the road also needs to see it: our consensus occurs between parties to deals, not between all participants.

VALIDITY

Furthermore, in Corda, the only people who need to be in agreement about a fact are the stakeholders to that fact:  if you and I agree about something that pertains only to us then why should we care what some completely unrelated third party thinks?  And why would we even think of sending them a copy so they could opine on it? So, in Corda, we let users write their validation logic in time-tested industry-standard tools and we define who needs to be in agreement on a transaction’s validity on a contract-by-contract basis.

UNIQUENESS

Just like every other distributed ledger out there, we need to be sure that two valid, but conflicting, transactions cannot both be simultaneously active in the system.  But we also recognise that different scenarios require different tradeoffs. So Corda’s design allows for a range of “uniqueness service” implementations, one of which is a “traditional blockchain”. But it doesn’t need to be and, for our purposes, we also need implementations that make different tradeoffs under Brewer’s CAP theorem: in particular, some financial services use-cases need to prioritise consistency at the expense of availability in the event of a network partition.

IMMUTABILITY AND AUTHENTICATION

Here, Corda’s design departs very little from existing systems: our data structures are immutable and our building block is the exchange of digitally-signed transactions.

So Corda is very traditional in some respects – we directly apply the “authentication”, “immutability” and “uniqueness service” features of blockchains but we depart radically when it comes to the scope of “consensus” (parties to individual deals rather than all participants) and “validation” (the legitimate stakeholders to a deal rather than the whole universe or some arbitrary set of ‘validators’).

How is Corda Different?

Hang on?  Isn’t this the same pitch that every other blockchain firm is making? Not quite.

Notice some of the key things:  firstly, we are not building a blockchain.   Unlike other designs in this space, our starting point is individual agreements between firms (“state objects”, governed by “contract code” and associated “legal prose”).  We reject the notion that all data should be copied to all participants, even if it is encrypted.

Secondly, our focus is on agreements: the need to link to legal prose is considered from the start. We know there will still always be some disputes and we should specify right up front how they will be resolved.

Thirdly, we take into the account the reality of managing financial agreements; we need more than just a consensus system. We need to make it easy to write business logic and integrate with existing code; we need to focus on interoperability. And we need to support the choreography between firms as they build up their agreements.

Different Solutions for Different Problems

But… we should be clear.  We are not viewing Corda as a solution to all problems.  This model is extremely powerful for some use-cases but likely to be less well suited to others.  It’s why we continue to engage extremely deeply with all our partners who are working on complementary platforms in this space; we are not omniscient.  Moreover, there are still many significant design and research questions we have to resolve: there is still a great deal of work to do.

Furthermore, I have been deeply impressed by the quality engineering embodied in the many platforms that have passed through our labs and you will continue to hear about projects we are delivering on platforms other than Corda: different solutions for different problems is our mantra.  Indeed, those who have attended panels or workshops in recent months will have heard me saying this for some time now.

Corda does not seek to compete with or overlap with what other firms are doing:  indeed, we are building it because no other platform out there seeks to solve the problems we’re addressing.  That’s what makes this space so endlessly exciting.

What next?

In the coming weeks and months, you’ll hear more about Corda, about our initial projects and about its design.  We will also be gearing up to release the core platform as open source, possibly as a contribution to other endeavours.  Watch this space.

And… we’re still hiring: there is a great deal of work still to do!

How to explain the value of replicated, shared ledgers from first principles

“Digital currencies” aren’t needed to explain why distributed ledgers are important.

In this post, I develop an argument for replicated shared ledgers from first principles. It is intended to be an “education piece” aimed at those, particularly in the finance industry, who prefer explanations of new technologies to be rooted in a description of a real-world business problem rather than beginning with a description of a purported solution.  So, in this piece, you’ll find no mention of digital currencies, etc., because it turns out you don’t need them to derive an argument for distributed ledger technologies!

(Note to regular readers: see the end of the piece for some context)

We’ll start with banking systems

Start by thinking about today’s banking systems. In what follows, I use a bank deposit and payments example. But the same logic applies everywhere you look, as I’ll argue later.

Let’s imagine a world with three banks: Bank A, Bank B and Bank C and two customers, Customer A and Customer B. Each bank runs their own IT systems that they use to keep track of balances. This is a world very much like today.

So Bank A’s systems record the balances for Bank A’s customers, Bank B’s systems record the balances for Bank B’s customers and so on.

Perhaps the picture looks something like this:

Bank Systems 1

Balances at three banks for two customers.

Two immediate observations jump out:

  • First, look at Banks A and B. Bank A’s systems record that it is owed £1m by Bank B. And Bank B’s systems also record this fact: they record that Bank B owes £1m to Bank A. So the same information is recorded twice, by two independently developed, maintained and operated systems. And in other domains, this duplication is much greater and more expensive, as we’ll discuss below.
  • Secondly, look at Customer A. They are owed money by banks A and C and are overdrawn at Bank B. Put another way, Banks A and C owe money to Customer A. Who records this fact? Banks A and C! We take this situation for granted but it does seem very odd that Customer A has to trust both that the bank will be good for the money and that the bank’s records will be accurate. That feels like a conflict of interest, if ever there was one…

So we have two interesting phenomena: deposit-makers have to trust their banks to be good for the money and to account for things correctly. And the banks themselves have to spend a lot of time and money developing systems that all do pretty much the same thing – and then spend even more time and money checking with each other to make sure their systems agree on common facts.

Even in our simple example, there are potentially 7 separate matching entries to be verified.

Bank Systems 2

Banking “facts” are usually recorded by at least two different entities and an expensive process of reconciliation is needed to make sure each party’s view of the world is the same

It’s not just bank deposits. Securities and Derivatives Markets have the same pattern

This story is about bank deposits. But exactly the same story could be told about securities systems and derivatives systems. Indeed, in the latter case, the problem could be even worse: not only do we need to be sure everybody agrees on who has done which deals with whom, we also need to be sure that their systems agree on the resulting obligations that arise – they also have to agree on the business logic.

Think about how many near-identical systems exist across the financial landscape, each one working slightly differently and producing ever-so-slightly different results that have to be investigated and resolved. It’s hugely expensive.

Back to the banking story

But let’s focus on the banking example for now.

You can do something really interesting with the five ledgers we’ve been working with. You can write them a different way, with all the same information stored in a single table, rather than spread across five different tables:

Bank Systems 3

The five separate ledgers on the left can be written, exactly equivalently, as the single table on the right – and vice versa. You can derive one from the other. The only difference is that the table on the right has an extra column so we can record both the issuer and the holder of a claim .

In other words, rather than having a partial view of the world held by each bank, we could have a single table that records everything and achieve the same outcome.

So why not just have a single banking ledger for the world?

This raises an interesting question. If it’s so expensive and complicated for each bank to run its own system that contains its own narrow view of the world – and then have to check it matches the other systems where the facts overlap – why not just pay somebody to run a single ledger that everybody agrees will be authoritative?

After all, as we showed above, any bank that wanted to could easily derive its own view of the world from this mega-table, completely trivially.

Of course, we’d have to give thought to how to mediate access to the ledger – who is allowed to observe or update which records – but we know how to do that… and it’s not an impossible problem.

Are you mad?!

Now, it is tempting to say that such a thing would be insane: imagine how powerful would be the firm who ran such a system. And imagine the catastrophic implications for the world if there was a system outage! Perhaps the expensive, error-prone, but fundamentally decentralised and robust (anti-fragile?) system we have today is a price worth paying.

But this means an interesting question arises: what if there a way to achieve the benefits of a globally shared system but without having to grapple with the difficult political question of how to control an all-powerful operator or how to deal with the risk of an outage of such an important, central piece of infrastructure?

Perhaps we can achieve this…

The Replicated, Shared Ledger

Remember what we achieved in the diagram above: we created a single table that could describe all bank balances and which was inherently shared: different actors had different permissions to update different parts of it.

But the worry in the section above was that a shared global ledger would be controlled by a single powerful entity and that this centralized system could be a systemic risk. So can we make two tweaks to the model?

  • First, why not replicate the ledger massively. So, rather than one copy, have lots of copies. Perhaps one copy at every bank. So now there isn’t a single point of failure. We would have to worry about how those copies are kept in sync, of course, so this isn’t an unambiguous “win” but having copies at each bank might also make integration with existing infrastructure somewhat easier, too. Perhaps this would also help ease adoption.
  • Secondly, why not have those who participate in the system – maybe just the banks or maybe their customers too – also be jointly responsible for maintaining and securing it. We know who everybody else is in this world, after all, so we know whom to punish if they cheat. So we replace a single powerful entity with a model where everybody contributes to the system’s security.

If so, perhaps the picture would look like this:

Bank Systems 4

If a single copy of the global, shared ledger is undersirable or risky, then replicating it to all the participants could give the best of both worlds. Now the problem becomes one of automatically keeping the systems in sync rather than manually reconciling and dealing with breaks.

The picture above looks superficially like the one I drew at the start of the article. But there’s a critically important difference. In this model, all participants have a copy of the ledger but only have the right to amend entries pertinent to them. So it is both replicated and shared.

And so this is why I call this concept the “replicated, shared ledger”.  I think this wording is better at evoking the right mental model than “distributed ledger”, for example.

And depending on whether you want to model balances, other assets or even agreements between parties, there are startups working on a project.  I wrote a piece last year that attempted to make sense of the various players out there – and many more have emerged since then.

“Smart Contracts”

It it worth paying particular attention to the idea of adding business logic to this concept: so that the “facts” being recorded aren’t just who owns what but actual agreements between parties.

This opens up the intriguing possibility of “smart contracts”: a world where derivatives counterparties agree that a shared piece of code represents the agreement they have made with each other and they execute it on the shared, replicated ledger – perhaps completely eliminating the need to build, maintain, operate and reconcile their own proprietary derivatives platforms? Maybe even allowing the code to take custody of assets on the ledger, to manage cashflows and margin automatically?

Outstanding questions

But I should stress that this approach raises lots of technical questions: it’s not an unambiguously good idea. For example, do we know that the underlying replication technology works as described? Under all plausible threat scenarios? How can we be sure that one bank (or customer) can’t see (or amend…) another’s information? How much data would such a system hold? Would it scale? Is it really a good idea to model legal agreements in code rather than English?!

Conclusion

There do appear to be multiple examples of expensively duplicated systems in multiple areas of the banking system. The idea of a shared ledger holds promise, with replication by participants being a mechanism to reduce risk and mutualise its operation.  But whether this argument holds in practice needs to be tested. So I fully expect to see more and more experimentation by banks and others in the coming months and years.

Thank you: I’m extremely grateful to Lee Braine for input/review of the logical argument in this post

 

Note to regular readers

For the avoidance of doubt, in the piece above, I was not talking about Bitcoin – I’ll post a separate follow-up that attempts a derivation for Bitcoin’s design given some plausible real-world requirements; this post is about the domain I sometimes call the non-“Bitcoin-like-world”, as defined in this post)