Announcing The Corda Public Beta

Corda enters Beta this week!

As reported last month, R3 has just completed the first two of three tranches of our Series A funding round, raising $107m to fund the development of Corda, our distributed ledger platform designed from the ground up for the needs of commerce. It means all those people waiting for a technology that is fit for purpose now have certainty: Corda is here for the long-term and you can adopt it with confidence.

Corda is an open-source, financial-grade, distributed ledger platform with a difference: it applies the concepts of blockchain technology, while enabling strict privacy in an open, global network.

In this post, I announce our first public beta of Corda – and share some thoughts on an exciting announcement from our newest partner, HP Enterprise. Bottom line: if you haven’t already started building on Corda, now is the time to start.

Corda running on HPE NonStop

News on the beta in a moment but first some exciting news from our partners at HPE, who this week demonstrated Corda running on their HPE Integrity NonStop Platform. You can read more about the announcement here. This is a big deal: our vision for Corda is that companies use it to record, manage and automate the contracts and other agreements they have with their trading partners and counterparts.  That is a broad vision and entails deep integration with core systems. Proving deployment of Corda on HPE’s NonStop platform, on which so many of the world’s most trusted companies run their most important applications, is a major step forward.

Corda First Public Beta Releases This Week

Now back to the beta…

In the six months since the open sourcing of Corda in November 2016, our development team and extended community have been hard at work enhancing the codebase. This public beta represents another step forward in our path towards API stabilisation for production applications.  Our new docsite is here, the code is here and you can download the DemoBench here.

The beta is also a good point to reflect on the big steps forward since the open-sourcing of Corda:

  • An industry-leading vault – supporting soft-locking, direct SQL queries, automated population of relational tables and streaming updates in order to deliver the industry’s most productive DLT programming model.
  • BFT Notary Clusters – delivery of a Byzantine fault tolerant (BFT) decentralised notary, based on the BFT-SMaRT protocol and demo. Contrary to some mistaken press reports, Corda’s consensus mechanism is fully distributed…!
  • Corda Shell – enables developers and node administrators to easily command the node by running flows, RPCs, and SQL queries, and monitor the node’s activities. Type “flow watch” at the Corda shell and watch your node at work…
  • Inter-node workflows (the flow framework) – the industry’s only framework for automating communication and negotiation between nodes across the network. Flows now also support versioning, laying the ground-work for nodes to run multiple versions of the same flow at the same time, increasing support for backwards-compatibility. This is all part of making Corda the easiest DLT platform to manage in real production settings.
  • Identity – an implementation of X.509 identity standards, to enable interoperability with other systems, and support for confidential identities.
  • Increased performance – multiple performance upgrades, including higher memory efficiency, better RPC performance and ability to send larger messages.
  • The Corda Explorer and DemoBench – watch this video to learn how to run real Corda nodes on your own machine, with a single click, and see the power of the Corda architecture for yourself.
  • Security – readying the key management service to support the use of Hardware Security Modules for key storage.

We’ve also responded to feedback from the community to simplify how flows are registered and how developers achieve multiple signatures for a transaction – eliminating the need for 150 lines of code per flow! Keep the feedback coming and together we’ll make Corda even better.

We’ve been proving Corda with our members on our TestNet, the world’s first long-running, permissioned, general purpose distributed ledger network.  And we plan to open this up to everybody in the coming months. Join our Slack to join the conversation and watch our blog for announcements.

Corda has proven to be a hugely popular platform for building distributed ledger applications, with the Corda developer community growing extensively since the codebase was first open sourced – dedicated Corda Meetups all over the world, thousands of active members on our Slack, and developers trained all over the world through our oversubscribed training programmes.

If you haven’t seen Corda for yourself, this short video shows you what you’re missing. You can also watch our training videos, read our extensive ‘getting started’ documentation, and use the ground-breaking Corda DemoBench to help your colleagues and clients understand the value. All this and more is accessible via corda.net.

What’s next?

This beta is an important milestone. It represents the last major checkpoint before we stabilise our APIs and announce full version 1.0. Unlike some other codebases, “1.0” means something very significant to us: it will be the point at which we commit to API compatibility. You will be able to take apps you deploy on 1.0 and migrate them to run on future versions of Corda unchanged. That is a massively important and bar-raising promise to make and we take it seriously.  The team doesn’t get to say Corda is at 1.0 until it hits that bar.

But we also know that few other platforms hold themselves to that standard and Corda is already more mature, stable and productive than many other distributed ledger platforms out there.

And that’s why today’s Beta announcement is so important.  If you haven’t already started building your production Corda applications, now is the time to join all the other firms who are already building sophisticated offerings on Corda.

Start at corda.net, access the docs and sign up for training.

We love our partners

As should be clear by now, R3 is an enterprise software company, with the Corda platform at our heart.  And that means something important: it means opportunity for you.

If you’re a blockchain startup or a FinTech startup more generally or an independent software vendor or consulting firm or systems integrator, the Corda platform is the enterprise DLT solution for you and your clients.

  • We’re not building applications or “top of stack” solutions; that’s where you come in.
  • We’re not delivering complex solutions for clients; that’s where you come in.

So it’s no surprise that so many of the world’s startups, ISVs and SIs also see the power of a truly open platform and why they’re flocking to build on Corda.  We would love to partner with you too.

Email us at partner@r3.com.

Corda: Designed For Commerce, Engineered for Deployment

When we say Corda is designed specifically for enterprises, we mean it!

I’ve spent a lot of time with clients recently and it’s been thrilling to hear how so many of the unique design decisions we’ve made with Corda really resonate.

R3 has been building this new open-source distributed ledger platform in close collaboration with hundreds of senior technologists from across our global financial services membership. And that’s why Corda resonates with business people, because it was designed from the ground up to solve a real business problem: helping firms automate and manage their dealings with each other with legal certainty and without duplication, error and unnecessary reconciliation. Applying the essential insight of blockchain intelligently to the world of commerce.

Corda: inspired by blockchain systems but built from the ground up with the needs of today’s businesses in mind.

But Corda also resonates with technologists in these firms.  This is because we designed Corda to be deployable and manageable in the complex reality of today’s IT landscape. This sounds mundane but turns out to be critical, as I’ll explain in this article.

Corda: the only DLT platform that has been designed to make your IT department smile!

The core reason that Corda appeals to IT departments is simple: we’ve designed it so they can understand it, deploy it and manage it without having to unnecessarily rethink everything about how they operate. For example, Corda runs on the Java platform, it uses existing enterprise database technology for its storage, and it uses regular message queue technology to move data around.

These details seem small, but they turn out to be absolutely crucial: they mean enterprise IT departments already know how to deploy and manage Corda! It means that firms who select Corda will be able to get their solutions live so much quicker than those who mistakenly choose a different blockchain fabric.

No other DLT platform is as standards-compliant, interoperable or designed from the ground up to be deployed successfully into enterprise IT departments. And we’re not just talking about finance, by the way. Corda is applicable to every industry where needless duplication of data and process is prevalent: it turns out that if you can make it in finance, you can make it anywhere…

But this also leads us to another key point that explains why Corda is gaining so much interest: to get DLT projects live, multiple firms will have to move as one.

They will have to collaborate.

Corda is the product of R3, the largest-scale such collaboration the financial world has ever seen and this need for collaboration is hard-wired into its design. We’ve already discussed how Corda reuses existing standards wherever possible – massively simplifying the steps each firm seeking to deploy it needs to go through. But these insights go deeper. For example, there is usually a need to manage complex interfirm negotiations prior to committing a transaction, something enabled by Corda’s unique “flows” feature and entirely missing from other plaforms.

This need for collaboration is not restricted to large institutions themselves, of course. Getting complex DLT applications live requires partnership with implementation firms and software vendors.  Our obsession with collaboration is why Corda is so attractive to so many of these firms – our partners: they see that Corda is the right platform for business and in R3 they see a partner with collaboration in its DNA.

The reality is that there are actually very few fully open-source, credible, enterprise blockchain and DLT platforms, so when systems integrators respond to client requests for proposals, Corda is the one that many of them choose to bid.  This is not only because it is perfectly tailored for commerce but because it is the result of a genuine collaborative effort over which no one technology vendor, who may also be a competitor to them, has outsize influence: they can compete on a level playing field to serve their clients.

When you bring these strands together, it quickly becomes clear why Corda is appearing on everybody’s shortlist for projects right now.

Corda is the only enterprise DLT…

  • … designed from the ground-up to solve real business problems with privacy, scalability and legal certainty engineered in from day one.
  • … built to make the IT department smile
  • … with collaboration in its DNA: engineered to be deployed between firms in a practical and realistic way
  • … with a true ecosystem of partners competing to serve clients on a level playing field: no conflicts of interest, no fear of vendor lock-in.

As always, you can learn more about Corda at corda.net.  You can join our thriving community at slack.corda.net.  Our code is open-source and available at github.com/corda. And you can email us at partner@r3.com if you want to grow your business by building or deploying Corda solutions for your clients as one of our growing community of partners!

 

The Corda Way of Thinking

The Corda Way of Thinking

Corda is a revolutionary new Distributed Ledger Platform, the only DLT specifically designed for the needs of financial services. This article introduces the “Corda Way of Thinking”: understand this article and you’ll be well on your way to being a Corda Expert Solution Designer…!

What problem are we trying to solve with Distributed Ledgers? 

The world is full of people who need to collaborate, trade and transact.  And, to do that, we need to know that the agreements – the contracts – that underpin their relationships are clearly documented, clearly understood and consistently recorded.

The promise of DLT is that we can all have our own records and yet somehow, as if by magic, they all stay in sync whenever somebody legitimately updates any of them.

Corda achieves this in a unique and massively powerful way. And this article explains it. And to prove how easy it is, we’re going to do it without computers…

Imagine we live in a world where all we have is paper, photocopiers and the postal service…  How could you keep a network of trading partners around the world in perfect sync with each other?

Building a Distributed Ledger with paper, photocopiers and the postal service

First, let’s imagine I have a filing cabinet filled with papers… each sheet represents a specific piece of information that I share in common with at least one other party… maybe it records a deal or a loan between us or is my healthcare history and you’re my doctor. Each piece of paper could record anything.

And you have a filing cabinet full of your papers … In fact, everybody has their own filing cabinet with their own papers.

And each paper is numbered – so it’s easy to find.

If you and I share a piece of information in common then we’ll both have a copy of that record: it will have the same number and contain precisely the same information.

picture1

In the diagram, we see that Richard and Albert both have a copy of record #128, Albert and Harrison both have a copy of record #140 and all three of them have a copy of record #132.

Each person’s filing cabinet only has papers that relate to their own business dealings with some other person or people. For example, Harrison doesn’t have a copy of record #128 because he is not involved in that deal.

OK – so that’s the set-up. I have a filing cabinet full of numbered papers, you have a filing cabinet of numbered papers, everybody has a filing cabinet of numbered papers.  Each sheet of paper represents a contract or agreement or deal or other interesting fact… and anybody who needs to have one has an identical copy.

Let’s look a little closer at record #128.   This is a record that I (Richard) and Albert both have.

picture2

Record #128 is a piece of paper that both Richard and Albert have.  It records a bet they have entered into.

It turns out that this piece of paper records a bet between me and Albert: if it rains in London on Wednesday, he owes me $10; if it doesn’t, I owe him $10. A weather report from a reputable newspaper will serve as proof. Pretty Simple. But note that the ideas we’ll be talking about work for more complex, multi-party situations, too.

So, as of right now, we’re in consensus. We both have the same details of the bet and are in agreement about it.

I know that what I see is what Albert sees.

picture3

Both Richard and Albert have a perfect copy of the record which records the existence of the bet and its current status (that we’re waiting to find out if it rained or not on Wednesday)

As you can tell, we’re starting in the middle of this story and you’re probably asking yourself how we came into consensus in the first place!  Don’t worry – we’ll get there!  It just happens to be easier to tell the story if we jump right into the middle.

Time passes…. And now it’s Thursday.  It’s time to resolve the outcome of the bet: did it rain yesterday or was it dry? We need to come to consensus on this and agree who owes the $10 to the other. And we need to update our records to record this updated information perfectly and consistently.

I can reveal that it… RAINED!  This means I win!!

And I have a report from the Wall Street Journal to confirm how wet it was. As I know you weren’t simply going to take my word for it…  And, as it happens, the terms of the bet demand that the winner provides proof to the loser for their records.

picture4

My newspaper confirms that it rained yesterday.  I will need to send a copy of this proof to Albert soon.

So we now have to update our shared record. We need to record that an external observation has been made, that it confirms that it rained and that you therefore owe me $10.

This is the key problem we’re trying to solve: bringing parties into consensus about the evolution of shared facts.

Our shared fact, of course, is the existence, nature and detail of a bet we’ve entered in to. And the evolution is that I now know that it rained and need to make sure that you know this and that you owe me the money.

Now – and this may seem a bit strange – we’re not actually going to edit or amend our pieces of paper; we’re going to create new records that completely replace the old ones.

That’s because in complex scenarios with huge numbers of events and updates, we’d be crossing things out all the time, making amendments and gluing new pieces of information at the bottom. It would be a total mess.  And we’d lose all ability to look back at history to remind ourselves what had happened in the past, with certainty that the historical record was completely tamper-free.

But that’s OK… paper is cheap… so it’s really no hardship to fill out a new blank sheet to record the new status of any agreement in its entirety from scratch each time something changes and replace the old version with this updated, newer version.

So what I’m going to do is fill in a brand new piece of paper with all the updated information of the bet.  And our challenge is to figure out a way to make this piece of paper replace the previous one as the dominant, current record of our deal for both me, Albert and anybody else who has a copy (perhaps a regulator or his accountant).

picture5

I know it rained and that I won the bet so I want our shared record (#128) to be replaced with an updated record (#156 in this case) that contains the latest information about the bet

So I pull out the original record, #128, from my filing cabinet – I’ll be putting a big red line through it later, and sending it away for archiving – and I start writing a letter to you.

In that letter, I reference this piece of paper – each piece of paper has a unique number at the top, remember. The letter goes something like this:

“Hello Albert!  Richard here.  Remember that bet we entered in to? You should find it in your filing cabinet under reference #128.  As you know, the bet related to yesterday and guess what? It rained in London! I’m attaching a copy of the newspaper report as proof. Sorry old chap, but that means you lost. So you owe me $10.  I’m attaching a new piece of paper that fully summarises the terms of our bet and memorialises that it did indeed rain and that this means you owe me $10.  I think you’ll find per the terms of the gambling rulebook we agreed to abide by mean you’re obliged to update your records with the attached sheet of paper.  I have done likewise.  You can pay me next time we meet.  Cheerio! Signed Richard”

picture6

I write a letter to Albert explaining that he needs to remove record #128 from his filing cabinet and replace it with record #156, which I’ve attached and which records that he now owes me $10. I include a copy of the newspaper report and put it all in a letter, which I post to Albert.

I send Albert a copy of this letter, with attached piece of paper recording the new state of the deal, and a copy of the newspaper report.  I might also send a full copy to a regulator if betting is regulated in my country and to Albert’s accountant if he also had a copy of #128… this doesn’t have to just be a bilateral conversation after all.  But let’s just focus on Albert for now.

When Albert receives the letter, he digs out his copy of the old record from his filing cabinet (it will have the same reference number) to remind himself about the details of the bet. He compares this record (#128) with the new one I’ve sent (#156), checks that the updates comply with the rules in the rulebook we’d agreed to use and, because it does, he puts a big red line through the old piece of paper and sends it away for archiving.  And he then adds the new piece of paper to his filing cabinet.

picture7

Record #128 has now been superseded by record #156

We both now have identical, updated versions of the contract in our filing cabinets.  And because we’d pre-agreed to use the same rulebook there can be no ambiguity: I had provided the proof necessary to move us to an updated version of the agreement. He makes a mental note to pay me next time we meet and we’re done.

Now, in this case, it was pretty simple. This example only required one letter to be sent:  mine to Albert.  The newspaper proof left him with little choice but to accept the updated record.   But you can imagine scenarios where Albert needs to reply back or maybe even write to somebody else before we can conclude that the new record has superseded the old one.  It’s these complexities that Corda handles for us – but which aren’t necessary for understanding the core concepts.

Now… if you’ve got this far and understood all the concepts then you know pretty much everything you need to know to build solutions on Corda.

Corda is deliberately and powerfully simple 🙂

So let’s map this example to how Corda actually works

What are the building blocks of a Corda solution?

picture2

  • All those pieces of paper representing deals, contracts, trades, balances, IOUs, loans?
    • These are Corda State Objects
    • Mental model: think of documents recording all the details pertaining to a single trade, balance, trade, agreement and so forth.
  • The filing cabinet?picture8
    • That is the Corda Vault
    • Mental model: think of the place where the most current versions of all
      your contracts and deals and trades and IOUs and bets are stored
  • Those covering letters telling your peers that you’d like them to put red lines through some pieces of paper and replace them with some new ones that you’ve attached with a paperclip and included in the envelope?
    • These are Corda Transactionspicture6
    • Mental model: think of them as letters from one party to all other interested parties suggesting it’s time to remove some papers from their filing cabinet and replace them with some new ones, which are attached to the letter, provided the recipients agree this complies with the rules we’d previously agreed.
  • The postal service?
    • That’s the Corda point-to-point messaging network
    • Mental model: think of it as being how information flows around the Corda network. We don’t send everything to everybody; just those with a need to know.
  • The rulebook that Albert used to check we’d done everything in accordance with the gambling rules we agreed to be subject to?
    • That’s Corda Contract Code
    • Mental model: think of this as being the test you apply whenever you get a new letter asking you to cross-out some pieces of paper and replace them with some new ones: is this letter asking me to do something that’s consistent with the rules I signed up to when I first entered into this agreement?  This is a fundamental component of Corda’s consensus architecture (the other being Consensus Clusters, discussed briefly below)

 

So that’s pretty much it:  pieces of paper, letters, photocopiers, the postal service. And yet it turns out to allow us to model all sorts of business problems and it lets us maximise privacy, scalability and performance: only the parties to deal need to process them and store them, for example.

Once these ideas click in your head, you’ll find yourself mentally trying to model all kinds of problems as Corda state objects, transactions and contract code. And you’ll start introducing some additional questions to your business analysis sessions.  You’ll ask questions like:

  • What’s written on the pieces of paper?
  • Whose responsibility is it to write the letters?
  • What business logic do they use to fill out the new pieces of paper and figure out which ones from the filing cabinet they’re going to replace?
  • To whom do they send them?
  • Can we do this with a single letter or do you need to provide additional information and reply to me? Do some steps require you explicitly to agree (by countersigning the letter?)
  • What rules do we need to have pre-agreed to use to check that the letters are asking us to do something valid?

And this, at heart, is the Corda Way of Thinking: a way of ensuring you’re in perfect synchrony with your trading partners that naturally and obviously maps to real-world ideas and which anybody can understand.

So, if you’ve tried to use other distributed ledger platforms and got frustrated at how quickly they got complex whenever you tried to do something that didn’t spray data to everybody or how you were expected to be a crypto expert, fear not:

Corda is different.  Corda is simpler.  Corda is better.

And now you know the secret of the “Corda Way of Thinking”.

Hang on… what about consensus algorithms??????

OK… 🙂  if you’ve studied other distributed ledger platforms, you may still have a couple of questions.  If you fall into that category, continue reading…

Consensus: How do we deal with the problem of two different letters crossing in the post, both referring to the same piece of paper in the filing cabinet? We can’t cross it out twice! So we need a way to choose which letter trumps the others.

  • Answer: Corda Consensus Services (aka “notary clusters”.)    In Corda, the question of whether an update is valid is purely a matter for those processing or verifying the transactions but we need to resolve conflicts if two transactions try to update the same record at the same time.  This is where Corda Consensus Services come in. Corda has a really innovative design here, too, allowing multiple Consensus Services on the same network, including consensus service clusters running  different consensus algorithms.   (Note: the Corda documentation also refers to notary clusters: this is the same concept, but consensus services is a more understandable name!)

Workflows: What do we do if I need a response from you before I can finish updating my filing cabinet? Do I just sit there, motionless, waiting?

  • Answer: Corda flows. These are also what we would have used to establish our first record of the deal, #128, where both of us would have needed to agree that the details of the bet were correct.

You can read more about both of these in our documentation!

Summary

I’m grateful to my colleague Chris Khan, for the awesome diagrams and Clemens Wan, for creating this table, which pulls it all together:

Analogy Component Responsibility Corda Component DLT Role
Filing Cabinet Keeps track of papers Vault Stores State Objects
Sheet of paper describing a contract or agreement or deal Represents specific facts or document State Object Stores data model and references to legal prose and contract code (the “rulebook”)
Newspaper Weather Report Provides weather at time of bet Oracle Third party trusted data source for the specific deal
Letter with cover note Tells other parties that somebody has calculated some updated papers, with evidence. Transactions Method of evolving the state objects as governed by contract code
Rulebook Provides rules that govern the bet Contract Code Provides verifications and rules that govern the state object’s evolution
Pinned paper to noticeboard Provides a copy of the cover note for others interested in the party to review and sign Corda Flow Manager Specifies transaction details for multi-party signature
Signature Proof that letter really did come from who it claims to be from Signature (digital) Proof that a transaction really did come from who it claims to be from – prevents repudiation
Postal Service Ensures letters are sent to the correct parties and delivered reliably Network Map Service & point-to-point messaging network Provides a reliable way of ensuring transactions get delivered to precisely the right parties and nobody else

 

 

Towards Deeper Collaboration in Distributed Ledgers: Thoughts on Digital Asset’s Global Synchronisation Log

It’s now almost two months since we open-sourced Corda and I’m delighted by the reception it has received. In our rapidly growing community, we’re already seeing new users grow into leaders who help other newcomers get to grips with the platform.

And I am amazed by the number of inbound messages from users who have been impressed by the quality of Corda’s design and codebase – and who are already building significant applications and products on top of it. Indeed, as I write this, one of our member banks is running a global hackathon, with over 150 of their developers building on Corda and I’ve just returned from our Asia Members’ Conference in Hong Kong, where I sat through so many presentations about Corda projects I didn’t even know were happening…

If you’re not already on our Slack or participating in our discussion forum, you’re missing out!

But one additional benefit from delivering on our commitment to make Corda Open Source is that it means we can explore opportunities to collaborate with peers, competitors and partners across the ecosystem: identifying areas where our visions are aligned, where we see things the same way and where we might be able to reuse rather than needlessly reinvent.

A good example of the potential for firms who some might see as competitors (but who actually aren’t…) to collaborate was provided late last year in the form of an excellent whitepaper from Digital Asset: The Global Synchronization Log.  The paper helps clarify some really important aspects of distributed ledger design and shows a really deep understanding of the tradeoffs that are inherent in the design of these platforms.

The first time I read the paper, I was struck by how closely our two firms’ visions for DLT are aligned. As Mike Hearn has written, there are two fundamentally different ways to design a DLT –“UTXO” or “replicated virtual machine” – and it was very encouraging when I realised that our two firms, completely independently, had both concluded that the correct architecture for a significant range of important financial services use-cases is the UTXO model.

This bears repeating: two firms who, in R3’s case, had worked with a huge consortium of financial institutions on a groundbreaking year-long Architecture Working Group and, in the case of Digital Asset, had begun delivery of implementations for clients, had reached extremely similar conclusions about what the “correct” architecture should look like.

But, in reading the paper, it was also clear that we had made some different decisions, too.  And the interesting thing is that the differences are almost all related to choices we’d made about acceptable tradeoffs.  As I’ve often written, there are no perfect solutions in DLT; just tradeoffs. But I will also freely admit that we made some additions to Corda’s technical vision in the light of the paper!

So it’s time, I thought, to share my thoughts on what I think are the key points in the paper and outline how I think Corda could be a perfect way to implement the concept.

What is the Global Synchronisation Log?

At the heart of this space is a beguilingly simple vision:

DLT allows me to build systems where “I know that what I see is what you see”

That is: if a computer system that I own and run and which exists to serve my needs tells me something about a deal you and I have done, I want to know that the system you’re looking at, that you own and run and which exists to serve your needs, is telling you the same thing.

Before Bitcoin and blockchains and Distributed Ledger Technology there were only two ways of doing this, neither of them perfect:  1) we could build a centralised infrastructure and just agree to agree that whatever they say is the truth… consensus by authority, if you like or 2) we could build our own systems and then spend all our lives checking that they had come to the same conclusion about everything… consensus by reconciliation.

Bitcoin and the systems it inspired showed us there was a third way:  we could use advances in cryptography, consensus algorithms and other technologies to give ourselves near total assurance that our systems were in sync without having to employ armies of people to check.

But there was a problem… and this problem is at the absolute heart of everything that’s going on in the DLT space today: the solution invented by Bitcoin and refined through subsequent systems depends on all data being shared with all parties.  So you gain something amazing on one hand: an end to errors, duplication, inconsistency and associated risk. But, on the other hand, you create a privacy nightmare and a system that goes slower the more things you use it for.

This is precisely the conundrum that motivated the invention and development of Corda. We decomposed the building blocks of existing blockchain platforms and reassembled them in the light of the different threat-model we have, the different use-cases and different tradeoffs we are prepared to accept.

One of the key insights in our work was that, for our scenarios, we can separate transaction verification from the question of whether two verified transactions conflict with each other. I wrote about this when we first announced Corda in April last year.

We think the question of transaction verification should be down to the transaction participants: if one of them pretends that their smart contract produced a different answer to what it actually did then we’ll deal with it out-of-band; it’s a permissioned system and we know who they are…   They gain nothing by playing games like that.

But the question of which transactions actually get confirmed is a question for an independent observer; we need somebody we all trust to choose between two equally valid but conflicting transactions. At R3, we call this observer a notary but that’s just the name we use to generalise the role performed by miners in a traditional blockchain.

In so doing, we addressed many of the privacy and scalability issues of other platforms at a stroke.

But it’s a tradeoff, of course.  Because there’s something that a full public blockchain gives you that this approach doesn’t. Both approaches assure you that only valid transactions can get confirmed, but a full public blockchain also ensures that everybody gets to know when this happens.

But, of course, a traditional blockchain does this by using full broadcast, in the clear, of pretty much everything that happens. A privacy and scalability disaster.

So we had some very heated debates when we designed Corda about which tradeoffs were acceptable and which ones were not. And the GSL paper touches on all of them really succinctly.

Two of the more important debates were as follows:

  • If I send a full transaction to a notary (think ‘miner’ in a traditional blockchain), that could be a privacy leak: the notary gets to see all the data in the transaction. But if I only send the pieces of the transaction that the notary actually needs to see in order to decide transaction ordering then I could execute a “denial of state” attack by having the notary confirm an invalid transaction that “consumes” an input and stops a valid transaction from subsequently being confirmed.
  • If I send a transaction to a notary, how does it know which other parties to inform? I could execute an attack whereby I get a transaction confirmed but the other side doesn’t learn about it… that might allow me to selectively choose not to reveal it if it so suits me.

In Corda, we made the following observations. We said:

  • The “notary privacy versus denial-of-state” question is one that should be solved on a case-by-case basis. So we support “validating notaries” that need to see all data and “non-validating notaries” that just see the subset that allows them to make a confirmation decision. But we require the non-validating notaries record who sent them the transactions they sign so we know who’s to blame if anybody does try to do something nefarious.
  • But the notification issue is more tricky: recall, the full-broadcast solution used in “traditional” blockchains just won’t cut it. Indeed, that’s why, in Corda, there is no global broadcast, by design.  So if a notary is going to inform you that something happened, it needs to know who you are and how to reach you. But that’s also a privacy issue if you implement it simplistically. So users effectively need the right to decide who they trust more: the notary or their counterparties.

So now to Digital Asset’s paper.   What they propose is very reasonable.  In essence, they say the following:

  • The Digital Asset GSL model is comfortable with the risk of a “denial of state” attack. (As are we at R3 for many scenarios, by the way, because the mitigations are robust; but Corda’s default mode is to protect against this threat).
  • So this means it’s fine if the notary only gets to see the subset of a transaction that is needed in order to determine ordering/uniqueness.
  • But GSL users are entirely not OK if a transaction can be confirmed and yet all the affected parties don’t get to hear about it at the same time as the transaction submitter.

And the paper goes on to explain how they think that last problem should be solved.

In essence, they do the following:

  • They effectively add the identities of all the parties who should know about the transaction to the outside of the transaction. This is the part that the notary sees.
    • They don’t actually put the interested parties’ identities on directly – that would be a privacy leak – but that’s the effect; you can think of them as “tagging” the transaction with the list of everybody who needs to know about it.
  • But that’s not enough, of course. The notary doesn’t get to see most of the transaction contents, remember… so the list could be wrong and the notary wouldn’t know! So they go a step further.
  • They add an additional rule to the transaction verification logic: if the transaction doesn’t “tag” the right set of intended recipients then it isn’t considered valid.
  • So now you have something pretty cool: you can get a transaction that fails to tag the right people notarised just fine (the notaries are ‘non-validating’ in the DA model, remember). But the “attacker” gains nothing because the transaction itself won’t be considered valid per the rules of the system.  So whatever nefarious scheme you were plotting fails…
  • And if you do construct a valid transaction then the act of getting it confirmed is also the irreversible act of having the notary inform all affected parties. So a bad guy doesn’t get to withhold valid, confirmed transactions.
  • This approach binds the question of transaction validity to the question of notification of affected parties.  You can’t have one without the other.

So you achieve something useful: transaction contents remain visible only to those who need to see them, transaction verification is in the hands of those to whom they pertain, notaries don’t see what they shouldn’t and if a transaction gets committed all relevant parties get to hear about it.  For a good number of use-cases, that’s a decent set of tradeoffs.

So can Corda provide a solution for the GSL?

(Spoiler: YES!)

It turns out that Corda’s design already has every single one of the features needed to implement the GSL – apart from one, which we added specifically to address this requirement.

  • Corda’s notaries already log the transaction submitters when operating in non-validating mode so we already solve the “denial of state issue” just fine.
  • Corda already supports “transaction tear-offs”, the mechanism whereby only the relevant information is shared with third parties such as notaries, using Merkle trees.
  • Corda already supports the concept of “participants” – aka“tags” – a list attached to each transaction that identifies interested parties
  • Corda’s transaction verification engine already allows contracts to verify that the participant list is correctly populated.

So we already have the mechanism to bind the verification to the population of the notification list.  But the “out of the box” design does not then ensure the notification actually happens…  This was a deliberate choice based on prioritisation of requirements and (yet another!) tradeoff around privacy.

In other words, there was one missing piece, albeit a deliberate one.  But reading this paper made us convinced adding that feature made sense and so we’ve added it to our design and will be added to the codebase in a future milestone release. The thinking is captured in section 7.5 of our technical whitepaper, starting page 33.

Note that our proposed implementation is slightly different to the design sketched in the Digital Asset paper because we deliberately and famously don’t have a blockchain: so there is no data structure that participants can passively browse to look for transactions of interest. Instead, we use a push point-to-point messaging network.  So the notary will directly inform affected parties.

Open Innovation: 2017 is the Year Corda Goes Mainstream

One of the many benefits of working on an open source project is that it becomes so easy and natural to explore these sorts of concepts with other firms, through initiatives such as the Hyperledger Project; through discussion of each other’s papers, like here; and through coding and direct collaboration between developers: we’re very much enjoying working with one of Digital Asset’s developers in our public Slack group, for example.

We think Corda is shaping up to be a perfect architecture for implementing the Global Synchronization Log concept and I am grateful to the team at Digital Asset for sharing their thinking – and their list of requirements – so openly and clearly.

Here’s to open innovation!

Countdown to Corda Open Source

R3 will soon be open-sourcing Corda. Here’s what to expect.

As I confirmed a few months back, R3’s Corda platform will be open-sourced, under the Apache 2 licence, on November 30.

Corda is a distributed ledger platform designed and built from the ground up for the recording and automation of legal agreements between identifiable parties. It is heavily influenced by the requirements of the financial industry but we believe the community will find the underlying architecture will lend itself to a broad range of applications.

We’ve built Corda because we see requirements – especially in finance – that need a distributed ledger but which cannot be met by existing platforms.

  • Corda is the only Distributed Ledger platform designed by the world’s largest financial institutions to manage legal agreements on an automatable and enforceable basis.
  • Corda only shares data with those with a need to view or validate it; there is no global broadcasting of data across the network.
  • Corda is the only Distributed Ledger platform to support multiple consensus providers employing different consensus algorithms on the same network, enabling compliance with local regulations.
  • Corda is designed to provide a great developer experience and to make integration and interoperability easy: query the ledger with SQL, join to external databases, perform bulk imports, and code contracts in a range of modern, standard languages.

We designed it with the members of R3, the world’s largest financial services DLT consortium, but we think its applicability is far broader.  You can find out more in our introductory whitepaper and my blog post on why we’re building Corda and what makes it different. If you prefer videos, here’s a short interview I did with Simon Taylor of 11:FS that explains the thought process behind Corda.

What we’ll release on November 30 is pretty much the full codebase as it exists today and we will be improving it actively and openly from then on. In fact, the only code we’ve held back pertains to laboratory projects we’re working on with our members and work on our own commercial business products that will run on top of Corda.

So do take a look around when the code is released: there’s a lot in there that is still work-in-progress and not yet integrated. For example, you’ll find a fascinating approach to writing financial contracts in the experimental branch and ongoing work on our deterministic sandbox for the JVM.   We will, of course, also be developing a commercial version of Corda for those who need specific enterprise features and support, but the open source codebase is the foundation of everything we do.

This is a really important point: distributed ledger technologies will have such phenomenally powerful network effects that it is unthinkable that serious institutions would deploy base-layer ledger software that is anything other than fully and wholeheartedly open. And it’s why we’ve been committed all along to releasing Corda just as soon as we were sure it was heading in the right direction.  It is and so we are.

We will also be publishing a draft of our technical whitepaper.  This whitepaper outlines our roadmap to version 1.0 of Corda and production-readiness.

What to expect on November 30

We’re really proud of Corda and its progress to date. But, that said, Corda is far from finished. Mike Hearn will soon be publishing a “warts and all” description of quite how much work we still have to do. This is true for all other platforms in this space, of course, but I feel a particular responsibility to be transparent given the ambitions we have for Corda and the uses to which it will be put.

By way of example, perhaps a good way to help you figure out what we still have to do is to look at some items on the list of work we’ve set for the months ahead of us:

  • Functional completeness: Corda still has gaps in its functional capabilities. The technical whitepaper outlines the full vision and you’ll see us working on and merging a lot of functional enhancements in the coming months to implement the full vision in the paper.
  • Non-functional characteristics: We focused first on design and then on implementation of Corda’s core functionality. The work to ensure we meet our non-functional requirements, such as performance, is still ahead of us but we have a clear roadmap and have designed the platform with these needs firmly in mind.
  • Security hardening: There are lots of areas where we need to tighten up security. Much of this we know about and we have called it out in the code or associated docs. But there will, of course, be others. So just as you shouldn’t be using other enterprise DLT platforms in production just yet, please don’t download Corda and put it straight into production just yet either!
  • API Stability: Corda’s development is iterative and organic – and it is heavily influenced by the range of projects and applications to which our members are choosing to put it. As we learn about common patterns and discover assumptions that prove to be wrong, we adapt. In particular, this means that we do not commit to API stability or backwards-compatibility until version 1.0.  Expect parts of the implementation to change in the coming months, perhaps quite significantly!

But these things are transient: we know how to fix them and we’ll knock the issues off one-by-one in the coming months as we head towards version 1.0.  But we want you to be fully aware of them.

Why are we open-sourcing Corda now?

We had a vigorous internal debate about when was the right time to release Corda: wait until it was more mature, when we were confident we’d ironed out the bugs and made it fly?  Or wait only until the design roadmap was clear and then share it immediately with the world for comment, criticism, contribution and collaboration?

We’ve wholeheartedly chosen the latter path: to release early and to work openly.

We’re serious about inviting the community to critique, collaborate and contribute. To take one example, our friends at Digital Asset recently published an excellent paper describing a set of requirements for what they call a “Global Synchronisation Log” (GSL), encouraging those in the community to incorporate these requirements into their platforms. We think that Corda’s vision is extremely well aligned to the GSL concept and by open-sourcing our work whilst there is still time to tweak our design it means we maximise the opportunities for firms such as ours to collaborate.

But open-sourcing Corda when it is still fairly young is not without its risks!  In fact, I’m a little apprehensive. I’m a completer-finisher and I obsess over every detail. So the idea of releasing something before it’s perfect makes me feel uncomfortable.  You will find gaps, issues, problems. But that’s fine: please do share what you find.  Even better, submit a fix…!

In fact, I also have a hope that some of those who come to critique will find that they nonetheless like much of what they see, and may even join the community.

What happens next?

I performed a thought experiment a while back… I asked: what will the enterprise distributed ledger world look like when everything settles down in a few years? How many independent enterprise DLT platforms will the world need and which ones will they be?

My conclusion was that there will probably be at most three such platforms, each carefully designed and adapted for a specific set of requirements. They will all be fully open source. And they will be surrounded by thriving, inclusive communities.

And we firmly intend to ensure Corda is one of them.

Our open-source release next week is a key step on that journey.

How to get Corda on November 30

Corda’s home will be corda.net.

Head over to corda.net on November 30 for links to the codebase, simple sample applications and a tutorial to get started writing your own CorDapps.

 

 

On Distributed Databases and Distributed Ledgers

Why can’t companies wanting to share business logic and data just install a distributed database? What is the essential difference between a distributed database and a distributed ledger?

Last month, I shared the thinking that led to the design of Corda, which we at R3 will be open sourcing on November 30; and Mike Hearn and I were interviewed by Brian and Meher of Epicenter last week. We’ve been delighted by the response and are looking forward to working with those seek to build on Corda, help influence its direction or contribute to its development and maturation;  there’s a lot of work ahead of us!

But one or two observers have asked a really good question. They asked me: “Aren’t you just reimplementing a distributed database?!”

The question is legitimate: if you strip away the key assumptions underpinning systems like Bitcoin and Ethereum, are you actually left with anything? What is actually different between a distributed ledger platform such as Corda and a traditional distributed database?

The answer lies in the definition I gave in my last blogpost and it is utterly crucial since it defines an entire new category of data management system:

“Distributed ledgers – or decentralised databases – are systems that enable parties who don’t fully trust each other to form and maintain consensus about the existence, status and evolution of a set of shared facts”

“Parties who don’t fully trust each other” is at the heart of this. To see why, let’s compare distributed databases and Corda.

Comparing Corda to a distributed database

In a distributed database, we often have multiple nodes that cooperate to maintain a consistent view for their users.   The nodes may cooperate to maintain partitions of the overall dataset or they may cooperate to maintain consistent replicas but the principle is the same:  a group of computers, invariably under the control of a single organisation, cooperate to maintain their state.  These nodes trust each other.   The trust boundary is between the distributed database system as a whole and its users.    Each node in the system trusts the data that it receives from its peers and nodes are trusted to look after the data they have received from their peers.  You can think of the threat model as all the nodes shouting in unison: “it’s us against the world!”

This diagram is a stylised representation of a distributed database:

 distributed-database

In a distributed database, nodes cooperate to maintain a consistent view that they present to the outside world; they cooperate to maintain rigorous access control and they validate information they receive from the outside world.

So it’s no surprise that distributed databases are invariably operated by a single entity: the nodes of the system assume the other nodes are “just as diligent” as them: they freely share information with each other and take information from each other on trust. A distributed database operated by mutually distrusting entities is almost a contradiction in terms.

And, of course, if you have a business problem where you are happy to rely on a central operator to maintain your records – as you sometimes can in finance it should be said – then a distributed database will do just fine: let the central operator run it for you.  But if you need to maintain your own records, in synchrony with your peers, this architecture simply won’t do.

And there are huge numbers of situations where we need to maintain accurate, shared records with our counterparts. Indeed, a vast amount of the cost and inefficiency in today’s financial markets stems from the fact that it has been so difficult to achieve this. Until now.

Corda helps parties collaborate to maintain shared data without fully trusting each other

Corda is designed to allow parties to collaborate with their peers to maintain shared records, without having to trust each other fully. So Corda faces a very different world to a distributed database.

A Corda node can not assume the data it receives from a peer is valid: the peer is probably operated by a completely different entity and even if they know who that entity is, it’s still extremely prudent to verify the information.   Moreover, if a Corda node sends data to another node, it must assume that node might print it all in an advert on the front page of the New York Times.

The trust boundaries – the red curves in the diagram- are drawn in a completely different place!

decentralised-database

In Corda, nodes are operated by different organisations and do NOT trust each other; but the outcome is still a consistent view of data.

To repeat, because this distinction is utterly fundamental:  nodes of a distributed database trust each other and collaborate with each other to present a consistent, secure face to the rest of the world.   By contrast, Corda nodes can not trust each other and so must independently verify data they receive from each other and only share data they are happy to be broadly shared.

And so we call Corda a distributed ledger, to distinguish it from distributed databases. A distributed ledger that is designed painstakingly for the needs of commercial entities.

Put more simply: you simply can’t build the applications we envisage for Corda with traditional database technology.  And that’s what makes this new field so exciting.

R3 Corda: What makes it different

As reported by Reuters last week, Corda, the Distributed Ledger platform we’ve been working hard on at R3 for the last year at will be open sourced on November 30.

What is it? Why are we building it? What happens next?

Corda is a distributed ledger platform designed and built from the ground up for the recording and automation of legal agreements between identifiable parties. It is heavily influenced by the requirements of the financial industry but we believe the community will find the underlying architecture will lend itself to a broad range of applications.

Corda is quite unlike any other Distributed Ledger platform that currently exists. So we’ll be releasing lots of information in the coming weeks and months. To understand why it looks the way it does, I thought I’d share the journey we went on to build it.  In subsequent articles, the team and I will share more detail about how it works and what to look for when it’s released on November 30.

But first, some history.

The very first decision made by the Steering Committee of the R3 consortium was to establish our Architecture Working Group, which I chair.  This group consists of hundreds of senior architects, technologists and developers, many with decades of experience in a dazzling array of areas, from across our membership over over seventy financial institutions.

We were given a simple-sounding mission:

“To establish the architecture for an open, enterprise-grade, shared platform for the immutable recording of financial events and execution of logic”.

There is quite a lot packed into that sentence..!  Let’s look at just two parts:

  • “Open”
    • I stated publicly in April that we would open-source Corda and I was serious. Our mandate, from our member banks, was that whatever base platforms we selected, built or adopted had to be open. We’re delivering on this commitment with the open-sourcing of Corda on November 30.
  • “Immutable recording of financial events and execution of business logic”
    • Notice what this doesn’t say. It doesn’t say “blockchain”.  Heck: it doesn’t even say “distributed ledger”!    Instead, it tries to get to the heart of what we think is the essence of this exciting new field.  And that’s what I want to talk about in this blog post.

We don’t like solutions looking for problems

I wanted us to be precise about what this field is all about.  After all, and as I wrote when we first announced Corda, Satoshi Nakamoto didn’t wake up one morning thinking: “I really need a blockchain!”.  No: he started with a well-defined business problem and engineered a solution to solve that problem.  And if you need a system of censorship-resistant digital cash, then Satoshi’s design – Bitcoin – is the elegant solution and it’s available today.

And that’s why Ethereum, to take another example, looks so different to Bitcoin.  Vitalik Buterin and his colleagues started with a different business problem, which I characterise as “I want an unstoppable world computer that can execute business logic and move value autonomously” and guess what? They ended up with a very different design!  Now sure:  there are many similarities between Bitcoin and Ethereum but also a lot of differences.

I was determined that we would not fall into the trap of taking technologies designed to solve completely different problems and blindly apply them to banking.  That way lies madness.

So we drove two key pieces of work: 1) characterising exactly what is new about this field and 2) identifying precisely where in finance it may have most applicability.

And the answer, as I outlined back in April, is that there is something genuinely new in this space and it’s something that is massively relevant to the financial system.

The definition I think best captures this is as follows:

“Distributed ledgers – or decentralised databases – are systems that enable parties who don’t fully trust each other to form and maintain consensus about the existence, status and evolution of a set of shared facts”

Let’s first test that this definition works for existing public systems:

Bitcoin: the participants don’t know each other’s identities and come to consensus about how many bitcoins there are, which addresses own them and what needs to happen for any of them to be spent without having to trust each other.  Check!

Ethereum: the participants don’t know each other’s identities and come to consensus about the state of a virtual computer.  Check!

In those systems’ cases, they achieve these outcomes in ways with which we’re both familiar and which address requirements related to the environment in which those systems are expected to run.

But how about finance… parties who don’t fully trust each other but whom need to be in consensus about a set of shared facts?

Where do we have that problem?

Erm… how about everywhere..?!!

It’s perhaps only a slight exaggeration to suggest that the financial industry is pretty much defined by the web of contracts that exists between its participants:  I deposit money with a bank? There’s a contract there that says the bank owes me that money.  You and I enter into a Credit Default Swap? There’s a contract there that describes our mutual rights and obligations.  And they’re recorded and managed in multiple places, on different systems, managed by different firms and it costs a fortune to keep them all in sync.

The shared facts in finance are the existence and state of financial agreements – ie contracts.

And the need for consensus is what amounts to the twenty-first century’s “paperwork crisis”: the tens of billions of dollars spent annually maintaining and managing the duplicated records that each firm maintain about the same deals.  The same information about a deal is recorded multiple times across these parties and in situations where a centralised solution can’t be deployed, which is in lots of places, small armies are required to ensure that these disparate records agree with each other, get updated correctly and in synchrony – and deal with the issues when they don’t.

A ha!  So now we have something phenomenally exciting: a new technology for establishing and maintaining consensus between parties who don’t trust each other.  And a multi-billion dollar business problem crying out for this solution!

There’s only one minor problem…

Public systems like Bitcoin were not designed to solve these problems. They’re excellent at what they do; but we’re doing something else.

And you only need to take a cursory look at the architecture of various public blockchain systems to see why this might be the case.  My business problem amounts to ensuring the Bank of Alice and Bank of Bob agree about a trade they just did and that it settles automatically and correctly.   A solution which not only shares this confidential data with every other bank in the world but which also requires them to process the deal and maybe even validate it doesn’t meet my needs. And yet… that’s how every single public platform back in 2015 worked.

Perhaps those architectures can be heavily re-engineered to solve such problems, as some groups are attempting but it’s not an obvious starting point, especially when you then layer on all the other requirements we identified.

So there’s a problem: Bitcoin and its successors taught us that a new way of building distributed systems was possible: one where mutually distrusting parties can maintain a shared database.  We identified a hitherto unsolved problem in finance. And yet the technology that existed simply wasn’t designed for this.

 

Coding, not talking

The reality is that finding fault is easy; proposing workable solutions is altogether harder.   So simply going out and shouting about how 2015’s platforms didn’t solve our problems was hardly a way to make friends.  No.  We needed to do better than that.

So once we had decided we needed to prototype the alternate approach we had identified, we made a critical decision to buttress my leadership team of James Carlyle and Ian Grigg:  we brought in Mike Hearn.

And he drove the prototyping effort to explore these concepts in the only way that gives you certainty that it can be done: by proving it in code.  As Mike enjoys reminding me: when it comes to core concepts, talk is cheap; at some point, the talking has to give way to coding.

 

Early results were promising: the reductive, bottom-up approach we took to architecture and design, which is explored in our introductory whitepaper and on which we’ll elaborate in the coming weeks, was solid: we could model a diverse range of instruments; the design would allow for significant parallel processing; we did not need to send all data to all participants in all scenarios; the use of a mainstream virtual machine and its libraries led to high developer productivity; we were able to support multiple consensus providers on a single network; the use of a flat, point-to-point queue-based, peer-to-peer network mapped well to real business scenarios; and more.

We worked with our members to test the maturing codebase in a variety of contexts: interfacing Ricardian Contracts and Smart Contracts in the context of an Interest Rate Swap with Barclays and others; managing trade finance flows; and more.

And this focus on validated client requirements and a willingness to question some hitherto sacred beliefs (we have no blocks! we have no miners! we don’t put ephemeral data in the consensus layer! we allow per-transaction specification of consensus providers!) led to a unique design.

Had Corda ended up being a minor variation on an existing platform or a me-too copy of something else, what would have been the point in pursuing the work?  But that isn’t what happened: we ended up with something quite distinct, something we believe is singularly well-suited to a wider variety of financial-services use-cases and something  adapted to the practical reality that the industry is regulated and some rules simply aren’t going to change overnight.

So that’s the backstory. Our large – and growing – technology team still has a large amount of work to do.  But now is the time to share our work with the broader community and encourage people – including in other industries – to use it for their own applications as it matures (it’s still a young codebase), to contribute to Corda itself, and to contribute to the architectural debate.

We’re looking forward to November 30. This is an exciting time!