Countdown to Corda Open Source

R3 will soon be open-sourcing Corda. Here’s what to expect.

As I confirmed a few months back, R3’s Corda platform will be open-sourced, under the Apache 2 licence, on November 30.

Corda is a distributed ledger platform designed and built from the ground up for the recording and automation of legal agreements between identifiable parties. It is heavily influenced by the requirements of the financial industry but we believe the community will find the underlying architecture will lend itself to a broad range of applications.

We’ve built Corda because we see requirements – especially in finance – that need a distributed ledger but which cannot be met by existing platforms.

  • Corda is the only Distributed Ledger platform designed by the world’s largest financial institutions to manage legal agreements on an automatable and enforceable basis.
  • Corda only shares data with those with a need to view or validate it; there is no global broadcasting of data across the network.
  • Corda is the only Distributed Ledger platform to support multiple consensus providers employing different consensus algorithms on the same network, enabling compliance with local regulations.
  • Corda is designed to provide a great developer experience and to make integration and interoperability easy: query the ledger with SQL, join to external databases, perform bulk imports, and code contracts in a range of modern, standard languages.

We designed it with the members of R3, the world’s largest financial services DLT consortium, but we think its applicability is far broader.  You can find out more in our introductory whitepaper and my blog post on why we’re building Corda and what makes it different. If you prefer videos, here’s a short interview I did with Simon Taylor of 11:FS that explains the thought process behind Corda.

What we’ll release on November 30 is pretty much the full codebase as it exists today and we will be improving it actively and openly from then on. In fact, the only code we’ve held back pertains to laboratory projects we’re working on with our members and work on our own commercial business products that will run on top of Corda.

So do take a look around when the code is released: there’s a lot in there that is still work-in-progress and not yet integrated. For example, you’ll find a fascinating approach to writing financial contracts in the experimental branch and ongoing work on our deterministic sandbox for the JVM.   We will, of course, also be developing a commercial version of Corda for those who need specific enterprise features and support, but the open source codebase is the foundation of everything we do.

This is a really important point: distributed ledger technologies will have such phenomenally powerful network effects that it is unthinkable that serious institutions would deploy base-layer ledger software that is anything other than fully and wholeheartedly open. And it’s why we’ve been committed all along to releasing Corda just as soon as we were sure it was heading in the right direction.  It is and so we are.

We will also be publishing a draft of our technical whitepaper.  This whitepaper outlines our roadmap to version 1.0 of Corda and production-readiness.

What to expect on November 30

We’re really proud of Corda and its progress to date. But, that said, Corda is far from finished. Mike Hearn will soon be publishing a “warts and all” description of quite how much work we still have to do. This is true for all other platforms in this space, of course, but I feel a particular responsibility to be transparent given the ambitions we have for Corda and the uses to which it will be put.

By way of example, perhaps a good way to help you figure out what we still have to do is to look at some items on the list of work we’ve set for the months ahead of us:

  • Functional completeness: Corda still has gaps in its functional capabilities. The technical whitepaper outlines the full vision and you’ll see us working on and merging a lot of functional enhancements in the coming months to implement the full vision in the paper.
  • Non-functional characteristics: We focused first on design and then on implementation of Corda’s core functionality. The work to ensure we meet our non-functional requirements, such as performance, is still ahead of us but we have a clear roadmap and have designed the platform with these needs firmly in mind.
  • Security hardening: There are lots of areas where we need to tighten up security. Much of this we know about and we have called it out in the code or associated docs. But there will, of course, be others. So just as you shouldn’t be using other enterprise DLT platforms in production just yet, please don’t download Corda and put it straight into production just yet either!
  • API Stability: Corda’s development is iterative and organic – and it is heavily influenced by the range of projects and applications to which our members are choosing to put it. As we learn about common patterns and discover assumptions that prove to be wrong, we adapt. In particular, this means that we do not commit to API stability or backwards-compatibility until version 1.0.  Expect parts of the implementation to change in the coming months, perhaps quite significantly!

But these things are transient: we know how to fix them and we’ll knock the issues off one-by-one in the coming months as we head towards version 1.0.  But we want you to be fully aware of them.

Why are we open-sourcing Corda now?

We had a vigorous internal debate about when was the right time to release Corda: wait until it was more mature, when we were confident we’d ironed out the bugs and made it fly?  Or wait only until the design roadmap was clear and then share it immediately with the world for comment, criticism, contribution and collaboration?

We’ve wholeheartedly chosen the latter path: to release early and to work openly.

We’re serious about inviting the community to critique, collaborate and contribute. To take one example, our friends at Digital Asset recently published an excellent paper describing a set of requirements for what they call a “Global Synchronisation Log” (GSL), encouraging those in the community to incorporate these requirements into their platforms. We think that Corda’s vision is extremely well aligned to the GSL concept and by open-sourcing our work whilst there is still time to tweak our design it means we maximise the opportunities for firms such as ours to collaborate.

But open-sourcing Corda when it is still fairly young is not without its risks!  In fact, I’m a little apprehensive. I’m a completer-finisher and I obsess over every detail. So the idea of releasing something before it’s perfect makes me feel uncomfortable.  You will find gaps, issues, problems. But that’s fine: please do share what you find.  Even better, submit a fix…!

In fact, I also have a hope that some of those who come to critique will find that they nonetheless like much of what they see, and may even join the community.

What happens next?

I performed a thought experiment a while back… I asked: what will the enterprise distributed ledger world look like when everything settles down in a few years? How many independent enterprise DLT platforms will the world need and which ones will they be?

My conclusion was that there will probably be at most three such platforms, each carefully designed and adapted for a specific set of requirements. They will all be fully open source. And they will be surrounded by thriving, inclusive communities.

And we firmly intend to ensure Corda is one of them.

Our open-source release next week is a key step on that journey.

How to get Corda on November 30

Corda’s home will be corda.net.

Head over to corda.net on November 30 for links to the codebase, simple sample applications and a tutorial to get started writing your own CorDapps.

 

 

On Distributed Databases and Distributed Ledgers

Why can’t companies wanting to share business logic and data just install a distributed database? What is the essential difference between a distributed database and a distributed ledger?

Last month, I shared the thinking that led to the design of Corda, which we at R3 will be open sourcing on November 30; and Mike Hearn and I were interviewed by Brian and Meher of Epicenter last week. We’ve been delighted by the response and are looking forward to working with those seek to build on Corda, help influence its direction or contribute to its development and maturation;  there’s a lot of work ahead of us!

But one or two observers have asked a really good question. They asked me: “Aren’t you just reimplementing a distributed database?!”

The question is legitimate: if you strip away the key assumptions underpinning systems like Bitcoin and Ethereum, are you actually left with anything? What is actually different between a distributed ledger platform such as Corda and a traditional distributed database?

The answer lies in the definition I gave in my last blogpost and it is utterly crucial since it defines an entire new category of data management system:

“Distributed ledgers – or decentralised databases – are systems that enable parties who don’t fully trust each other to form and maintain consensus about the existence, status and evolution of a set of shared facts”

“Parties who don’t fully trust each other” is at the heart of this. To see why, let’s compare distributed databases and Corda.

Comparing Corda to a distributed database

In a distributed database, we often have multiple nodes that cooperate to maintain a consistent view for their users.   The nodes may cooperate to maintain partitions of the overall dataset or they may cooperate to maintain consistent replicas but the principle is the same:  a group of computers, invariably under the control of a single organisation, cooperate to maintain their state.  These nodes trust each other.   The trust boundary is between the distributed database system as a whole and its users.    Each node in the system trusts the data that it receives from its peers and nodes are trusted to look after the data they have received from their peers.  You can think of the threat model as all the nodes shouting in unison: “it’s us against the world!”

This diagram is a stylised representation of a distributed database:

 distributed-database

In a distributed database, nodes cooperate to maintain a consistent view that they present to the outside world; they cooperate to maintain rigorous access control and they validate information they receive from the outside world.

So it’s no surprise that distributed databases are invariably operated by a single entity: the nodes of the system assume the other nodes are “just as diligent” as them: they freely share information with each other and take information from each other on trust. A distributed database operated by mutually distrusting entities is almost a contradiction in terms.

And, of course, if you have a business problem where you are happy to rely on a central operator to maintain your records – as you sometimes can in finance it should be said – then a distributed database will do just fine: let the central operator run it for you.  But if you need to maintain your own records, in synchrony with your peers, this architecture simply won’t do.

And there are huge numbers of situations where we need to maintain accurate, shared records with our counterparts. Indeed, a vast amount of the cost and inefficiency in today’s financial markets stems from the fact that it has been so difficult to achieve this. Until now.

Corda helps parties collaborate to maintain shared data without fully trusting each other

Corda is designed to allow parties to collaborate with their peers to maintain shared records, without having to trust each other fully. So Corda faces a very different world to a distributed database.

A Corda node can not assume the data it receives from a peer is valid: the peer is probably operated by a completely different entity and even if they know who that entity is, it’s still extremely prudent to verify the information.   Moreover, if a Corda node sends data to another node, it must assume that node might print it all in an advert on the front page of the New York Times.

The trust boundaries – the red curves in the diagram- are drawn in a completely different place!

decentralised-database

In Corda, nodes are operated by different organisations and do NOT trust each other; but the outcome is still a consistent view of data.

To repeat, because this distinction is utterly fundamental:  nodes of a distributed database trust each other and collaborate with each other to present a consistent, secure face to the rest of the world.   By contrast, Corda nodes can not trust each other and so must independently verify data they receive from each other and only share data they are happy to be broadly shared.

And so we call Corda a distributed ledger, to distinguish it from distributed databases. A distributed ledger that is designed painstakingly for the needs of commercial entities.

Put more simply: you simply can’t build the applications we envisage for Corda with traditional database technology.  And that’s what makes this new field so exciting.

R3 Corda: What makes it different

As reported by Reuters last week, Corda, the Distributed Ledger platform we’ve been working hard on at R3 for the last year at will be open sourced on November 30.

What is it? Why are we building it? What happens next?

Corda is a distributed ledger platform designed and built from the ground up for the recording and automation of legal agreements between identifiable parties. It is heavily influenced by the requirements of the financial industry but we believe the community will find the underlying architecture will lend itself to a broad range of applications.

Corda is quite unlike any other Distributed Ledger platform that currently exists. So we’ll be releasing lots of information in the coming weeks and months. To understand why it looks the way it does, I thought I’d share the journey we went on to build it.  In subsequent articles, the team and I will share more detail about how it works and what to look for when it’s released on November 30.

But first, some history.

The very first decision made by the Steering Committee of the R3 consortium was to establish our Architecture Working Group, which I chair.  This group consists of hundreds of senior architects, technologists and developers, many with decades of experience in a dazzling array of areas, from across our membership over over seventy financial institutions.

We were given a simple-sounding mission:

“To establish the architecture for an open, enterprise-grade, shared platform for the immutable recording of financial events and execution of logic”.

There is quite a lot packed into that sentence..!  Let’s look at just two parts:

  • “Open”
    • I stated publicly in April that we would open-source Corda and I was serious. Our mandate, from our member banks, was that whatever base platforms we selected, built or adopted had to be open. We’re delivering on this commitment with the open-sourcing of Corda on November 30.
  • “Immutable recording of financial events and execution of business logic”
    • Notice what this doesn’t say. It doesn’t say “blockchain”.  Heck: it doesn’t even say “distributed ledger”!    Instead, it tries to get to the heart of what we think is the essence of this exciting new field.  And that’s what I want to talk about in this blog post.

We don’t like solutions looking for problems

I wanted us to be precise about what this field is all about.  After all, and as I wrote when we first announced Corda, Satoshi Nakamoto didn’t wake up one morning thinking: “I really need a blockchain!”.  No: he started with a well-defined business problem and engineered a solution to solve that problem.  And if you need a system of censorship-resistant digital cash, then Satoshi’s design – Bitcoin – is the elegant solution and it’s available today.

And that’s why Ethereum, to take another example, looks so different to Bitcoin.  Vitalik Buterin and his colleagues started with a different business problem, which I characterise as “I want an unstoppable world computer that can execute business logic and move value autonomously” and guess what? They ended up with a very different design!  Now sure:  there are many similarities between Bitcoin and Ethereum but also a lot of differences.

I was determined that we would not fall into the trap of taking technologies designed to solve completely different problems and blindly apply them to banking.  That way lies madness.

So we drove two key pieces of work: 1) characterising exactly what is new about this field and 2) identifying precisely where in finance it may have most applicability.

And the answer, as I outlined back in April, is that there is something genuinely new in this space and it’s something that is massively relevant to the financial system.

The definition I think best captures this is as follows:

“Distributed ledgers – or decentralised databases – are systems that enable parties who don’t fully trust each other to form and maintain consensus about the existence, status and evolution of a set of shared facts”

Let’s first test that this definition works for existing public systems:

Bitcoin: the participants don’t know each other’s identities and come to consensus about how many bitcoins there are, which addresses own them and what needs to happen for any of them to be spent without having to trust each other.  Check!

Ethereum: the participants don’t know each other’s identities and come to consensus about the state of a virtual computer.  Check!

In those systems’ cases, they achieve these outcomes in ways with which we’re both familiar and which address requirements related to the environment in which those systems are expected to run.

But how about finance… parties who don’t fully trust each other but whom need to be in consensus about a set of shared facts?

Where do we have that problem?

Erm… how about everywhere..?!!

It’s perhaps only a slight exaggeration to suggest that the financial industry is pretty much defined by the web of contracts that exists between its participants:  I deposit money with a bank? There’s a contract there that says the bank owes me that money.  You and I enter into a Credit Default Swap? There’s a contract there that describes our mutual rights and obligations.  And they’re recorded and managed in multiple places, on different systems, managed by different firms and it costs a fortune to keep them all in sync.

The shared facts in finance are the existence and state of financial agreements – ie contracts.

And the need for consensus is what amounts to the twenty-first century’s “paperwork crisis”: the tens of billions of dollars spent annually maintaining and managing the duplicated records that each firm maintain about the same deals.  The same information about a deal is recorded multiple times across these parties and in situations where a centralised solution can’t be deployed, which is in lots of places, small armies are required to ensure that these disparate records agree with each other, get updated correctly and in synchrony – and deal with the issues when they don’t.

A ha!  So now we have something phenomenally exciting: a new technology for establishing and maintaining consensus between parties who don’t trust each other.  And a multi-billion dollar business problem crying out for this solution!

There’s only one minor problem…

Public systems like Bitcoin were not designed to solve these problems. They’re excellent at what they do; but we’re doing something else.

And you only need to take a cursory look at the architecture of various public blockchain systems to see why this might be the case.  My business problem amounts to ensuring the Bank of Alice and Bank of Bob agree about a trade they just did and that it settles automatically and correctly.   A solution which not only shares this confidential data with every other bank in the world but which also requires them to process the deal and maybe even validate it doesn’t meet my needs. And yet… that’s how every single public platform back in 2015 worked.

Perhaps those architectures can be heavily re-engineered to solve such problems, as some groups are attempting but it’s not an obvious starting point, especially when you then layer on all the other requirements we identified.

So there’s a problem: Bitcoin and its successors taught us that a new way of building distributed systems was possible: one where mutually distrusting parties can maintain a shared database.  We identified a hitherto unsolved problem in finance. And yet the technology that existed simply wasn’t designed for this.

 

Coding, not talking

The reality is that finding fault is easy; proposing workable solutions is altogether harder.   So simply going out and shouting about how 2015’s platforms didn’t solve our problems was hardly a way to make friends.  No.  We needed to do better than that.

So once we had decided we needed to prototype the alternate approach we had identified, we made a critical decision to buttress my leadership team of James Carlyle and Ian Grigg:  we brought in Mike Hearn.

And he drove the prototyping effort to explore these concepts in the only way that gives you certainty that it can be done: by proving it in code.  As Mike enjoys reminding me: when it comes to core concepts, talk is cheap; at some point, the talking has to give way to coding.

 

Early results were promising: the reductive, bottom-up approach we took to architecture and design, which is explored in our introductory whitepaper and on which we’ll elaborate in the coming weeks, was solid: we could model a diverse range of instruments; the design would allow for significant parallel processing; we did not need to send all data to all participants in all scenarios; the use of a mainstream virtual machine and its libraries led to high developer productivity; we were able to support multiple consensus providers on a single network; the use of a flat, point-to-point queue-based, peer-to-peer network mapped well to real business scenarios; and more.

We worked with our members to test the maturing codebase in a variety of contexts: interfacing Ricardian Contracts and Smart Contracts in the context of an Interest Rate Swap with Barclays and others; managing trade finance flows; and more.

And this focus on validated client requirements and a willingness to question some hitherto sacred beliefs (we have no blocks! we have no miners! we don’t put ephemeral data in the consensus layer! we allow per-transaction specification of consensus providers!) led to a unique design.

Had Corda ended up being a minor variation on an existing platform or a me-too copy of something else, what would have been the point in pursuing the work?  But that isn’t what happened: we ended up with something quite distinct, something we believe is singularly well-suited to a wider variety of financial-services use-cases and something  adapted to the practical reality that the industry is regulated and some rules simply aren’t going to change overnight.

So that’s the backstory. Our large – and growing – technology team still has a large amount of work to do.  But now is the time to share our work with the broader community and encourage people – including in other industries – to use it for their own applications as it matures (it’s still a young codebase), to contribute to Corda itself, and to contribute to the architectural debate.

We’re looking forward to November 30. This is an exciting time!

Corda: An Introduction

Announcing the Corda Introductory Whitepaper

The Wall Street Journal had a couple of good pieces this morning that describe some of the work we’re doing at R3 and our vision for the future of financial services.

Project Concord is our codename for the overall vision, with Corda as our underlying distributed ledger software.

I first wrote about Corda back in April and we demonstrated it in public for the first time a few weeks later.  Since then, we’ve been continuing to develop the code base in collaboration with our members, trialling it through an ongoing series of proofs-of-concept, prototypes and more advanced deployments, refining the design and maturing our thinking.

As part of this process, we wanted to share more information with the broader community about what we’re doing.  I’m pleased to announce the release of our first whitepaper on Corda: an introductory, non-technical overview that explains our vision, some design choices and outlines the key concepts underpinning the platform.  We’ll follow this up in the coming months with a more detailed technical whitepaper.

whitepaperThe whitepaper, which you can download here, explains how we set ourselves the challenge of starting with the financial industry’s pain points: duplicated, inconsistent data and business logic and redundant business processes – and asked ourselves if we could apply breakthroughs in distributed ledger and blockchain technology to solve them.

Our conclusion is that distributed ledger and blockchain technology represents a once-in-a-generation opportunity to transform the economics of data management across the financial industry. But there’s a problem because the blockchain and distributed ledger platforms that led us to this exciting moment were never designed to solve the problems of financial institutions and do not meet all our needs: we need tight linkage to the legal domain; we have an obligation to prevent client data being shared inappropriately and so can’t send all transactions to all network participants; we must integrate and interoperate with existing financial infrastructure; and more.

Corda is the outcome of the analysis we did on how to achieve as many of the benefits of distributed ledger and blockchain technology as possible but in a way that is sympathetic to and addresses the needs of regulated financial institutions. Corda is intended to be a contribution to the plurality of technologies that will be adopted in the coming years, one that is targeted specifically and with a laser-focus on the needs of financial institutions.

I hope you find the whitepaper interesting and illuminating and we would love to hear your feedback.

 

 

Introducing the R3 Technical Leadership Team

I joined R3 in September as our Chief Technology Officer. Regular readers may have noticed a drop-off in my blogging at precisely the same time.   It turns out that joining a high-profile, fast-growing startup consumes a lot of time..!

In this post, I want to share some early thoughts and to introduce my senior leadership team.  Regular readers of my blog will know that I have thought deeply and written often about the applications of blockchain and distributed ledger technology in finance.  But as I set out on my journey at R3, I tried to imagine myself in a few years, sitting in front of the CIO of one of the world’s largest banks, having a conversation about our project. What would we talk about?  How would I describe what we had built?  How would I explain why we built it one way rather than another?

I figured it would be an extremely difficult conversation if my opening line was: “well… you know….  I built the platform like this because blockchains were cool in 2015”…  No. That simply won’t do.   The rules of engineering and architecture don’t fly out of the window just because somebody pulls out the “shared ledger” trump card.

If we aspire to reduce cost, free up capital, improve controls and enable innovation in finance and beyond, we need to build our vision on more than hype and hope.   So I’ve gone back to basics:   what properties does a technology platform need to possess if it is going to enable the world’s banks – and other firms – to deploy shared platforms to record, manage and report on their contractual agreements with each other and with their customers?   What is the irreducible set of functional requirements we must provide?  What are the non-negotiable non-functional requirements?

So I’ve spent my first few weeks building my leadership team, establishing an Architecture Working Group with our members and developing a detailed view on what a shared ledger for financial firms needs to look like if it’s going to gain widespread adoption and solve real business problems.

In the coming weeks, I’ll share thoughts on these questions.   I’m probably wrong about huge portions of it (I usually am…).  But my strong desire is to have this debate in the open:  just as we’re driving this discussion with our members, we also want to debate this with other practitioners, firms and projects.  Not least, because it’s manifestly obvious that a base “fabric” for the recording of financial events and execution of logic has to be open and if I can persuade you of my vision (or you can persuade me of yours…), perhaps we can work together to drive some standardisation too.  Watch this space.

In the meantime, I’d like to introduce my senior leadership team.

First, I’m delighted to announce that James Carlyle, formerly Chief Engineer at Barclays Personal and Corporate Bank, is joining R3 as our Chief Engineer.  He is almost too-good-to-believe:  he built hugely complex systems for a hugely complex bank, founded two startups and he happens to to be one of the few people I know who can both talk about ethereum and develop for it.

Secondly, I am beyond excited that Mike Hearn has joined us as our Lead Platform Engineer. He brings half a decade of experience of blockchain and cryptocurrency development and over seven years of experience helping run some of Google’s most heavily-trafficked websites.  The combination of deep understanding of blockchain technologies and real-life experience of building rock-solid internet-scale production platforms is truly unmatched in the industry.  And his involvement in the recent bitcoin blocksize debate gives me confidence he can hold his own against a group of very opinionated bank architects…

Thirdly, I would like to welcome Ian Grigg, our Architecture Consultant.   Ian has been building cryptographic ledger platforms for over two decades.  He invented the concept of the “Ricardian Contract”, co-invented the concept of triple-entry accounting and astounds me every day with the experience and perspective he brings to the team.   You would be amazed how many of the concepts in the shared ledger space today can be traced back to Ian’s work.

Fourthly, Tim Swanson joins as our Head of Research. I have to believe there are people in this space who Tim doesn’t know, but I’ve not met one yet.   He teaches me every day that it’s OK to be opinionated, provided you can justify the opinions. And Tim can; his most recent report is a fascinating demonstration. I lean on him heavily for advice and insight and am delighted to have him as a colleague.

They join a fast-growing team, which also includes Jo Lang and Ayoub Naciri, amongst others.

 

… and what about you? We’re hiring!

We are working on the most interesting and exciting project I can imagine in technology today. We’ll be sharing details of our open roles and how to contact us shortly.   In the interim, if you’re interested in working with us, I’d encourage you to think about a few questions that just might come up in interview…

  • If you were building a system to enable multiple parties to come to consensus about the state of an agreement between them and maintain that in lockstep for the life of that agreement, what are some of the most important non-functional requirements you would want to explore to validate your design?
  • If you were building a shared ledger system between large numbers of regulated financial entities with hugely sophisticated IT infrastructures, what would be your approach to co-existence and integration?
  • What would be your answer to the CIO’s follow-up question? “Tell me… why did you build your shared ledger using a blockchain rather than another technology?”

 

 

 

Think Payment Cards are Insecure? Just Wait Until Push-Payments Hit Primetime…

What Brazil’s Boleto Fraud Tells Us About Bitcoin and other Push Solutions

When I explain to people how payment cards work, they are usually aghast. I point out that when you hand your card to a merchant and sign your name or enter your PIN, you’re authorising them to suck funds out of your account and the only thing that stops somebody draining all your money is trust. The picture below shows the standard “four-party” model for payment cards and I stress that the consumer is merely authorising payment; it’s the merchant and all the other actors who actually move the money.

PushSecurity1 

The Payment Card “Four-Party” Model: Consumers authorise merchants to pull money out of their account.

(Aside: I’ve never understood why this is called the four-party model. I count at least five parties on that picture…)

Online, the problem is more stark: you type your card details, including your CVV2 “secret number on the back” into your browser and hope for the best: you have to trust the merchant, their IT supplier, the acquiring bank, their third-party processor, the card network and your own card issuer – and everybody who works for them and has access to their systems. If a bad guy gets hold of your card details at any point in this process, they could drain your account. The picture below shows the scope of all the entities with access to your critical card information:

PushSecurity2

Your Primary Account Number – PAN – passes through the hands of pretty much everybody involved in processing the transaction.

It seems mad: why would you spray such sensitive information all over the place willy-nilly? Whoever thought it was a good idea to build the system this way?!  Except… the system works.

Fraud is surprisingly low given the design – and consumers get compensated if something goes wrong. And the design isn’t actually as mad as it seems: how else would you build a consumer payment network in a world where you can’t assume the consumer has a smart device with guaranteed network connectivity?

Payment card networks also have the advantage of decades of experience and refinement. For example, the Payment Card Industry Data Security Standards (PCI-DSS) lay down rules and guidance on how to protect the sensitive card data. The EMV smartcard standards make it harder to clone cards. Issuers have sophisticated heuristics to block suspicious transactions. And forthcoming moves to standardise “tokenisation” (something I should blog about one day) will further mitigate the risk of card details getting into the wrong hands. So an underlying architecture that appears wholly unsuited to the web age has actually been patched up to be good enough (but not perfect – and it still has lots of problems)

The Push Pay Revolution – a better way to do retail payments?

As I’ve written often, there is an entirely different way to design a retail payment system, one where the consumer doesn’t have to trust nearly as many people. I call these sort of payments push payments.  Bitcoin follows this model, as does M-Pesa, iDEAL, ZAPP and the Boleto system in Brazil. The defining characteristic of push-payments is that the consumer is in the driving seat.

With Push, it is the consumer who instructs a payment – from their bank or telco or Bitcoin wallet

This is unlike pull-payments, where the consumer merely authorises the merchant to pull the funds from their account.   The difference may seem subtle but it turns out to be hugely important. The picture to have in mind for push-payments is this one:

PushSecurity4

Push payments have a very different threat model to pull payments. Now the consumer only has to trust their payment provider and their own device.

In previous articles, I talked about the benefits of push payments in terms of innovation and the reduced need to trust quite so many people.   In this post, I look at one of the downsides: push payments can be compromised in hard-to-detect ways if they are not implemented really carefully.

So what’s the problem with push payments?

First, let’s remind ourselves about what we do have to trust and what we don’t have to trust in the pull world.

In the pull world, the consumer has to trust everybody else – and, as I’ve discussed above, there are various safeguards in place to fix things when they inevitably go wrong. One might argue that the safeguards don’t always work and that they come at a cost. Both arguments are, of course, valid but let’s leave them to one side for now.

In the push world, it’s different. The way it’s supposed to work is like this:

  • Step 1: The merchant “tells” the consumer how much they’d like to be paid and to where the payment should be sent. Examples:
    • With M-Pesa, this is usually done in-person, verbally
    • With Bitcoin, it is either done ad-hoc or via a QR-code displayed by the merchant or via the emerging BIP70

To illustrate the point, here is a picture of me in Shoreditch trying to tell a Bitcoin ATM where to send some Bitcoins I’d bought. On my laptop screen is a QR code that represents my Bitcoin wallet address. Note how it’s me as the Bitcoin receiver who is telling the sender (the ATM) where to send the coins.     In the more common case, where I am paying Bitcoins, this means it is the merchant who has to show the QR code to me. I need to know where to send the money to.

PushSecurity5

This is me using a QR code on my laptop to tell a Bitcoin ATM where to send my Bitcoins. (The Apple Bitcoin ban was still in force when we took this photo… so I had to use my laptop rather than my iPhone…)

  • Step 2: Once the consumer has the payment request, they use a program or app on their smart device (laptop, smartphone, whatever) to instruct the payment. Examples:
    • An M-Pesa user launches the M-Pesa SIM app and instructs the payment
    • A Bitcoin user pastes the destination address and value into their Bitcoin wallet
    • … or uses their wallet to read the recipient’s QR code
    • … or opens the BIP70 Payment Request with their wallet)

When you put it like this, push payments are obviously superior, right? The consumer is in control, they don’t have to trust all those people and there’s no danger of a rogue agent sucking all their money out of their account!

Not so fast…

The analysis above neglects one small, but rather important, fact: devices get hacked.

In the pull model, the only devices that can get hacked are those inside the “circle of trust” – your plastic card is pretty impregnable.  And as the utterly disastrous Target breach suggests, consumers were made whole when the disaster happened. It was the big firms who messed up who suffered the consequences.  

Yes… I know this is counterintuitive… you must be asking yourselves: “is this guy seriously arguing that the Target disaster is an argument in favour of the current payment card model?!” Obviously, no…. the episode was clearly a catastrophe and it was really, really bad.   But… it did eventually get sorted out and the roll-out of EMV, tokenisation and better enforcement of PCI-DSS should reduce the risks of something similar in the future.   So I raise this merely as prelude to the push scenario.

Now ask yourself what happens if a device gets hacked in the push scenario.  The obvious question is: which device?   Well… the only device in the circle-of-trust this time is the consumer’s smartphone. Uh-oh.

This is the device from which we’re instructing real-time payments, right? The one that could be riddled with malware?

This might have been merely a theoretical risk…. And then the Brazilian Boleto fraud happened.

RSA have a great write-up of a country-scale real-life example of what can go wrong when push-payment systems get breached… and it’s really scary.

The Brazilian Boleto system is very cool.   At core, it is a way for fund requestors (utility firms, etc) to send a payment request to consumers. The request is known as a Boleto and they can be physical or electronic.

PushSecurity7

A Brazilian Boleto. Think of it as a mainstream equivalent of a Bitcoin BIP70 Payment Request…

The idea is this: the Boleto has details of the payment request and includes details of how much to pay and to where.   This is in coded text format and a bar code… basically, something that a consumer can take and feed into their banking app: scan the code with your mobile banking app, approve and you’re done. Or you could take it to a bank branch. And if you’re online, you could copy and paste the code into your online banking website and achieve the same end.

Except… the RSA paper shows all the ways it can and has gone wrong.

First, there’s a simple problem of authentication. How do you know the Boleto really did come from who it says it’s from?   The RSA paper documents examples of people receiving Boletos via email that look convincingly genuine but which have the fraudster’s payment account details in place of the firm from which they purport to come.

This is a real problem but it’s nothing new… it’s not really any different to fake websites that masquerade as real ones. We solved it in the pull world with SSL certificates and the like for websites. And the Bitcoin Payment Protocol includes the option to use the same PKI system, for precisely these reasons.

However, the RSA paper also discusses another attack – and this one’s scarier.

This second attack comes in the form of malware that runs in the consumer’s browser.   When it sees a document that looks like a Boleto, it silently changes the details that the consumer sees on their screen: the payment details are changed from the genuine recipient to the attacker. So when the consumer copies and pastes the details into their banking app, it’s the attacker’s account they’re sending the money to.

Variations on this theme are included in the paper but they all amount to the same thing: if the consumer’s device is compromised then it’s game over. And you don’t even need to compromise the whole device or get root-access… you just need to compromise the browser in this scenario.

There are various mitigation mechanisms one can implement (e.g. tying the payment instruction to a signed representation of the payment request and so forth) but the underlying problem remains: if you’re using the consumer device to instruct payments, you have an issue if that device is compromised.

Now, this risk is perhaps over-blown: the risks identified here apply equally to standalone mobile banking apps and we happily run these on mobile devices today, albeit with the belief that their bank will bail them out if something goes wrong. (It’s no surprise that banks are big users of technology like IBM Trusteer).

Similarly, Bitcoin users run their wallets on their devices, in the full knowledge that there is nobody who will bail them out if malware runs amok on the device.  

But I think the two-step dance of an end-to-end push payment request/instruction – where the device is responsible for turning the request into the instruction – is something new that needs deeper study.  So I think the Boleto story tells us is that we need to think very hard about things like:

  • User experience: how is the linkage between Step 1 (receive and authenticate request) and Step 2 (populate and instruct payment) executed and communicated to the user? If step 1 is done by a different app to step 2, what is the hand-off? What security assumptions are being msde?
  • Validation and Reconciliation: what work should (can?) the “network” do to validate that a payment instruction purporting to be in response to a payment request, really is traceable to that request?
  • Malware detection systems: what new behaviours should anti-virus and other technologies be looking out for?
  • Wallet providers: which scenarios are you willing and able to protect your consumers against?

It is possible that this is just a variation on the age-old theme that end-point security is hard – but when things like the Boleto fraud happen, we should use it as an opportunity to look at the other systems being built along similar lines and ask: are there any lessons we can learn and apply?

Welcome to Bitcoin Island

Forget currencies and commodities… perhaps the right analogy for Bitcoin is LAND!

Oleg Andreev posted an insightful tweet the other day:

You could argue this is a trivial observation: how else could it work?!  But thinking in terms of ownership and protocols for transfer of ownership is a surprisingly helpful way to think about how the system works.  And that’s because the “protocol of ownership” insight means there is a whole other world of history, tradition and precedent to learn from: land!

Here are some observations to motivate the thought:

  • In the end-state the quantity of Bitcoin will be fixed, just like land.
  • Bitcoin is not perfectly fungible and neither is land
  • Bitcoin is not “consumed” through use – just transformed and transferred. This is similar to land and dissimilar to many commodities, which are consumed (or at least degraded) through use.

OK – not a perfect parallel but let’s go with it for now…. What happens if we think about Bitcoin through the lens of land?

Well, first, it allows us to think about coins that haven’t been mined yet… we can think of them as parcels of land on “Bitcoin Island” that haven’t been released yet:

Image

The “Land Interpretation” of Bitcoin. Think “Bitcoin Island”

Second, it helps us put some intuition behind the concept of the “unspent transaction output”.  These are Bitcoins that have been sent somewhere but not yet themselves been spent.  So the set of all unspent transaction outputs (UTXOs) can be thought of as the latest state of every Bitcoin that has ever been mined.

The UTXO is absolutely crucial to everything in Bitcoin and yet very few people think in these terms, talking instead about misleading terms like “address balances” and so forth.

But the interesting thing is: if we take a “land interpretation” of Bitcoin, then UTXOs have a really simple explanation: they are plots of land! And Bitcoin transactions are simply actions that merge or split these plots of land.

Imagine I own twenty Bitcoins. My Bitcoin wallet software will show a “balance” of twenty. But it’s likely that this balance actually consists of multiple unspent-transaction outputs. Even if I had bought all twenty Bitcoins in one go, it’s likely that the seller merged several smaller UTXOs that added up in total to twenty Bitcoins.    So perhaps I received three plots of “land”: 7 Bitcoins in one, 7 in another and 6 in the third.  My total “holdings” are 20 – but it is formed from three “UTXOs”.

Perhaps my holdings on Bitcoin Island look like this:

Image

We can think of Unspent Transaction Outputs as plots of land on “Bitcoin Island”. Plots A, B, C represent three unspent transaction outputs controlling 20 Bitcoins

And now it’s possible to teach people about Bitcoin transactions without completely confusing them!

Imagine I wanted to buy a second-hand car for 11 of my Bitcoins.  Let’s also imagine that I pay a transaction fee of 1 BTC to keep things simple (a HUGE over-estimate, of course)

I need to do a few things:

  • Step One: I need to prove ownership of the coins I’m trying to spend
  • Step Two: I need to say how the coins are going to be allocated – how many am I sending and to where? 11 to the seller, 8 back to me and 1 for the miner in this case.
  • Step Three: I need to specify what the new “owners” will need to do to prove they do indeed own the coins. In other words, I need to specify what they will need to do in their Step one when they try to spend their coins in the future.

I do this in Bitcoin by issuing a transaction that accomplishes all three steps in one.  Here’s what it might say:

“I own three unspent transaction outputs: A, B and C. In total they represent twenty Bitcoins. Here is my proof I am entitled to spend A.  Here is my proof I am entitled to spend B.  Here is my proof I am entitled to spend C. I hereby reshape my plot into two new plots: one plot 8 units in size, which I call X and a second plot 11 units in size, which I call Y. Whoever mines this transaction can claim the remaining 1 BTC. If you can satisfy the following conditions then you will be considered to own X: … . If you can satisfy the following conditions then you will be considered to own Y: …”

I will set the conditions so that only the seller of the car could satisfy the Y condition and so that only I could satisfy the X condition (that’s my change and I don’t want anybody else spending it!)

The end result is that I have simply rearranged the land holdings:

Image

Transaction outputs A, B, C are now spent, replaced by two new unspent transaction outputs: X and Y.  X is my change, Y now belongs to the car dealer and F goes to the miner.  

 

But we can go further… we can now have an informed discussion about what “ownership” means in Bitcoin.  When I “send” Bitcoins to somebody, I’m not assigning ownership to an individual.  What I’m actually doing is laying down a condition – and anybody who can satisfy that condition will be considered the owner.

Now, normally, the condition is very simple.  It says something like:

“To spend this output you must prove you know the public key that hashes to the following address: …   And you must prove you own the corresponding private key by issuing a digital signature”.  

That’s what the “OP_DUP OP_HASH160 …” stuff you sometimes see is usually saying.

But the conditions can be far more complex than that…. It’s all down to how you write your transaction.

Where is this going?

OK – so thinking of Bitcoin in terms of land helps us build some intuition around UTXOs, which we can think of as “parcels of land on Bitcoin Island” and we see that Bitcoin transactions are really just a way to merge or split these parcels of land and impose conditions that allow people to assert ownership.

And now things get really interesting.  Because there are all sorts of interesting phenomena that happen with land transactions that we can use to think about Bitcoin problems.

Fungibility

The land analogy works because Bitcoins are not perfectly fungible. Sure – there are projects trying to overcome this but this feels like an arms race between developers and law-enforcement agencies. To the extent that fungibility remains imperfect, what drivers could force different “land parcels” to have different values?

For me, the biggest topic on the horizon for fungibility is “coin tainting”, “whitelisting” and the other schemes intended to “tag” Bitcoin addresses or UTXOs.

I see these schemes as directly analogous to concepts like land “blight” on the one hand and maybe “planning gain” on the other.  For example, if you own a “plot of Bitcoin land” that has been “whitelisted” by an exchange or finance firm such that you can access their services, presumably your “plot” would be worth more than one that didn’t have that property?

It is perhaps no surprise that the fungibility issue is so hot right now.

Mineral Rights and Colored Coins

Two pieces of seemingly identical land can be worth vastly different sums: if one is sitting on oil and the owner has mineral rights, a purchaser will be willing to pay them more for their land than if it didn’t! Perhaps this is a useful analogy for colored coins: two identical Bitcoins can trade for different prices if one of them has been “colored” by a trusted issuer. What are the taxation implications? What happens when projects trying to add coin coloration to Bitcoin conflict with projects trying to create fungibility?

Alt coins

Perhaps Altcoins are just different islands! If there is a Bitcoin Island, then presumably Litecoin has Litecoin Island and Dogecoin has Dogecoin Island?

This interpretation now helps us think more clearly about the role and value of altcoins.   Perhaps the innate characteristic of a currency (faster confirmation? Use of scrypt?) makes the island a more attractive place to live. But if all the infrastructure and population is on Bitcoin Island then these features may not be enough.  Who knows.

Charges and Liens

It is possible to impose conditions on land parcels in many jurisdictions. A mortgage company can prevent sale of land unless the debt is settled and some landowners in the UK have been dismayed to discover that their land ownership came with an expensive obligation to pay for the upkeep of a local church.

In some cases, the obligation is short-lived (e.g. the mortgage charge) but in others, it persists across transactions (e.g. chancel repair liability).    A question I don’t know the answer to is: can you write a Bitcoin transaction that imposes conditions on a UTXO that propagate?  That is: can you write a transaction such that whoever spends the UTXO must impose the same condition on their transaction output?

Conclusion

Of course, the land analogy is imperfect but I do think there is something to it.  If nothing else, the mental image of “Bitcoin Island” with UTXOs being the plots of land feels like a really useful one… it has certainly helped my understanding…