On Distributed Databases and Distributed Ledgers

Why can’t companies wanting to share business logic and data just install a distributed database? What is the essential difference between a distributed database and a distributed ledger?

Last month, I shared the thinking that led to the design of Corda, which we at R3 will be open sourcing on November 30; and Mike Hearn and I were interviewed by Brian and Meher of Epicenter last week. We’ve been delighted by the response and are looking forward to working with those seek to build on Corda, help influence its direction or contribute to its development and maturation;  there’s a lot of work ahead of us!

But one or two observers have asked a really good question. They asked me: “Aren’t you just reimplementing a distributed database?!”

The question is legitimate: if you strip away the key assumptions underpinning systems like Bitcoin and Ethereum, are you actually left with anything? What is actually different between a distributed ledger platform such as Corda and a traditional distributed database?

The answer lies in the definition I gave in my last blogpost and it is utterly crucial since it defines an entire new category of data management system:

“Distributed ledgers – or decentralised databases – are systems that enable parties who don’t fully trust each other to form and maintain consensus about the existence, status and evolution of a set of shared facts”

“Parties who don’t fully trust each other” is at the heart of this. To see why, let’s compare distributed databases and Corda.

Comparing Corda to a distributed database

In a distributed database, we often have multiple nodes that cooperate to maintain a consistent view for their users.   The nodes may cooperate to maintain partitions of the overall dataset or they may cooperate to maintain consistent replicas but the principle is the same:  a group of computers, invariably under the control of a single organisation, cooperate to maintain their state.  These nodes trust each other.   The trust boundary is between the distributed database system as a whole and its users.    Each node in the system trusts the data that it receives from its peers and nodes are trusted to look after the data they have received from their peers.  You can think of the threat model as all the nodes shouting in unison: “it’s us against the world!”

This diagram is a stylised representation of a distributed database:


In a distributed database, nodes cooperate to maintain a consistent view that they present to the outside world; they cooperate to maintain rigorous access control and they validate information they receive from the outside world.

So it’s no surprise that distributed databases are invariably operated by a single entity: the nodes of the system assume the other nodes are “just as diligent” as them: they freely share information with each other and take information from each other on trust. A distributed database operated by mutually distrusting entities is almost a contradiction in terms.

And, of course, if you have a business problem where you are happy to rely on a central operator to maintain your records – as you sometimes can in finance it should be said – then a distributed database will do just fine: let the central operator run it for you.  But if you need to maintain your own records, in synchrony with your peers, this architecture simply won’t do.

And there are huge numbers of situations where we need to maintain accurate, shared records with our counterparts. Indeed, a vast amount of the cost and inefficiency in today’s financial markets stems from the fact that it has been so difficult to achieve this. Until now.

Corda helps parties collaborate to maintain shared data without fully trusting each other

Corda is designed to allow parties to collaborate with their peers to maintain shared records, without having to trust each other fully. So Corda faces a very different world to a distributed database.

A Corda node can not assume the data it receives from a peer is valid: the peer is probably operated by a completely different entity and even if they know who that entity is, it’s still extremely prudent to verify the information.   Moreover, if a Corda node sends data to another node, it must assume that node might print it all in an advert on the front page of the New York Times.

The trust boundaries – the red curves in the diagram- are drawn in a completely different place!


In Corda, nodes are operated by different organisations and do NOT trust each other; but the outcome is still a consistent view of data.

To repeat, because this distinction is utterly fundamental:  nodes of a distributed database trust each other and collaborate with each other to present a consistent, secure face to the rest of the world.   By contrast, Corda nodes can not trust each other and so must independently verify data they receive from each other and only share data they are happy to be broadly shared.

And so we call Corda a distributed ledger, to distinguish it from distributed databases. A distributed ledger that is designed painstakingly for the needs of commercial entities.

Put more simply: you simply can’t build the applications we envisage for Corda with traditional database technology.  And that’s what makes this new field so exciting.

R3 Corda: What makes it different

As reported by Reuters last week, Corda, the Distributed Ledger platform we’ve been working hard on at R3 for the last year at will be open sourced on November 30.

What is it? Why are we building it? What happens next?

Corda is a distributed ledger platform designed and built from the ground up for the recording and automation of legal agreements between identifiable parties. It is heavily influenced by the requirements of the financial industry but we believe the community will find the underlying architecture will lend itself to a broad range of applications.

Corda is quite unlike any other Distributed Ledger platform that currently exists. So we’ll be releasing lots of information in the coming weeks and months. To understand why it looks the way it does, I thought I’d share the journey we went on to build it.  In subsequent articles, the team and I will share more detail about how it works and what to look for when it’s released on November 30.

But first, some history.

The very first decision made by the Steering Committee of the R3 consortium was to establish our Architecture Working Group, which I chair.  This group consists of hundreds of senior architects, technologists and developers, many with decades of experience in a dazzling array of areas, from across our membership over over seventy financial institutions.

We were given a simple-sounding mission:

“To establish the architecture for an open, enterprise-grade, shared platform for the immutable recording of financial events and execution of logic”.

There is quite a lot packed into that sentence..!  Let’s look at just two parts:

  • “Open”
    • I stated publicly in April that we would open-source Corda and I was serious. Our mandate, from our member banks, was that whatever base platforms we selected, built or adopted had to be open. We’re delivering on this commitment with the open-sourcing of Corda on November 30.
  • “Immutable recording of financial events and execution of business logic”
    • Notice what this doesn’t say. It doesn’t say “blockchain”.  Heck: it doesn’t even say “distributed ledger”!    Instead, it tries to get to the heart of what we think is the essence of this exciting new field.  And that’s what I want to talk about in this blog post.

We don’t like solutions looking for problems

I wanted us to be precise about what this field is all about.  After all, and as I wrote when we first announced Corda, Satoshi Nakamoto didn’t wake up one morning thinking: “I really need a blockchain!”.  No: he started with a well-defined business problem and engineered a solution to solve that problem.  And if you need a system of censorship-resistant digital cash, then Satoshi’s design – Bitcoin – is the elegant solution and it’s available today.

And that’s why Ethereum, to take another example, looks so different to Bitcoin.  Vitalik Buterin and his colleagues started with a different business problem, which I characterise as “I want an unstoppable world computer that can execute business logic and move value autonomously” and guess what? They ended up with a very different design!  Now sure:  there are many similarities between Bitcoin and Ethereum but also a lot of differences.

I was determined that we would not fall into the trap of taking technologies designed to solve completely different problems and blindly apply them to banking.  That way lies madness.

So we drove two key pieces of work: 1) characterising exactly what is new about this field and 2) identifying precisely where in finance it may have most applicability.

And the answer, as I outlined back in April, is that there is something genuinely new in this space and it’s something that is massively relevant to the financial system.

The definition I think best captures this is as follows:

“Distributed ledgers – or decentralised databases – are systems that enable parties who don’t fully trust each other to form and maintain consensus about the existence, status and evolution of a set of shared facts”

Let’s first test that this definition works for existing public systems:

Bitcoin: the participants don’t know each other’s identities and come to consensus about how many bitcoins there are, which addresses own them and what needs to happen for any of them to be spent without having to trust each other.  Check!

Ethereum: the participants don’t know each other’s identities and come to consensus about the state of a virtual computer.  Check!

In those systems’ cases, they achieve these outcomes in ways with which we’re both familiar and which address requirements related to the environment in which those systems are expected to run.

But how about finance… parties who don’t fully trust each other but whom need to be in consensus about a set of shared facts?

Where do we have that problem?

Erm… how about everywhere..?!!

It’s perhaps only a slight exaggeration to suggest that the financial industry is pretty much defined by the web of contracts that exists between its participants:  I deposit money with a bank? There’s a contract there that says the bank owes me that money.  You and I enter into a Credit Default Swap? There’s a contract there that describes our mutual rights and obligations.  And they’re recorded and managed in multiple places, on different systems, managed by different firms and it costs a fortune to keep them all in sync.

The shared facts in finance are the existence and state of financial agreements – ie contracts.

And the need for consensus is what amounts to the twenty-first century’s “paperwork crisis”: the tens of billions of dollars spent annually maintaining and managing the duplicated records that each firm maintain about the same deals.  The same information about a deal is recorded multiple times across these parties and in situations where a centralised solution can’t be deployed, which is in lots of places, small armies are required to ensure that these disparate records agree with each other, get updated correctly and in synchrony – and deal with the issues when they don’t.

A ha!  So now we have something phenomenally exciting: a new technology for establishing and maintaining consensus between parties who don’t trust each other.  And a multi-billion dollar business problem crying out for this solution!

There’s only one minor problem…

Public systems like Bitcoin were not designed to solve these problems. They’re excellent at what they do; but we’re doing something else.

And you only need to take a cursory look at the architecture of various public blockchain systems to see why this might be the case.  My business problem amounts to ensuring the Bank of Alice and Bank of Bob agree about a trade they just did and that it settles automatically and correctly.   A solution which not only shares this confidential data with every other bank in the world but which also requires them to process the deal and maybe even validate it doesn’t meet my needs. And yet… that’s how every single public platform back in 2015 worked.

Perhaps those architectures can be heavily re-engineered to solve such problems, as some groups are attempting but it’s not an obvious starting point, especially when you then layer on all the other requirements we identified.

So there’s a problem: Bitcoin and its successors taught us that a new way of building distributed systems was possible: one where mutually distrusting parties can maintain a shared database.  We identified a hitherto unsolved problem in finance. And yet the technology that existed simply wasn’t designed for this.


Coding, not talking

The reality is that finding fault is easy; proposing workable solutions is altogether harder.   So simply going out and shouting about how 2015’s platforms didn’t solve our problems was hardly a way to make friends.  No.  We needed to do better than that.

So once we had decided we needed to prototype the alternate approach we had identified, we made a critical decision to buttress my leadership team of James Carlyle and Ian Grigg:  we brought in Mike Hearn.

And he drove the prototyping effort to explore these concepts in the only way that gives you certainty that it can be done: by proving it in code.  As Mike enjoys reminding me: when it comes to core concepts, talk is cheap; at some point, the talking has to give way to coding.


Early results were promising: the reductive, bottom-up approach we took to architecture and design, which is explored in our introductory whitepaper and on which we’ll elaborate in the coming weeks, was solid: we could model a diverse range of instruments; the design would allow for significant parallel processing; we did not need to send all data to all participants in all scenarios; the use of a mainstream virtual machine and its libraries led to high developer productivity; we were able to support multiple consensus providers on a single network; the use of a flat, point-to-point queue-based, peer-to-peer network mapped well to real business scenarios; and more.

We worked with our members to test the maturing codebase in a variety of contexts: interfacing Ricardian Contracts and Smart Contracts in the context of an Interest Rate Swap with Barclays and others; managing trade finance flows; and more.

And this focus on validated client requirements and a willingness to question some hitherto sacred beliefs (we have no blocks! we have no miners! we don’t put ephemeral data in the consensus layer! we allow per-transaction specification of consensus providers!) led to a unique design.

Had Corda ended up being a minor variation on an existing platform or a me-too copy of something else, what would have been the point in pursuing the work?  But that isn’t what happened: we ended up with something quite distinct, something we believe is singularly well-suited to a wider variety of financial-services use-cases and something  adapted to the practical reality that the industry is regulated and some rules simply aren’t going to change overnight.

So that’s the backstory. Our large – and growing – technology team still has a large amount of work to do.  But now is the time to share our work with the broader community and encourage people – including in other industries – to use it for their own applications as it matures (it’s still a young codebase), to contribute to Corda itself, and to contribute to the architectural debate.

We’re looking forward to November 30. This is an exciting time!

Corda: An Introduction

Announcing the Corda Introductory Whitepaper

The Wall Street Journal had a couple of good pieces this morning that describe some of the work we’re doing at R3 and our vision for the future of financial services.

Project Concord is our codename for the overall vision, with Corda as our underlying distributed ledger software.

I first wrote about Corda back in April and we demonstrated it in public for the first time a few weeks later.  Since then, we’ve been continuing to develop the code base in collaboration with our members, trialling it through an ongoing series of proofs-of-concept, prototypes and more advanced deployments, refining the design and maturing our thinking.

As part of this process, we wanted to share more information with the broader community about what we’re doing.  I’m pleased to announce the release of our first whitepaper on Corda: an introductory, non-technical overview that explains our vision, some design choices and outlines the key concepts underpinning the platform.  We’ll follow this up in the coming months with a more detailed technical whitepaper.

whitepaperThe whitepaper, which you can download here, explains how we set ourselves the challenge of starting with the financial industry’s pain points: duplicated, inconsistent data and business logic and redundant business processes – and asked ourselves if we could apply breakthroughs in distributed ledger and blockchain technology to solve them.

Our conclusion is that distributed ledger and blockchain technology represents a once-in-a-generation opportunity to transform the economics of data management across the financial industry. But there’s a problem because the blockchain and distributed ledger platforms that led us to this exciting moment were never designed to solve the problems of financial institutions and do not meet all our needs: we need tight linkage to the legal domain; we have an obligation to prevent client data being shared inappropriately and so can’t send all transactions to all network participants; we must integrate and interoperate with existing financial infrastructure; and more.

Corda is the outcome of the analysis we did on how to achieve as many of the benefits of distributed ledger and blockchain technology as possible but in a way that is sympathetic to and addresses the needs of regulated financial institutions. Corda is intended to be a contribution to the plurality of technologies that will be adopted in the coming years, one that is targeted specifically and with a laser-focus on the needs of financial institutions.

I hope you find the whitepaper interesting and illuminating and we would love to hear your feedback.



Introducing the R3 Technical Leadership Team

I joined R3 in September as our Chief Technology Officer. Regular readers may have noticed a drop-off in my blogging at precisely the same time.   It turns out that joining a high-profile, fast-growing startup consumes a lot of time..!

In this post, I want to share some early thoughts and to introduce my senior leadership team.  Regular readers of my blog will know that I have thought deeply and written often about the applications of blockchain and distributed ledger technology in finance.  But as I set out on my journey at R3, I tried to imagine myself in a few years, sitting in front of the CIO of one of the world’s largest banks, having a conversation about our project. What would we talk about?  How would I describe what we had built?  How would I explain why we built it one way rather than another?

I figured it would be an extremely difficult conversation if my opening line was: “well… you know….  I built the platform like this because blockchains were cool in 2015”…  No. That simply won’t do.   The rules of engineering and architecture don’t fly out of the window just because somebody pulls out the “shared ledger” trump card.

If we aspire to reduce cost, free up capital, improve controls and enable innovation in finance and beyond, we need to build our vision on more than hype and hope.   So I’ve gone back to basics:   what properties does a technology platform need to possess if it is going to enable the world’s banks – and other firms – to deploy shared platforms to record, manage and report on their contractual agreements with each other and with their customers?   What is the irreducible set of functional requirements we must provide?  What are the non-negotiable non-functional requirements?

So I’ve spent my first few weeks building my leadership team, establishing an Architecture Working Group with our members and developing a detailed view on what a shared ledger for financial firms needs to look like if it’s going to gain widespread adoption and solve real business problems.

In the coming weeks, I’ll share thoughts on these questions.   I’m probably wrong about huge portions of it (I usually am…).  But my strong desire is to have this debate in the open:  just as we’re driving this discussion with our members, we also want to debate this with other practitioners, firms and projects.  Not least, because it’s manifestly obvious that a base “fabric” for the recording of financial events and execution of logic has to be open and if I can persuade you of my vision (or you can persuade me of yours…), perhaps we can work together to drive some standardisation too.  Watch this space.

In the meantime, I’d like to introduce my senior leadership team.

First, I’m delighted to announce that James Carlyle, formerly Chief Engineer at Barclays Personal and Corporate Bank, is joining R3 as our Chief Engineer.  He is almost too-good-to-believe:  he built hugely complex systems for a hugely complex bank, founded two startups and he happens to to be one of the few people I know who can both talk about ethereum and develop for it.

Secondly, I am beyond excited that Mike Hearn has joined us as our Lead Platform Engineer. He brings half a decade of experience of blockchain and cryptocurrency development and over seven years of experience helping run some of Google’s most heavily-trafficked websites.  The combination of deep understanding of blockchain technologies and real-life experience of building rock-solid internet-scale production platforms is truly unmatched in the industry.  And his involvement in the recent bitcoin blocksize debate gives me confidence he can hold his own against a group of very opinionated bank architects…

Thirdly, I would like to welcome Ian Grigg, our Architecture Consultant.   Ian has been building cryptographic ledger platforms for over two decades.  He invented the concept of the “Ricardian Contract”, co-invented the concept of triple-entry accounting and astounds me every day with the experience and perspective he brings to the team.   You would be amazed how many of the concepts in the shared ledger space today can be traced back to Ian’s work.

Fourthly, Tim Swanson joins as our Head of Research. I have to believe there are people in this space who Tim doesn’t know, but I’ve not met one yet.   He teaches me every day that it’s OK to be opinionated, provided you can justify the opinions. And Tim can; his most recent report is a fascinating demonstration. I lean on him heavily for advice and insight and am delighted to have him as a colleague.

They join a fast-growing team, which also includes Jo Lang and Ayoub Naciri, amongst others.


… and what about you? We’re hiring!

We are working on the most interesting and exciting project I can imagine in technology today. We’ll be sharing details of our open roles and how to contact us shortly.   In the interim, if you’re interested in working with us, I’d encourage you to think about a few questions that just might come up in interview…

  • If you were building a system to enable multiple parties to come to consensus about the state of an agreement between them and maintain that in lockstep for the life of that agreement, what are some of the most important non-functional requirements you would want to explore to validate your design?
  • If you were building a shared ledger system between large numbers of regulated financial entities with hugely sophisticated IT infrastructures, what would be your approach to co-existence and integration?
  • What would be your answer to the CIO’s follow-up question? “Tell me… why did you build your shared ledger using a blockchain rather than another technology?”




Think Payment Cards are Insecure? Just Wait Until Push-Payments Hit Primetime…

What Brazil’s Boleto Fraud Tells Us About Bitcoin and other Push Solutions

When I explain to people how payment cards work, they are usually aghast. I point out that when you hand your card to a merchant and sign your name or enter your PIN, you’re authorising them to suck funds out of your account and the only thing that stops somebody draining all your money is trust. The picture below shows the standard “four-party” model for payment cards and I stress that the consumer is merely authorising payment; it’s the merchant and all the other actors who actually move the money.


The Payment Card “Four-Party” Model: Consumers authorise merchants to pull money out of their account.

(Aside: I’ve never understood why this is called the four-party model. I count at least five parties on that picture…)

Online, the problem is more stark: you type your card details, including your CVV2 “secret number on the back” into your browser and hope for the best: you have to trust the merchant, their IT supplier, the acquiring bank, their third-party processor, the card network and your own card issuer – and everybody who works for them and has access to their systems. If a bad guy gets hold of your card details at any point in this process, they could drain your account. The picture below shows the scope of all the entities with access to your critical card information:


Your Primary Account Number – PAN – passes through the hands of pretty much everybody involved in processing the transaction.

It seems mad: why would you spray such sensitive information all over the place willy-nilly? Whoever thought it was a good idea to build the system this way?!  Except… the system works.

Fraud is surprisingly low given the design – and consumers get compensated if something goes wrong. And the design isn’t actually as mad as it seems: how else would you build a consumer payment network in a world where you can’t assume the consumer has a smart device with guaranteed network connectivity?

Payment card networks also have the advantage of decades of experience and refinement. For example, the Payment Card Industry Data Security Standards (PCI-DSS) lay down rules and guidance on how to protect the sensitive card data. The EMV smartcard standards make it harder to clone cards. Issuers have sophisticated heuristics to block suspicious transactions. And forthcoming moves to standardise “tokenisation” (something I should blog about one day) will further mitigate the risk of card details getting into the wrong hands. So an underlying architecture that appears wholly unsuited to the web age has actually been patched up to be good enough (but not perfect – and it still has lots of problems)

The Push Pay Revolution – a better way to do retail payments?

As I’ve written often, there is an entirely different way to design a retail payment system, one where the consumer doesn’t have to trust nearly as many people. I call these sort of payments push payments.  Bitcoin follows this model, as does M-Pesa, iDEAL, ZAPP and the Boleto system in Brazil. The defining characteristic of push-payments is that the consumer is in the driving seat.

With Push, it is the consumer who instructs a payment – from their bank or telco or Bitcoin wallet

This is unlike pull-payments, where the consumer merely authorises the merchant to pull the funds from their account.   The difference may seem subtle but it turns out to be hugely important. The picture to have in mind for push-payments is this one:


Push payments have a very different threat model to pull payments. Now the consumer only has to trust their payment provider and their own device.

In previous articles, I talked about the benefits of push payments in terms of innovation and the reduced need to trust quite so many people.   In this post, I look at one of the downsides: push payments can be compromised in hard-to-detect ways if they are not implemented really carefully.

So what’s the problem with push payments?

First, let’s remind ourselves about what we do have to trust and what we don’t have to trust in the pull world.

In the pull world, the consumer has to trust everybody else – and, as I’ve discussed above, there are various safeguards in place to fix things when they inevitably go wrong. One might argue that the safeguards don’t always work and that they come at a cost. Both arguments are, of course, valid but let’s leave them to one side for now.

In the push world, it’s different. The way it’s supposed to work is like this:

  • Step 1: The merchant “tells” the consumer how much they’d like to be paid and to where the payment should be sent. Examples:
    • With M-Pesa, this is usually done in-person, verbally
    • With Bitcoin, it is either done ad-hoc or via a QR-code displayed by the merchant or via the emerging BIP70

To illustrate the point, here is a picture of me in Shoreditch trying to tell a Bitcoin ATM where to send some Bitcoins I’d bought. On my laptop screen is a QR code that represents my Bitcoin wallet address. Note how it’s me as the Bitcoin receiver who is telling the sender (the ATM) where to send the coins.     In the more common case, where I am paying Bitcoins, this means it is the merchant who has to show the QR code to me. I need to know where to send the money to.


This is me using a QR code on my laptop to tell a Bitcoin ATM where to send my Bitcoins. (The Apple Bitcoin ban was still in force when we took this photo… so I had to use my laptop rather than my iPhone…)

  • Step 2: Once the consumer has the payment request, they use a program or app on their smart device (laptop, smartphone, whatever) to instruct the payment. Examples:
    • An M-Pesa user launches the M-Pesa SIM app and instructs the payment
    • A Bitcoin user pastes the destination address and value into their Bitcoin wallet
    • … or uses their wallet to read the recipient’s QR code
    • … or opens the BIP70 Payment Request with their wallet)

When you put it like this, push payments are obviously superior, right? The consumer is in control, they don’t have to trust all those people and there’s no danger of a rogue agent sucking all their money out of their account!

Not so fast…

The analysis above neglects one small, but rather important, fact: devices get hacked.

In the pull model, the only devices that can get hacked are those inside the “circle of trust” – your plastic card is pretty impregnable.  And as the utterly disastrous Target breach suggests, consumers were made whole when the disaster happened. It was the big firms who messed up who suffered the consequences.  

Yes… I know this is counterintuitive… you must be asking yourselves: “is this guy seriously arguing that the Target disaster is an argument in favour of the current payment card model?!” Obviously, no…. the episode was clearly a catastrophe and it was really, really bad.   But… it did eventually get sorted out and the roll-out of EMV, tokenisation and better enforcement of PCI-DSS should reduce the risks of something similar in the future.   So I raise this merely as prelude to the push scenario.

Now ask yourself what happens if a device gets hacked in the push scenario.  The obvious question is: which device?   Well… the only device in the circle-of-trust this time is the consumer’s smartphone. Uh-oh.

This is the device from which we’re instructing real-time payments, right? The one that could be riddled with malware?

This might have been merely a theoretical risk…. And then the Brazilian Boleto fraud happened.

RSA have a great write-up of a country-scale real-life example of what can go wrong when push-payment systems get breached… and it’s really scary.

The Brazilian Boleto system is very cool.   At core, it is a way for fund requestors (utility firms, etc) to send a payment request to consumers. The request is known as a Boleto and they can be physical or electronic.


A Brazilian Boleto. Think of it as a mainstream equivalent of a Bitcoin BIP70 Payment Request…

The idea is this: the Boleto has details of the payment request and includes details of how much to pay and to where.   This is in coded text format and a bar code… basically, something that a consumer can take and feed into their banking app: scan the code with your mobile banking app, approve and you’re done. Or you could take it to a bank branch. And if you’re online, you could copy and paste the code into your online banking website and achieve the same end.

Except… the RSA paper shows all the ways it can and has gone wrong.

First, there’s a simple problem of authentication. How do you know the Boleto really did come from who it says it’s from?   The RSA paper documents examples of people receiving Boletos via email that look convincingly genuine but which have the fraudster’s payment account details in place of the firm from which they purport to come.

This is a real problem but it’s nothing new… it’s not really any different to fake websites that masquerade as real ones. We solved it in the pull world with SSL certificates and the like for websites. And the Bitcoin Payment Protocol includes the option to use the same PKI system, for precisely these reasons.

However, the RSA paper also discusses another attack – and this one’s scarier.

This second attack comes in the form of malware that runs in the consumer’s browser.   When it sees a document that looks like a Boleto, it silently changes the details that the consumer sees on their screen: the payment details are changed from the genuine recipient to the attacker. So when the consumer copies and pastes the details into their banking app, it’s the attacker’s account they’re sending the money to.

Variations on this theme are included in the paper but they all amount to the same thing: if the consumer’s device is compromised then it’s game over. And you don’t even need to compromise the whole device or get root-access… you just need to compromise the browser in this scenario.

There are various mitigation mechanisms one can implement (e.g. tying the payment instruction to a signed representation of the payment request and so forth) but the underlying problem remains: if you’re using the consumer device to instruct payments, you have an issue if that device is compromised.

Now, this risk is perhaps over-blown: the risks identified here apply equally to standalone mobile banking apps and we happily run these on mobile devices today, albeit with the belief that their bank will bail them out if something goes wrong. (It’s no surprise that banks are big users of technology like IBM Trusteer).

Similarly, Bitcoin users run their wallets on their devices, in the full knowledge that there is nobody who will bail them out if malware runs amok on the device.  

But I think the two-step dance of an end-to-end push payment request/instruction – where the device is responsible for turning the request into the instruction – is something new that needs deeper study.  So I think the Boleto story tells us is that we need to think very hard about things like:

  • User experience: how is the linkage between Step 1 (receive and authenticate request) and Step 2 (populate and instruct payment) executed and communicated to the user? If step 1 is done by a different app to step 2, what is the hand-off? What security assumptions are being msde?
  • Validation and Reconciliation: what work should (can?) the “network” do to validate that a payment instruction purporting to be in response to a payment request, really is traceable to that request?
  • Malware detection systems: what new behaviours should anti-virus and other technologies be looking out for?
  • Wallet providers: which scenarios are you willing and able to protect your consumers against?

It is possible that this is just a variation on the age-old theme that end-point security is hard – but when things like the Boleto fraud happen, we should use it as an opportunity to look at the other systems being built along similar lines and ask: are there any lessons we can learn and apply?

Welcome to Bitcoin Island

Forget currencies and commodities… perhaps the right analogy for Bitcoin is LAND!

Oleg Andreev posted an insightful tweet the other day:

You could argue this is a trivial observation: how else could it work?!  But thinking in terms of ownership and protocols for transfer of ownership is a surprisingly helpful way to think about how the system works.  And that’s because the “protocol of ownership” insight means there is a whole other world of history, tradition and precedent to learn from: land!

Here are some observations to motivate the thought:

  • In the end-state the quantity of Bitcoin will be fixed, just like land.
  • Bitcoin is not perfectly fungible and neither is land
  • Bitcoin is not “consumed” through use – just transformed and transferred. This is similar to land and dissimilar to many commodities, which are consumed (or at least degraded) through use.

OK – not a perfect parallel but let’s go with it for now…. What happens if we think about Bitcoin through the lens of land?

Well, first, it allows us to think about coins that haven’t been mined yet… we can think of them as parcels of land on “Bitcoin Island” that haven’t been released yet:


The “Land Interpretation” of Bitcoin. Think “Bitcoin Island”

Second, it helps us put some intuition behind the concept of the “unspent transaction output”.  These are Bitcoins that have been sent somewhere but not yet themselves been spent.  So the set of all unspent transaction outputs (UTXOs) can be thought of as the latest state of every Bitcoin that has ever been mined.

The UTXO is absolutely crucial to everything in Bitcoin and yet very few people think in these terms, talking instead about misleading terms like “address balances” and so forth.

But the interesting thing is: if we take a “land interpretation” of Bitcoin, then UTXOs have a really simple explanation: they are plots of land! And Bitcoin transactions are simply actions that merge or split these plots of land.

Imagine I own twenty Bitcoins. My Bitcoin wallet software will show a “balance” of twenty. But it’s likely that this balance actually consists of multiple unspent-transaction outputs. Even if I had bought all twenty Bitcoins in one go, it’s likely that the seller merged several smaller UTXOs that added up in total to twenty Bitcoins.    So perhaps I received three plots of “land”: 7 Bitcoins in one, 7 in another and 6 in the third.  My total “holdings” are 20 – but it is formed from three “UTXOs”.

Perhaps my holdings on Bitcoin Island look like this:


We can think of Unspent Transaction Outputs as plots of land on “Bitcoin Island”. Plots A, B, C represent three unspent transaction outputs controlling 20 Bitcoins

And now it’s possible to teach people about Bitcoin transactions without completely confusing them!

Imagine I wanted to buy a second-hand car for 11 of my Bitcoins.  Let’s also imagine that I pay a transaction fee of 1 BTC to keep things simple (a HUGE over-estimate, of course)

I need to do a few things:

  • Step One: I need to prove ownership of the coins I’m trying to spend
  • Step Two: I need to say how the coins are going to be allocated – how many am I sending and to where? 11 to the seller, 8 back to me and 1 for the miner in this case.
  • Step Three: I need to specify what the new “owners” will need to do to prove they do indeed own the coins. In other words, I need to specify what they will need to do in their Step one when they try to spend their coins in the future.

I do this in Bitcoin by issuing a transaction that accomplishes all three steps in one.  Here’s what it might say:

“I own three unspent transaction outputs: A, B and C. In total they represent twenty Bitcoins. Here is my proof I am entitled to spend A.  Here is my proof I am entitled to spend B.  Here is my proof I am entitled to spend C. I hereby reshape my plot into two new plots: one plot 8 units in size, which I call X and a second plot 11 units in size, which I call Y. Whoever mines this transaction can claim the remaining 1 BTC. If you can satisfy the following conditions then you will be considered to own X: … . If you can satisfy the following conditions then you will be considered to own Y: …”

I will set the conditions so that only the seller of the car could satisfy the Y condition and so that only I could satisfy the X condition (that’s my change and I don’t want anybody else spending it!)

The end result is that I have simply rearranged the land holdings:


Transaction outputs A, B, C are now spent, replaced by two new unspent transaction outputs: X and Y.  X is my change, Y now belongs to the car dealer and F goes to the miner.  


But we can go further… we can now have an informed discussion about what “ownership” means in Bitcoin.  When I “send” Bitcoins to somebody, I’m not assigning ownership to an individual.  What I’m actually doing is laying down a condition – and anybody who can satisfy that condition will be considered the owner.

Now, normally, the condition is very simple.  It says something like:

“To spend this output you must prove you know the public key that hashes to the following address: …   And you must prove you own the corresponding private key by issuing a digital signature”.  

That’s what the “OP_DUP OP_HASH160 …” stuff you sometimes see is usually saying.

But the conditions can be far more complex than that…. It’s all down to how you write your transaction.

Where is this going?

OK – so thinking of Bitcoin in terms of land helps us build some intuition around UTXOs, which we can think of as “parcels of land on Bitcoin Island” and we see that Bitcoin transactions are really just a way to merge or split these parcels of land and impose conditions that allow people to assert ownership.

And now things get really interesting.  Because there are all sorts of interesting phenomena that happen with land transactions that we can use to think about Bitcoin problems.


The land analogy works because Bitcoins are not perfectly fungible. Sure – there are projects trying to overcome this but this feels like an arms race between developers and law-enforcement agencies. To the extent that fungibility remains imperfect, what drivers could force different “land parcels” to have different values?

For me, the biggest topic on the horizon for fungibility is “coin tainting”, “whitelisting” and the other schemes intended to “tag” Bitcoin addresses or UTXOs.

I see these schemes as directly analogous to concepts like land “blight” on the one hand and maybe “planning gain” on the other.  For example, if you own a “plot of Bitcoin land” that has been “whitelisted” by an exchange or finance firm such that you can access their services, presumably your “plot” would be worth more than one that didn’t have that property?

It is perhaps no surprise that the fungibility issue is so hot right now.

Mineral Rights and Colored Coins

Two pieces of seemingly identical land can be worth vastly different sums: if one is sitting on oil and the owner has mineral rights, a purchaser will be willing to pay them more for their land than if it didn’t! Perhaps this is a useful analogy for colored coins: two identical Bitcoins can trade for different prices if one of them has been “colored” by a trusted issuer. What are the taxation implications? What happens when projects trying to add coin coloration to Bitcoin conflict with projects trying to create fungibility?

Alt coins

Perhaps Altcoins are just different islands! If there is a Bitcoin Island, then presumably Litecoin has Litecoin Island and Dogecoin has Dogecoin Island?

This interpretation now helps us think more clearly about the role and value of altcoins.   Perhaps the innate characteristic of a currency (faster confirmation? Use of scrypt?) makes the island a more attractive place to live. But if all the infrastructure and population is on Bitcoin Island then these features may not be enough.  Who knows.

Charges and Liens

It is possible to impose conditions on land parcels in many jurisdictions. A mortgage company can prevent sale of land unless the debt is settled and some landowners in the UK have been dismayed to discover that their land ownership came with an expensive obligation to pay for the upkeep of a local church.

In some cases, the obligation is short-lived (e.g. the mortgage charge) but in others, it persists across transactions (e.g. chancel repair liability).    A question I don’t know the answer to is: can you write a Bitcoin transaction that imposes conditions on a UTXO that propagate?  That is: can you write a transaction such that whoever spends the UTXO must impose the same condition on their transaction output?


Of course, the land analogy is imperfect but I do think there is something to it.  If nothing else, the mental image of “Bitcoin Island” with UTXOs being the plots of land feels like a really useful one… it has certainly helped my understanding…



How I explain Bitcoin and Cryptocurrencies to new audiences

I’ve given several informal talks on Bitcoin and cryptocurrencies recently – mostly to mixed audiences of finance and tech professionals, with varying levels of knowledge about Bitcoin.    I’ve found that the structure below has been working well to so I thought I’d share it in case it helps others explain it to their clients, colleagues or friends.

I don’t claim any of the Bitcoin concepts below are mine… just this approach to explaining them.

Bitcoin’s core idea is audacious

I point out that money is weird: it behaves completely differently when you try to spend it in person to when you try to spend it at a distance.   Here’s what I mean:

If you’re physically with somebody, it’s easy to pay them: you just hand over the cash. You don’t need anybody else’s help and you don’t need anybody’s permission.

But if you’re at a distance, you can’t do the same thing: you always need somebody else’s help…. a bank, a remittance firm, a telco….  Somebody has to be there to transmit the value to the other end.   And they get to set the rules, they get to watch and in some parts of the world, they don’t even exist.


The “audacious announcements”: Nakamoto’s announcement email and Bitcoin whitepaper

And then Satoshi Nakamoto’s white-paper came along and said something utterly audacious.  It said that you could transmit value at a distance with no trusted third party. Five years on, that is still a mind-blowing claim. And it’s all the more mind-blowing because it actually works.   If you take nothing else away from one of my talks, take away the realisation of just how audacious this idea is.  

I also point audiences at this IBM Academy of Technology TechNote, which I co-wrote.  It includes a simple analogy for how Bitcoin works under the covers.  But I don’t talk about the details during the presentation – it simply isn’t necessary.

But forget payments… this is all about a platform

I then say that the amazing thing is that this isn’t even the best bit.  The real story is what you can do with it and what it might mean for the world.  I then make the analogy with the World-Wide Web.

I’m just about old enough to remember the world before the web.  I remember dial-up BBSs, TV “Teletext”, the closed-gardens of AOL and Compuserve.      They were all examples of closed “information-exchange systems” and they were all completely blown away by the web, the world’s first internet-scale open platform for information exchange.

It was owned by nobody, nobody needed permission to build on it and we all know what happened next.

Now look at today’s value exchange networks: the card networks, large bank custody networks, domestic net settlement systems, real-time gross settlement systems, remittance networks.   They look similar:  broadly non-interoperable, closed, proprietary.  Sure: there are good reasons for this – security, financial crime prevention and so on — and those who built them surely deserve to earn a return from running them! But the parallels are eerie.


The big VC firms investing in Bitcoin are all making the same bet:  Bitcoin will be the world’s pre-eminent internet-scale open platform for value exchange


So I invite my listeners to ask themselves:  could Bitcoin and the cryptocurrency revolution be the value-transfer equivalent of the HTTP/HTML/Mosaic information-transfer platform?

If so, it’s not difficult to understand why so many people are so focused on this area.    I also point out that this doesn’t mean the existing value-transfer networks will be replaced – but that it could mean we’re going to see disruption.

What are the implications?

I then share ideas with the audience related to their business…. Perhaps how colored coins could teach us how to build next-generation custody systems.  Or how the openness of the network means non-human systems can become economic actors.  Audiences tend to find Mike Hearn’s self-driving taxis quite amusing and I see eyes open wide when I introduce the idea of the “economy of things”, where a fridge has an internet connection and a Bitcoin wallet


We should expect to see innovation in law, retail, banking, custody, …

So where could this be going?

I then end with a claim…. there are surely only three ways this can go:

Scenario 1:  “It’s all a collective hysteria”

Cryptocurrencies could be a huge collective madness – driven by too many geeks with too much time (and cheap capital) on their hands.  Possible. Plausible.  But it ignores the audacious invention of “peer-to-peer value transfer at a distance”.  I consider this scenario unlikely  (Although, as Ken Tindell points out we should – of course – expect a crash and associated disillusionment at some point)     [Edited 2013-3-27]

Scenario 2: “Banks are doomed; sell everything!”

This scenario invites us to believe that Bitcoin could destroy the banks, replace sovereign currencies as we know them and unleash the era of the feudal geek lords.   Yeah, right.

Scenario 3: Coexistence

So we’re left with only one possibility: we’re going to see co-existence.

This would mean that the winners in the finance sector will probably be the fast (or slow) followers – not the early adopters – and we should regard today as analogous to the web in 1994/1995 when thinking about what infrastructure needs to be built…   Indeed, as I argue in my write-up of a recent panel discussion, we should also remember that the transformational impact of the web wasn’t fully felt for another decade – don’t expect the world to  change overnight.

And that’s why I find this so interesting…

[Updated 2014-05-14]

Bitcoin exchanges are more centralised than traditional exchanges. We can do so much better than this.

What The Traditional Financial System Can Teach Us about The MtGox Disaster

Imagine you were an equity trader and used a Stock Exchange to trade between equities and cash and back.  What would happen if they unexpectedly filed for bankruptcy? How much money would you stand to lose? The answer is zero.    You would lose nothing.  Your equities would be safe at your custodian bank and your cash would be wherever you left it.

However, if you were a Bitcoin trader and your Bitcoin exchange went bankrupt, you could have lost everything – as users of Mt.Gox discovered to their cost last week.

How can this be?  Isn’t Bitcoin supposed to be the ultimate decentralized financial system?  Well, yes… the Bitcoin network is decentralized but many of the major players are not.  And, worse, exchanges like Mt.Gox acted as more than just exchanges: they are also the Bitcoin custodian, clearing house and bank.

The diagram below shows the problem.  From the time a buyer deposits cash or a seller deposits Bitcoins, they are utterly dependent on the solvency of that exchange until they withdraw their funds at some later date.  You have counterparty exposure to the exchange for all this time.


Buyers and Sellers have Counterparty exposure to the exchange for an extended period of time

That’s not how it works in the equity world. I wrote about the mechanics in my article on how equities move around the securities settlement system.

The key point of that article is that your shares and cash never go anywhere near the exchange.

Instead, a custodian bank looks after your equities in a segregated account and they usually also hold your cash.  And a Clearing House will step into the middle of the trade to protect you from non-performance by your counterparty. So you don’t even need to worry about the other party going bust. Things would have to be really bad before you stood to lose any money.

For example, when Lehman Brothers defaulted on US$9 trillion notional of Interest Rate Swap derivatives, LCH.Clearnet (a clearing house) resolved the situation with no loss to anybody else at all:

In September 2008, we successfully managed Lehman Brothers’ US$9 trillion interest rate swap default,  comprising over 66,000 trades, by implementing SwapClear’s unique default management process.   

Less than a week after default, market risk had been reduced by 90% by comprehensive hedging and, within  three weeks, the default was fully resolved well within the margin held and at no loss to other market participants.

The diagram below shows what the risk situation looks like when trading equities: at no time are you exposed to the exchange’s solvency and that your risk exposure is with respect to well-capitalised, hopefully well-run clearing houses and custodian banks rather than the exchange itself.


Users of equity exchanges have no exposure to the exchange

Note that you can choose your custodian and you could, by choosing your exchange, also choose your clearing house….   The equity world is more decentralized than the Bitcoin world!

Can We Do Better?

It’s tempting to conclude that Bitcoin exchanges should move to this model.  But I think we can do better.  Perhaps it is possible to leapfrog a stage of evolution. Here’s what I have in mind:

The Bitcoin “multi-signature” feature allows you to “encumber” funds so that they can only be spent with the agreement of more than one party.   So here’s what you could do if you’re a seller of Bitcoins…

Just before you want to sell some coins through an exchange, send them to a new 2-of-3 address that can be spent by any two of the following three entities:

1)   You

2)   The exchange’s “Clearing House”

3)   An “arbiter” that you and the clearing house both trust

Your coins are now encumbered.  They’re locked up until the trade is done and the outcome agreed.

The “clearing house” would be a new concept in the Bitcoin world but it could be quite simple:  it just needs to be an entity that takes temporary custody of the buyer’s fiat payment and, once received, facilitates the transfer of the Bitcoins, before releasing the fiat payment to the seller.    Note that this still means somebody has to be trusted with the fiat funds but it’s not obvious how you would ever escape from that requirement.  And one could imagine, in the future, that a real Bank might step up to perform this function.

Under this model, when you execute a trade, the exchange informs the clearing house and they then manage the processes necessary to settle the trade.

First, the clearing house can request fiat payment from the buyer (if they don’t already have the funds). When the clearing house confirms to you that they have received payment from the buyer, they sign and submit a 2-of-3 transaction to you for co-signature that will release the encumbered Bitcoins to the buyer.

The clearing house will have populated the buyer’s address and signed their part of the transaction.  So if you agree, you co-sign and publish it into the Bitcoin network.  The recipient gets their Bitcoins, the clearing house waits for confirmation and then releases the cash to you.

At no point does the clearing house or exchange have the ability to steal or lose your coins.  And the 2-of-3 address prevents you from running away with the coins. You all need to co-operate.

In the event of a dispute, you can turn to the third-party “arbiter”, who controls the third key, which they can use to co-sign a transaction with whichever party they decide for.  Notice how there could be competition amongst arbiters – you just need an entity that both you and the clearing house trust.

So provided the arbiter is not in collusion with the clearing house (not a given, of course), we have a way of resolving the transaction even in the event of dispute.

Now this is not perfect and it still has points of centralisation… but it’s a big step forward from where we are today.  And notice how it’s simpler than the equity world: there is no need for a dedicated Bitcoin “custodian” service here – the multi-sig feature allows us to do without.  We just need the clearing houses entities, which could be spun out from the existing exchanges as separate legal entities, and a network of arbiters – the one new function.

From a risk perspective, you end up with the diagram below:


Using Multi-Sig transactions as a step towards lower risk (and greater decentralization) for Bitcoin exchanges

 I don’t claim this as an original idea but I think it does have the virtue of being implementable fairly easily (easy for me to say, of course…)

What are the future Bitcoin battlegrounds?

A throwaway remark by Ken Tindell in a conversation with Marc Andreessen and Joseph Weisenthal caught my eye the other day:

It’s a great question:  assume a cryptocurrency like Bitcoin achieves some level of adoption, what happens next? What products and services will be needed? What institutions should we expect to see? Where are the opportunities?

Sadly, that question is just too hard for me.  But I can try answering a different question: what upcoming battles can we foresee that might shape the future landscape?

Here are some thoughts…

What happens when the Banks remember why they exist?

Never Believe What You Read in the Press: People Do Trust Their Bank. Forget this Lesson at your Peril

When I look at an industry, I ask myself: “If it didn’t exist today, would anybody invent it?” It’s instructive to ask that question about retail banks. If I can hold funds on a prepaid debit card, do payments over SMS, get loans from Wonga and invest with Zopa, why do I need a relationship with a bank?

People usually give one of two answers to that question.

The first explanation is to say that Banks exist to facilitate maturity transformation: most people want to borrow long and lend short – and banks are the institutions that meet that demand by taking the other side of the “trade”.  It’s an inherently unstable business but serves a useful purpose so we allow it to exist. Unfortunately, the FinTech revolution and relentless grind of disintermediation makes it look increasingly anachronistic.  So this explanation doesn’t help.

However, there’s an older, simpler model of banking. This model says they exist to look after our stuff.  We deposit our valuables with the bank and they keep them safe.

When I see services like Barclays “CloudIt”, I wonder if we’re seeing a renewed interest in this business model by the banks. And it’s one that could work: despite what they say most people do still trust their banks to look after their stuff. Interest rates on savings are effectively zero and yet people still leave their money with the banks rather than under their matresses or in the stock market. Ignore what people say; look at what they do.

Now look at today’s Bitcoin world. Take me as an example: my Bitcoin holdings are scant yet I barely trust myself to look after them and I’m supposed to be an expert. How is everybody else supposed to manage?  The default solution to this sort of problem is to do what we’ve always done: outsource the problem to specialists.  Today, that might be Coinbase or elliptic or BitGo but if we apply the logic above, would it really be a surprise if banks realized this is also an opportunity for them?

So we can foresee a showdown: what happens if the Banks realize their association with “safekeeping” gives them brand permission to offer Bitcoin wallet services?  Will they create their own offerings? Partner? Acquire?

Who knows… but the point is this: if you assume a valid model for banking is “safekeeping” then Banks could surprise us all and make a claim for a dominant role in Bitcoin’s future.

And I think some people will find that extremely distasteful…

The Mother of All Forks: A Stake Through the Heart of Privacy?

Imagine the banks follow the logic above and they consider offering Bitcoin safekeeping services.  What happens then?  They quickly realize there’s a problem: Anti-Money Laundering and Know Your Customer rules. What do they mean in this world and how do you comply? To be completely safe, they’d want to track your Bitcoin activity closely.

On one level, that’s easy: if they host your wallet, they can see all your Bitcoin transactions.   But they don’t know who you’re transacting with. And they obviously don’t see anything you do with any other wallets.

But they have a way round that: imagine if they said the following:

“We will provide Bitcoin safekeeping services, facilitate the exchange of Bitcoins for sovereign currencies and provide Bitcoin payment services to you provided you agree to identify upon request the identities of any entities with whom you transact. Failure to do so will result in the immediate termination of your account”.

Now, most current Bitcoin users would never sign up to such a condition.  But current users wouldn’t be the target market; the target market would be the mass market… and that is a lot of people.    Now imagine the regulators get involved and insist that Bitcoin exchanges and payment processors insist on similar conditions.

Suddenly, the banking and regulatory sphere has driven a stake through the heart of Hierarchical Deterministic Wallets, Stealth Addresses, CoinJoin and all the rest.  Sure… you can use all those privacy-protecting technologies…. But you just can’t interact with the exchanges or merchants or anybody else in the “real” world.

We could expect a furious backlash and increased focus on decentralized exchanges and other technologies but it’s not hard to imagine a system that is effectively forked: Bitcoins owned by addresses “inside” the system and Bitcoins owned by addresses “outside”.   It’s interesting to imagine which ‘flavour’ of Bitcoin would be worth more…

Robots With Checking Accounts (Silks, Hit the Road?)

If you read the thinking behind projects such as Ethereum, their ambition is stunning:  they foresee whole classes of interaction that today are governed by law that they think can be mapped into code.  What happens to the legal profession in that world?

I don’t think the lawyers need to worry about their jobs just yet, however….  I observed last year that “on the blockchain, nobody knows you’re a fridge”. But  what happens when this becomes a reality?  Is society ready for devices that can initiate and receive financial transactions with their own accounts, accounts to which no human has access?

What does this mean for the legal system? What does “liability” mean in this world? How do you arrest a fridge?

If my fridge detects a design fault in my washing machine and shorts the stock of the manufacturer, is it committing insider trading?


We shouldn’t assume that the best technical solutions will prevail: on “push” versus “pull” payments

I’ve just posted a new piece on the IBM “Insights on Business” platform discussing “push” and “pull” payments.  I point out how push payments, in general, would remove whole classes of threat from the retail payments landscape but that there are some real problems around adoption that may prove insurmountable.

Who will decide the future of retail payments?

Ultimately, we need to remember that consumers actually like their credit cards and most phone-based solutions today are clunky.  So some recent research from Denmark that studied how real people respond to different options is very enlightening.  It’s obvious that we should spend more time thinking about things from users’ perspectives, but difficult to do in reality!