Qredo Labs Update: Narwhal and Tusk

In the first update from the newly formed Qredo Labs, co-founder Brian Spector reflects on the latest groundbreaking blockchain research and drops some big hints about the development of Qredo V2.

The title alone got your attention didn't it?

You know narwhals are called the Unicorns of the Sea? True. Super cute too. That tusk though...

What has this got to do with Qredo Labs, furiously working on Version 2?

Well, Narwhal and Tusk ¹ is the title of a groundbreaking piece of blockchain research and design that came out a few months ago.

It's not merely theoretical. It exists, and is running on my laptop right now. The brilliant minds behind this ballsy breakthrough also released well-developed Proof of Concept open source code (in Rust) on GitHub. They brought the goods with them.

It's being developed out in the open, and is already attracting meaningful interest from the blockchain community. Why?

From a theoretical perspective, the hypothesis that the researchers put forward is totally at odds with industry consensus. The bomb that these folks have set off is akin to Galileo and his fight with the Church: No, the earth really does revolve around the sun!

The mempool bombshell

The central hypothesis of the research is that the curse of byzantine fault tolerant systems is the consensus layer.

This bit isn't news to anyone, particularly our outstanding engineers and network crew who are fighting an impossibility every damn day like brave soldiers in a foxhole.

These protocols were an attempt to solve (a still yet unsolved) information theory problem in software, driven by the desire to have a competitive chain to Bitcoin. But existing Proof of Stake (PoS) consensus mechanisms are fragile and struggle to scale, with the most successful projects often having the most centralized validator sets.

That's not the biggest problem with these protocols though, although we have all been convinced it was. And, because of that conviction, we accepted ridiculous conventions and settled with subpar performance that would never be acceptable in any other serious field trying to deliver real business value.

This research team rejected that notion and did something brave: They hypothesized — publicly — that the biggest problem plaguing these PoS protocols isn't the design of the consensus mechanism, but the design of the mempool.

As it turns out, the design of the networking flows between nodes chokes the mempool with duplicate packets and unnecessary traffic. Anyone with a network analyzer can see it when you roll up a PoS test network at scale. It's plain as day and as consistent as the sunrise. It's in your face. It's undeniable.

Why didn't anyone figure this out before? Groupthink! If you haven't read 1984 by George Orwell, I'd start there. It's amazing what you can discover when you follow a well-theorized hunch and are brave enough to go against the grain.

Visa-level super-scaling

What's really cool though, is that the researchers not only proved it academically, but they created a Rust lang blockchain using a new, more secure, model that was made possible by ripping out the mempool of a PoS Byzantine Fault Tolerant protocol and replacing it with a high-speed transaction ordering system that was economical on network traffic. Then they let her rip, in a carefully monitored lab designed to create the conditions of a worldwide geo-distributed adversarial deployment. And then they put it on GitHub. Take that, doubters.

So what kind of performance improvement did they get? This team was able to achieve, consistently, 120,000 tx a second. Under the right conditions, they were able to get it up to 600,000 tx per second. 6x faster than Visa's Network. Read that again!

Are you thinking about Visa's announcement that they are building a Layer 2 blockchain for cross chain crypto payments in a different context now? Good, you're going to be able to connect the dots when I start talking about our strategy and product plans.

You know what conditions enabled the researchers to go super-scale?

No, it wasn't by removing steps in the protocol and making it less secure. Those right conditions were horizontally scaling it to hundreds and hundreds of Validator nodes. You know, like the kind of network that CIOs at Citibank are used to buying. More memory, more CPU, better performance, better business results, etc.

These researchers have inverted the current paradigm and shown that the emperor has no clothes. So what comes next?

Beyond the horizon

Qredo CEO Anthony Foy likes to joke that I have this ability to 'see over the horizon' —  a useful superpower if you are the CTO or running product. Truth is, I'm an obsessive compulsive hoarder when it comes to academic research in cryptography. I always have been. It's probably why I got my first job at RSA many moons ago... I digress.

The interesting thing about having this vice is that you start to get good at reading where the market is going by noticing what corporates are funding which talented research groups at the leading universities across the globe, and who is collaborating together. There you have it: The source of my superpower.

One of the hottest research topics of the moment is stateless blockchains. As worked on by projects such as Mina which is offering a new blockchain design that is very mobile friendly, small, fast, and extremely competitive.

A research team at VMWare is also working on stateless blockchains, and they have a great partnership with the crypto research team at Stanford, led by the great Dan Boneh. The approach they are taking, detailed in a paper² released last year, involves innovating over a novel field of mathematics called subvector commitment schemes — a different path than Mina, but it gets the same result.

To find out how this plays into the future of Qredo Network, you'll have to wait for the next instalment from Qredo Labs.

Stay tuned on Twitter to make sure you don't miss it!

— Brian Spector