The Web3 Stack
By Kyle SamaniJuly 10, 2018 | 10 Minute Read
I have yet to see an actual illustration of the Web3 stack. So this is an attempt. Given how expansive the Web3 ecosystem is, this diagram is almost certainly incomplete, and likely has a few inaccuracies.
For the remainder of this post, I’ll highlight some of the most interesting observations that I drew from this diagram.
The Core Stack
What does the core development stack actually need to provide for dapp developers?
It must provide a single canonical ledger of ordered, valid transactions.
The P2P, consensus, and state transition machine layers collectively provide these functions. Both Ethereum and Bitcoin today perform only those functions, though the core Ethereum protocol will eventually expand to include sharding.
Relative to the number of teams building the optional components that are on the right side of the diagram, there are a small number of teams building out the components on the left side of the diagram. Including optional components and excluding the Internet Protocol, from the bottom of the stack, moving up:
- Oasis Labs is building out Ekiden, a neutral platform that will allow many chains to support private, off-chain, TEE-based computation.
- Handshake is building decentralized DNS. Gaining adoption will be very difficult given OS-defaults for DNS.
- Monero is building Kovri to enable privacy-preserving packet routing between nodes that will guarantee IP-level privacy. Kovri is being built to support many chains.
- BloxRoute is working on a chain-agnostic block delivery network.
- The Ethereum Foundation built DevP2P and Protocol Labs built LibP2P. Most new chains are adopting one of those two frameworks, and there are even rumblings about Ethereum moving to LibP2P.
- Ethereum and Polkadot are working on full-state sharding.
- There are over a dozen teams experimenting at the consensus layer:
- Leader-based blockchain consensus
- Bitcoin and Bitcoin Cash – ASIC-optimized proof of work (POW)
- Ethereum 1.0, Monero, Zcash, others – ASIC-resistant POW
- Kadena – braided POW
- Chia – proof of space and time (POST) and proof of elapsed time (POET)
- Filecoin – POST with useful data
- Ethereum 2.0 – Casper TFG proof of stake (POS)
- Thunder – POS with POW fallback
- Decred – hybrid POW/POS
- Polkadot – Honeybadger POS
- EOS – delegated proof of stake (DPOS)
- Tezos – another DPOS variant
- Tendermint – another DPOS variant
- Solana – proof of history (POH)
- Dfinity – threshold relay + probabilistic slot consensus
- Algorand – byzantine agreement with leader election (BA⋆) Leaderless blockchain consensus
- Ripple Consensus Protocol
- Stellar Consensus Protocol
- Avalanche Block DAGs
- Byteball – Byteball Main Chain Consensus
- Hashgraph – Hashgrah Consensus Protocol
- DAGlabs – Spectre
- Blink – Blink Consensus Protocol
Spacemesh – POST for committee selection, then tortoise + hare
There are a few major state transition machines: - Ethereum Virtual Machine (EVM) – Ethereum 1.0, Ethermint, Hashgraph, WANchain, others - Web Assembly Virtual Machine (WASM) – Dfinity, EOS, Polkadot, Ethereum 2.0 - Direct LLVM exposure – Cardano, Solana
This listing begs the question: Relative to the other layers of the core stack, why are there so many teams working on the consensus layer?
The obvious answer is because that’s where the money is.
This naturally leads to the question: Why does all the value accrue at the consensus layer?
The consensus layer is the single greatest bottleneck in blockchains, and consensus schemes are bounded by fundamental tradeoffs. Moreover, consensus schemes are mutually exclusive: A single chain cannot employ two consensus schemes simultaneously. The world’s leading protocol designers recognize the limits of Bitcoin’s POW, and see the opportunity to build a massive, global digital reserve asset, and as such are rationally competing in a market that will be worth tens of trillions.
To date, it can be reasonably argued that no one has built a consensus algorithm that delivers a fundamental Pareto improvement given the dynamics of the scalability trilemma. Most of the teams innovating at the consensus layer are all making the fundamental bet that they have chosen the best set of trade-offs for some subset of use cases.
If a team did come up with a mechanism that represents a fundamental Pareto improvement, how would we know this? The simple answer would be to quantify throughput for a given level of decentralization of block production. The Dfinity, Algorand, Solana, and other teams believe this possible, though Vitalik does not.
There’s a reasonable argument to be made that consensus algorithms can be copied. Although this is technically manageable (see Cosmos with the EVM on Tendermint), it may not be politically achievable to change consensus algorithms on existing chains, especially in systems with tightly coupled, on-chain governance. The more tightly coupled the on-chain governance, the harder it will be to convince entrenched interests to change (try convincing one of 21 EOS block producers to forfeit one of those coveted spots for a theoretical system that’s more decentralized and provides equivalent performance). This is exactly why Ethereum can ditch POW to move to POS, whereas native POS chains are unlikely to be willing to make a change of similar magnitude.
The other layer of the core stack where we’re seeing a fair bit experimentation is the state transition machine layer. The teams that are not building on WASM tend to have very pointed views about what a blockchain state transition machine should and should not do:
- Kadena believes that smart contracts must be human readable.
- Tezos believes all smart contracts must be formally verified.
- Rchain believes that smart contracts should run concurrently across dapp chains and be formally verified.
- Coda believes everything must run through SNARKs in order to guarantee that even tiny nodes can verify the integrity of chains, thereby maximizing the probability of maintaining maximum decentralization in perpetuity.
The most interesting observations at the state transition machine layer are: 1) the fact that many major projects have come to consensus to use WASM (despite the fact that WASM was never intended to be used by blockchains); and 2) the degree of idiosyncrasy in the bets that Kadena, Tezos, Rchain, and Coda are making.
Because everything in crypto is open source, network effects could be considered relative to each layer of the stack. For example, the EVM has achieved a critical mass, and now has a network effect due to the tooling, developer education, libraries, etc. built for the EVM. This is exactly why so many other projects – Hashgraph, Cosmos Ethermint, Wanchain, RSK, Blink, and others – have decided to adopt the EVM, despite the fact that Ethereum itself wants to move off of the EVM!
In practice, I’m a bit skeptical of the view that state transition machines are portable across chains. In the case of the of the EVM, not even the Ethereum Foundation wants to keep it around. My sense is that developers don’t generally want to code in solidity, but rather they are only doing so because they are Ethereum ideologues with regards to decentralization of block production and open source software development. If my hypothesis is correct, non-Ethereum chains that adopt the EVM will not benefit from the “network effect of the EVM.”
However, things could be different in the case of WASM. Because most major crypto teams want to stand on the shoulders of giants that are building and maintaining WASM, there may in fact be a portable, cross-chain network effect that builds around WASM as the state transition machine for blockchains.
If this network effect around WASM manifests, it will be difficult for teams who are developing unique state transition machines to compete in the long run.
The Extended Core Stack
There are a few elements that are not and should not be part of the base-layer chain, and are not yet considered essential to dapp development, but that I believe will become a core component to the development stack. Moving up the stack:
- Many teams are building sidechains. The most notable implementations in Bitcoin are drivechains and Liquid. In the Ethereum universe, the most important implementations are SKALE within the plasma framework, and Cosmos Ethermint chains as sovereign dapp chains.
- There are a number of teams working on payment channel and state channel network networks in Bitcoin, most notably Lightning Labs and Blockstream. In the Ethereum ecosystem, they are Raiden and Celer. Many, especially those within the Bitcoin community, consider this approach as the only way to scale.
- The Interledger Protocol (ILP) was finalized a few months ago. Quite a few teams are working with it to enable cross-chain interoperability. Most developers and investors, including myself until recently, haven’t been paying enough attention to ILP. However, it’s entirely plausible that ILP becomes the most important layer in the Web3 stack (analogous to this) as value accrues to secure chains like BTC, while commerce happens on more functional chains like Ethereum.
- To the best of my knowledge, The Graph is the only team building a decentralized query layer for Ethereum. Previously, every single team building dapps on Ethereum had to build a proprietary indexing infrastructure.
- A number of teams such as BigchainDB, OrbitDB, and Bluezelle are building immutable structured databases (bottom right of the Web3 diagram) as permissionless, free standing chains. Given the performance boost gains that can come from using structured databases, It will be interesting to see if developers choose to use these systems natively, or if teams like SKALE implement these open source systems as plasma chains.
In general, the extended core stack is much less mature than the core stack. Basically none of these components are in production at any meaningful scale. As such, dapp developers have not been able to use these components for now.
As the extended core stack matures, I expect the rate of dapp development to accelerate substantially. Dapp developers shouldn’t have to think about any of the issues that these components aim to solve, but given the current state of the stack, they’re forced to build their own components of the stack, which is inefficient at best.
Just as there are cloud APIs for everything from sending emails (Sendgrid) to SMS (Twilio) to mapping (Google Maps), there will be a massive range of “decentralized libraries” in the form of the optional decentralized components. Each of these will perform a unique set of functions that dapp developers can choose to incorporate.
Only a few of these optional components services are live on mainnet. Some examples include: Livepeer, 0x, Kyber, Storj, Sia, Oraclize, and Civic. But most of the teams building these components have not yet released production-ready tools.
This at least partially explains the lack of dapps: It’s hard to build useful dapps without usable libraries. Without usable libraries, every dapp has to re-invent the proverbial wheel.
It’s also interesting to recognize that a substantial majority of these components are being built for the Ethereum ecosystem. Some teams, for example, Keep and Truebit, have publicly discussed supporting Dfinity as cross-chain services. But the overwhelming majority of infrastructure in the crypto ecosystem today is being built to support the EVM at a minimum, and the Ethereum ecosystem more broadly.
Over the next year, as EOS, Tezos, Kadena, Dfinity, Solana, Tari, Hashgraph, and others launch and mature their chains, they’re going to compete for the attention of the teams building dapp infrastructure components. The tooling required to seamlessly support cross-chain development certainly isn’t in place today, so the teams building the core chains are going to need to compete to get these dapp infrastructure providers to support their respective chains.
As noted above, the network effects of a state transition machine like WASM are likely to be largely portable across chains, it’s unlikely that they’re 100% portable without any developer effort. It will be interesting to see if base-layer chain teams can make the process of porting infrastructure across chains seamless to attract the first wave of infrastructure developers who are building for Ethereum.
The Top Of The Stack
In most stack diagrams, the app is the top of the stack. But basically all crypto development is back-end focused, not-front end. And so there are a few elements that live above the dapp in the Web3 stack.
Interestingly, almost no one in crypto talks about decentralizing application hosting. This is particularly curious because this layer is centralized in basically every dapp today. It’s centralized either because the web server hosting the app is centralized, or because there’s a single download link to download the app and install it on the client device, pushing the ecosystem back towards the walled-garden model, which is by definition centralized. In an ideal world, the application host would be both decentralized and delivered just in time, a la modern web applications.
Why are there basically no solutions aiming for decentralized application hosting? I can identify two reasons: either it doesn’t matter that this layer is decentralized, or it’s just such a hard problem that no one has even bothered to tackle it. I suspect the answer is a little bit of both.
In any dapp, so long as the database and asset storage is sufficiently decentralized, it doesn’t really matter if the application host is centralized. If a government censors a dapp by shutting down the application host, the dapp author can just open source the front end code, and let someone else spin up a new host that provides a portal to the same back end. This is likely to play out like torrent trackers in the 2000s: shut one down, 5 more pop up somewhere else.
But what if you could decentralize an application host? Codius, an open source project started by Ripple, attempted to do this before Ripple stopped allocating resources to it in 2015. Recently, Stefan Thomas, former CTO of Ripple, left Ripple to start Coil, which is picking up where Codius left off. It’ll be interesting to see how this layer of the stack works in practice, and the implications of integrating into the existing DNS, which assumes centralized, trusted application hosts.
Layer 2 Scaling Solutions
I’ve already touched on these layers in this essay, but I think it’s worth revisiting them strictly within the context of scaling blockchains.
First is sharding. Based on setbacks that both the Ethereum and Polkadot teams have faced, sharding appears to be the most technically challenging scaling solution to implement. Even if it works, it’s not clear that it will be the holy grail that many, myself included, hope it will be. The single greatest problem with sharding is cross-shard latency, which is likely to be on the order of minutes in Ethereum. This may severely limit the efficacy of sharding in practical settings. Moreover, sharding creates all kinds of downstream problems, e.g. clients may not know which shard(s) to read from depending on a user query.
The layer 2 scaling solutions – side chains, payment and state channel networks, and ILP – suffer from the same general problems. As side chains proliferate, things are going to get confusing as users forget which chains their assets are in. Payment and state channel networks have substantial latency challenges, and create all kinds of new problems related to liquidity routing, money transmission, and privacy. And ILP will be subject to real latency challenges given that the store of value chain, Bitcoin, has 10 minute block times.
To make this all even more confusing, imagine a user has funds in a payment channel that exists in one shard and that she wants to move those assets into a side chain on a different shard.
Things are going to get a lot more confusing before they become clear.
Perhaps the most fascinating part of the Web3 stack is just how decentralized the stack itself is. Most of the traditional application development stacks – Windows, iOS, Android – are almost entirely centralized, with only a handful of 3rd party developer libraries and services that have achieved critical mass. This stands in stark contrast to the Web3 stack, which is being built by hundreds of teams all over the world simultaneously! Although theoretically one doesn’t need anything beyond what’s provided by the core Ethereum protocol to build dapps, in practice dapp developers need all kinds of tooling that the Ethereum Foundation has not and will not build. This is the ultimate testament that crypto is the pinnacle of the bazaar.
Considering how much of the Web3 stack is still under development, it’s no wonder that dapp usage is abysmal: it’s practically impossible to build usable dapps given the state of the Web3 stack today! Like many other technologies, the Web3 stack will progress slowly, and then quickly after surpassing some tipping point.
The dapp revolution will happen shortly after the Web3 stack achieves some level of usability, stability, and feature-completeness. I suspect this is 2-3 years out.
Multicoin has invested in over a dozen protocols across many layers of the Web3 stack. And we continue to evaluate projects who are building out Web3 infrastructure. If you’re building out part of the Web3 stack, please reach out! We’d love to learn more.
I wrote this post looking for feedback. I’m sure that the Web3 diagram I depicted is incomplete. If you have any ideas, comments, or feedback, please email me at firstname.lastname@example.org.