Infinite Scale

Today, the vast majority of economic activity is conducted via corporations. Corporations were invented in the 1400s, with the first versions manifesting as joint-stock companies. Capital structures, limited liability, and other parameters have evolved since then, but the fundamental premise of the corporation as the conduit to facilitate economic activity remains largely unchanged.

In 2009 Satoshi Nakamoto invented a new way to organize large scale economic activity— without the need for a centralized corporation. In time, historians will look back on the invention of Bitcoin as one of the most important breakthroughs coordinating human behavior at scale.

Bitcoin’s Proof-of-Work (POW) mining makes this possible by encoding an incentive system in software that is enforced by all participants running the Bitcoin software. Millions of independent, economically-motivated actors respond to the incentives codified in the Bitcoin software. There is not a central authority coordinating this activity (unlike Uber, AirBnB, or eBay).

Today, the scale of the POW mining industry—which includes several multi billion dollar companies—is testament to the success and scale of this new model for coordinating economic activity. There is no reason to believe that we’re close to the limits of this model either. In fact, as I’ll argue throughout this essay, we’re on the cusp of applying this model to hundreds of new markets.

Scaling Web2 Infrastructure

Over the last 15 years, the tech industry has developed sets of tools and technologies to enable cloud services to scale to serve billions of people. However, the process for scaling is by no means simple. As systems grow from serving 1,000 to 1,000,000 to 1,000,000,000 users, they are re-architected several times. This is now widely accepted as best practice because engineering a system for 1B users is simply overkill when there are only 1M users using the service.

A number of companies have built platforms that auto-magically scale specific functions from 0 to millions of users of users (without requiring application developers to think about it). One of the first attempts at this was Ning, a company that Marc Andreessen founded in 2005. Ning aimed to make it trivial for developers to build social media applications that could scale to millions of users.

More recently, Amazon has taken this idea of auto-magical scaling to its logical extreme in the form of AWS Lambda. In the Lambda model, developers don’t build and scale applications on servers that they control; instead, they just write function calls that perform specific computations, chron jobs, and database operations. Lambda then automatically scales those function calls on demand (and bills the developer per function call). It’s a radical new model for scaling cloud applications both technically—by reducing complexity for developers—and therefore unlocks a new business model for developers (no need to pay for unutilized server capacity).

But, Lambda is subject to quite a few technical limitations, in addition to being a fundamentally permissioned system (which breaks function composability across developers).

While Lambda is awesome, it doesn’t quite fulfill its stated vision of “write a function once, scale infinitely.” There are a lot of caveats, and a lot of things that Lambda simply cannot do.

The Crypto-Economic Infinite Scale Architecture

The foundation of all crypto networks is the encoding of incentives (and disincentives) into software in order to facilitate large-scale collaboration among distrusting parties without a centralized, coordinating entity. This is what makes these systems trust-minimized.

In abstract, this framing presents a novel way to scale function calls infinitely: design a crypto-economic game to incentivize rational, economical-driven actors to perform function calls for people who are calling those functions, thereby allowing the supply-side of the network to self-organize to fulfill the demand.

POW mining is the first large-scale incarnation of this idea. This can be generalized to all BFT consensus algorithms intended to run in permissionless settings, including Proof-of-Stake systems. But this framework isn’t just useful for adding entries to the end of a ledger in a permissionless setting. It can be generalized to support many types of useful function calls.

The challenge in building these self-organizing systems is designing the crypto-economic game such that the protocol rewards honest actors and punishes malicious and lazy actors.

POW was the first self-organizing system because it’s the simplest economic game to design and implement: POW protocols don’t have to encode any penalties for malicious or lazy actors at all (because POW externalizes these costs outside of the protocol in the form of electricity consumption). However, for other types of more useful functions, verifying both honest behavior (to reward those actors) and dishonest behavior (to punish those actors) is far more complex than just verifying a POW hash.

This paradigm is generally applicable to many off-chain services (e.g., The Graph and Livepeer) and on-chain protocols (e.g., Uniswap, Compound, and Maker) as described in the Web3 Stack. It also applies to novel consensus systems (e.g. Arweave and Helium).

Below I’ll highlight four examples: two off-chain services, and two novel consensus layer implementations. All four of these examples are useful, mostly-fungible, commodity services (as opposed to just adding entries to the end of a ledger).

The Graph is a protocol for indexing and querying data from Ethereum, IPFS, and other Web3 data sources. Assuming that blockchains become critical pieces of infrastructure for the world, consumers and businesses will query these systems trillions of times per day. In the Web2 model, building a system to support this level of scale is extraordinarily difficult.

Traditionally, companies built centralized services and scaled them out the old fashioned way: by hiring lots of programmers, dev-ops teams, and managing a bunch of databases. They’d spend millions of hours architecting and re-architecting the system to optimize performance and cost.

In the Web3 model, The Graph team is building a crypto-economic game that 1) incentivizes independent, rational actors to store and index subsets of a massive dataset, 2) helps users of this service figure out which actors are storing each subsets, 3) ensures that these query providers are returning valid responses (not returning false results), and 4) facilitates payment.

While there are lots of subtleties in designing this game, there are two major classes of problems: 1) What happens if indexers (people who run the queries) are unable to service all the demand, and 2) what happens if indexers return false results?

In the case where demand outstrips supply, market participants - both existing indexers and outsiders - will observe this in real time by monitoring payment flows on the blockchain. Those who have excess resources (or who can easily acquire rent resources from Amazon) will download and run The Graph software, register themselves on The Graph’s smart contract to become discoverable, index in-demand datasets, and begin processing queries for users. This entire cycle will play out over the course of just a few minutes, and can be 100% automated.

Stated more simply: as demand for the query services grows, supply will self-organize to fulfill that demand.

In the case that an indexer returns a false response, they can be heavily penalized. So long as anyone - either the person who requested the query or a 3rd party fisherman - detects that the indexer produced an invalid results, she can report the invalid response to the blockchain, and the blockchain itself will be the final arbiter of truth. Assuming the indexer lied, the blockchain will slash the indexer’s bond (posted in Graph tokens), and reward the person who reported the malicious behavior. Additionally, The Graph protocol leverages some novel zero knowledge proofs to make it even more difficult for indexers to lie.

You can learn more about The Graph’s crypto-economic game here.

The beauty of this model is that, taken to the logical extreme, it will provide a single, uniform, well-documented, globally-accessible, permissionless, censorship-resistant API that can query all public data sets on the internet trillions of times per day.

This is impossible in the Web2 paradigm. This is infinite scale.

Next let’s consider Livepeer.

Livepeer is a protocol that facilitates decentralized video transcoding for live streaming video.

Today, there are only a handful of cloud providers—primarily Amazon, Microsoft, and Google—that can transcode live streaming video at scale. The big cloud providers know that smaller players simply do not have enough data centers around the world with enough processing power to compete for large scale live streaming, allowing the big providers to overcharge for a commodity service.

Livepeer allows anyone with a video encoding/decoding application specific integrated circuited (ASIC) to transcode live-streaming video, and get paid for doing so. Modern graphics cards contain many different kinds of chips, including highly-parallel processors, and dedicated video encode/decode ASICs. Therefore, all computers can offer transcoding services to the Livepeer network.

There are many large mining operations—each with thousands of graphics cards—and these facilities are geographically dispersed all over the world. However, mining farms are only utilizing the general-purpose processors on these graphics cards; they’re not utilizing the dedicated video encode/decode ASICs.

The Livepeer protocol allows GPU miners—who are already in the business of maximizing resource utilization, and who have strong internet connections—to increase the utilization of their existing resources. Miners can run both POW hashing and Livepeer transcoding on a single graphics card without any performance degradation.

The Livepeer protocol is a smart contract that facilitates this economic transaction. At a high level, it does a few things: 1) provides a way for transcoders to signal that they have excess capacity, 2) allows streamers to discover transcoders, 3) enables streamers to verify that transcoders are transcoding correctly via sampling (not just sending black pixels), and 4) supports micropayments.

Again, the beauty of this model is that it scales infinitely. As more streamers choose to transcode using the Livepeer network, economically-motivated actors with excess capacity will notice. They will download and install the Livepeer software, register themselves with the Livepeer smart contracts, and begin transcoding videos. You can learn more about Livepeer’s crypto-economic game here.

This is infinite scale.

Next, let’s consider Arweave.

The Arweave protocol defines a crypto-economic game to incentivize actors to store data forever. Unlike The Graph and Livepeer—who define the rules of the crypto-economic game in a smart contract—Arweave’s crypto-economic game is enforced via consensus rules.

In addition to hash-based POW, Arweave consensus requires miners to demonstrate Proof of Access (POA) to an already existing, randomly selected block in Arweave’s blockweave data structure. By adding this additional constraint to produce new blocks, Arweave miners must allocate some of their scarce resources away from hashing, and towards permanent file storage. You can learn more about Arweave’s crypto-economic game here.

If outside observers notice that block times are increasing without a corresponding decline in the price of Arweave tokens, that means that existing miners are not adequately serving users. Outsiders can download the Arweave software, download the blockweave, and begin generating revenue.

These self-organizing systems also work in reverse: if demand for a service decreases over time, eventually some of the service providers will operate at a loss or stop providing the service. This will in turn increase the profitability of the remaining service providers. This dynamic is already evident in Bitcoin mining.

Last, let’s consider Helium. Helium is particularly interesting because of the consumer hardware and local community component.

Helium represents a radically new business model for deploying and managing wireless networks. Traditionally, telecom companies like ATT, Verizon, Vodafone, Orange, China Mobile, China Unicom, SK Telecom, NTT Docomo, and Softbank centrally plan, manage, and deploy wireless networks. They rent land for towers, build towers, pay for labor, run back haul, and buy all the equipment up front before generating any revenue. Building out a wireless network is extraordinarily capital intensive and slow.

V1 of the Helium network isn’t going after 3G / 4G / 5G that are in phones. Instead, v1 of the Helium network serves low-power IOT devices that require long-range wireless and battery life measured in months and years not hours or days. Examples include environmental sensors, scooters, bicycles, dog collars, and more.

In the Helium model, consumers buy a Helium hotspot, hook it up to their Wifi or Ethernet at their home or small business, and provide wireless coverage to nearby IOT devices that leverage the network. Those devices then pay the hotspot owners per byte of data.

Today, there are more than 3,000 hotspots deployed around the US, and the 10 largest metro areas in the country are now thoroughly covered. While there is not yet any meaningful demand flowing over the network (that feature will be enabled soon), this is remarkable. Thousands of people all over the country have each invested hundreds of dollars to help build out the network on a speculative basis, and a robust network has emerged in only a manner of months.

The Helium network is the fastest wireless network roll out in human history. This is only possible because of the infinite scale model.

One of the most obvious applications for this technology is location-aware dog collars. Today, these dog collars are inferior devices because they’re built on energy-intensive, 3G networks, and battery life is only measured in days. Who wants to 1) charge their dog’s collar every other day, and 2) pay for a dedicated data plan from Verizon that utilizes just a few megabytes of data each month?

As the Helium network matures and becomes ubiquitous , we expect dog lovers—who want to ensure they have adequate coverage for miles around their home—to purchase both dog collars and Helium hotspots. Not only will they use their own hotspots, they’ll also generate income from other nearby dog owners and users. You can learn more about Helium’s crypto-economic game here.

Like the other networks highlighted above, economically motivated actors will self-organize to scale the network. For example, scooter companies are likely to work with juicers (the people who pick up and charge scooters) to maximize wireless coverage. The beauty of these open systems are that anyone can observe the supply-demand mismatch, and rectify on a localized basis.

Over the last six months, enthusiastic groups of super-users have purchased dozens of hotspots at a time, and deployed them all over their local metro areas. For example, Brad Feld of The Foundry Group is organizing a public effort to deploy the Helium network in his hometown of Boulder, Colorado. The Helium team never expected this, and has since formalized a patron program for those who want to deploy dozens of hotspots.

The Helium network is particularly interesting because of 1) how many people are participating on the supply side of the network, 2) the geographic distribution of those people, 3) the fact that these people are investing in dedicated hardware (not just renting servers from Amazon), and 4) that they’re investing in the success of the network before demand is understood. Ultimately, as the Helium network matures, we expect demand to drive supply. However, it’s been incredible to see the supply side of this network self-organize over the last six months.

Challenges

This new model for organizing economic activity is not without its challenges. Perhaps the most significant limitation is that this model is not fully generalizable to support arbitrary computations over persistent data over long periods of time (e.g. running a web server indefinitely)—at least not yet.

There are two major challenges therein: 1) the data persistence problem: if a computer is running a computation on some piece of data and then just stops responding, how does another computer get the data for the last known state to continue to the computation, and 2) the verification problem: how does the person who requested the computation know that the computer tasked with running the computation did so correctly.

I’m optimistic that both of these problems will be solved with advances cryptography. The data persistence problem is addressable with more efficient accumulators, and the verification problem is addressable with more efficient proofs of computational integrity (which are an integral part of zero knowledge proofs).

In the interim, protocols such as Livepeer, Arweave, and Helium are dealing with the verification problem via clever forms of sampling in their respective verification games. In the case of Livepeer, it’s almost trivial for a verifier to sample video frames (e.g., 1 of every 1,000 frames). In the case of Arweave, Proof of Access is by definition sampling (I expect over time, as the blockweave grows larger, that the protocol will increase the sampling rate). In the case of Helium, Proof-of-Coverage samples hotspots.

While these sampling-based verification games don’t offer cryptographic guarantees around correctness, they still codify incentives for honest behavior, and disincentives for dishonest behavior, and so should work at scale (also, sampling-based verification is common in all kinds of other systems such as writing to CDs).

The Road Ahead

We are on the cusp of an explosion in self-organizing systems. In 2020, dozens of these new systems are going to launch at every layer of the Web3 Stack.

As Marc Andreessen wrote in 2007, the biggest internet platforms will abstract the complexities of scaling complex systems. Web3 represents a step-function improvement in the development of internet platforms on two fronts: by making it easier than ever to scale infinitely by encoding incentives in software, and by enabling permissionless, trust-minimized, composable, interoperable, censorship-resistant access to these services for billions of people.

If you’re building any self-organizing systems to coordinate economic activity, whether as an off-chain service or on-chain protocol, please reach out. We’d love to learn more about what you’re building.

Thanks to Yaniv Tal, Brandon Ramirez, Doug Petkanics, Sam Williams, Amir Haleem, Alex Pruden, and Ali Yahya for their feedback on this post.