Foundations of Distributed Consensus and Blockchains

Comprehensive Textbook on Distributed Systems and Blockchain Technologies

Elaine Shi | Cornell University | Preliminary Draft, Summer 2020

Book Overview

"Foundations of Distributed Consensus and Blockchains" is a comprehensive textbook by Elaine Shi of Cornell University, providing a deep dive into the mathematical foundations of distributed consensus and blockchain technologies. The book covers both classical distributed consensus protocols and modern blockchain systems like Bitcoin and proof-of-stake protocols.

Key Insight: Distributed consensus forms the backbone of modern distributed systems and cryptocurrencies. This textbook bridges classical distributed systems theory with modern blockchain technologies, providing both theoretical foundations and practical implementations.

Key Concepts

Byzantine
Fault Tolerance Model
Dolev-Strong
Consensus Protocol
FLP
Impossibility Result
Streamlet
Blockchain Protocol

Core Topics Covered

Byzantine Broadcast and the Dolev-Strong Protocol

Explores the fundamental Byzantine Generals' Problem and presents the Dolev-Strong protocol, a foundational solution for distributed consensus with digital signatures.

Consensus Without Digital Signatures

Examines the limitations and possibilities of achieving Byzantine Broadcast without relying on digital signatures, including important impossibility results.

Blockchain and State Machine Replication

Connects classical state machine replication with modern blockchain systems, providing a unified framework for understanding distributed ledgers.

Streamlet: A Simple Blockchain Protocol

Presents Streamlet, an elegant and simple blockchain protocol that provides strong consistency guarantees and serves as an excellent pedagogical tool.

Partial Synchrony and Lower Bounds

Analyzes consensus under partial synchrony and establishes important lower bounds for what is achievable in various network models.

Asynchronous Consensus and FLP Impossibility

Discusses the famous FLP impossibility result for deterministic asynchronous consensus and presents randomized approaches to overcome this limitation.

Content Overview

Chapter 1: Distributed Consensus: from Aircraft Control to Cryptocurrencies

This chapter traces the history of distributed consensus from its origins in mission-critical systems like aircraft control to modern applications in cryptocurrencies. It introduces the Byzantine Generals problem and explains how distributed consensus has become a fundamental building block for reliable distributed systems.

Chapter 2: Preliminaries

Provides the necessary cryptographic foundations for understanding consensus protocols, including negligible functions, collision-resistant hash functions, random oracles, digital signatures, and probability bounds.

Chapter 3: Byzantine Broadcast and the Dolev-Strong Protocol

Formalizes the Byzantine Broadcast problem and presents the celebrated Dolev-Strong protocol, which solves Byzantine Broadcast using digital signatures in f+1 rounds. The chapter includes detailed analysis and connections to the "muddy children" puzzle.

Chapter 4: Byzantine Broadcast without Digital Signatures (Lower Bound)

Explores the limitations of achieving Byzantine Broadcast without digital signatures, proving the important impossibility result that BB is impossible when at least n/3 nodes are corrupt without setup assumptions.

Chapter 5: Byzantine Broadcast without Digital Signatures (Upper Bound)

Shows that the 1/3 lower bound is tight by presenting a protocol that achieves Byzantine Broadcast without digital signatures when strictly fewer than n/3 nodes are corrupt.

Chapter 6: Blockchain and State Machine Replication

Connects the classical concept of state machine replication with modern blockchains, defining the blockchain abstraction and its security properties (consistency and liveness). Shows how to construct blockchains from Byzantine Broadcast protocols.

Chapter 7: A Simple Blockchain Protocol — Streamlet

Introduces Streamlet, an extremely simple yet powerful blockchain protocol that defends against f < n/3 corruptions. The protocol follows a natural propose-vote paradigm with an elegant finalization rule.

Chapter 8: Lower Bound for Partial Synchrony

Proves that no partially synchronous consensus protocol can tolerate n/3 or more corruptions, establishing an important fundamental limit for what is achievable in partially synchronous networks.

Chapter 9: Round Complexity of Deterministic Consensus

Establishes that any deterministic Byzantine Broadcast protocol must incur at least f+1 rounds, showing that the Dolev-Strong protocol achieves optimal round complexity for deterministic protocols.

Chapter 10: Round Complexity of Randomized Consensus

Explores how randomization can help circumvent the f+1 round lower bound for deterministic protocols, presenting lower bounds and recent results for randomized consensus protocols.

Chapter 11: Communication Complexity of Consensus

Analyzes the communication requirements of consensus protocols, proving quadratic lower bounds for deterministic protocols and presenting communication-efficient randomized approaches using committee election.

Chapter 12: Asynchronous Consensus: The FLP Impossibility

Presents the famous FLP impossibility result, which shows that deterministic asynchronous consensus is impossible even with just one crash fault. The chapter includes a detailed proof of this fundamental result.

Chapter 13: A Randomized Asynchronous Consensus Protocol

Shows how randomization can overcome the FLP impossibility by presenting a randomized asynchronous consensus protocol that works in the presence of f < n/3 corruptions, assuming a common coin oracle.

Chapter 14: Bitcoin and Nakamoto's Blockchain Protocol

Analyzes Bitcoin's consensus protocol, explaining Nakamoto's ingenious idea of proof-of-work and providing a formal description and analysis of Bitcoin's blockchain properties.

Chapter 15: The Selfish Mining Attack and Incentive Compatibility

Discusses the selfish mining attack on Bitcoin and presents Fruitchain, an incentive-compatible blockchain protocol that mitigates such attacks.

Chapter 16: A Simple, Deterministic Longest-Chain-Style Protocol

Presents a deterministic longest-chain-style consensus protocol and analyzes its properties, providing insights into the design space of blockchain protocols.

Chapter 17: Analysis of Nakamoto's Blockchain

Provides a detailed analysis of Nakamoto's blockchain protocol, including chain growth, chain quality, consistency, and liveness properties.

Chapter 18: Proof of Stake (Brief Overview)

Offers a brief overview of proof-of-stake consensus mechanisms, contrasting them with proof-of-work approaches and discussing their potential advantages and challenges.

Note: The above is only a summary of the book content. The complete document contains extensive mathematical proofs, detailed protocol descriptions, and comprehensive analysis. We recommend downloading the full PDF for in-depth study.