Database Replication

Metadata

1. Overview

What problem does it solve?

A single database server is a Single Point of Failure (SPOF). If the hard drive dies or the network drops, your entire application goes down. Furthermore, a single server has a hard limit on how many reads it can process per second.

Database Replication is the process of keeping a copy of the exact same data on multiple servers.

What breaks without it?

• Availability: Your system goes offline whenever the single database goes offline for maintenance, network issues, or hardware failures.
• Data Durability: If the single database disk is corrupted, you lose all user data.
• Read Scalability: Once your traffic exceeds the CPU/Disk capacity of one database, your application becomes incredibly slow.

2. Motivation

In the early days of the web, scaling a database meant buying a bigger server (Vertical Scaling). Eventually, websites became too large for even the most expensive mainframes.

Engineers realized that for most applications, reads heavily outnumber writes (e.g., you read 100 tweets for every 1 tweet you write). The solution was to have one server handle writes (the Master) and duplicate the data to several cheaper servers (the Slaves/Replicas) to handle the reads.

3. Real-World Usage

4. Prerequisites

5. Visual Explanation

Master-Slave (Active-Passive) Architecture

%%{init: {'theme': 'base', 'themeVariables': { 'primaryColor': '#ffcc00', 'edgeLabelBackground':'#ffffff'}}}%%
graph TD
    classDef client fill:#f9f9f9,stroke:#333,stroke-width:2px;
    classDef master fill:#ffb3b3,stroke:#cc0000,stroke-width:2px,color:#660000;
    classDef replica fill:#d4edda,stroke:#28a745,stroke-width:2px,color:#155724;

    C1((Client A)):::client
    C2((Client B)):::client
    
    M[(Master DB<br/>Writes + Reads)]:::master
    R1[(Replica 1<br/>Read-Only)]:::replica
    R2[(Replica 2<br/>Read-Only)]:::replica

    C1 -- "1. WRITE (INSERT / UPDATE)" --> M
    M -. "2. Replicate (Binlog)" .-> R1
    M -. "2. Replicate (Binlog)" .-> R2
    
    C2 -- "3. READ (SELECT)" --> R1
    C2 -- "3. READ (SELECT)" --> R2

Workflow:

1. Clients send all writes (INSERT/UPDATE/DELETE) to the Master.
2. The Master records the change in its log (e.g., MySQL's binlog or Postgres's WAL).
3. The Master streams this log to all Replicas.
4. The Replicas apply the log to their own disks to stay in sync.
5. Clients route their read queries (SELECT) to any of the Replicas, taking the load off the Master.

6. Internal Working

Sync vs Async Replication

Synchronous Replication The Master waits for the Replicas to confirm they have saved the data before telling the Client "Success".

• Pros: Zero data loss if the Master crashes. Strong consistency.
• Cons: Slow. If a Replica is offline, the Master cannot accept writes.

Asynchronous Replication The Master tells the Client "Success" immediately after saving locally, and streams the data to Replicas in the background.

• Pros: Extremely fast writes. Tolerates Replica failures.
• Cons: Replication Lag. If a client reads from a replica immediately after writing, they might not see their own data. If the Master dies before replicating, data is permanently lost.

Semi-Synchronous Replication (The Sweet Spot) The Master waits for at least one Replica to confirm before telling the Client "Success".

Multi-Master vs Leaderless

• Multi-Master (Active-Active): Multiple nodes can accept writes. Requires complex conflict resolution (e.g., two people booking the same seat on different masters).
• Leaderless: Any node can accept writes, but the client must write to a quorum of nodes (W+R > N) to ensure consistency. Used by Cassandra and DynamoDB.

7. Implementation

Why Go? Go's channels and goroutines make it exceptionally easy to model network streaming and concurrent replication between a master and multiple slaves within a single process.

/*
019 - Database Replication Simulation
Simulates an Asynchronous Master-Slave replication setup.
The Master receives writes and asynchronously streams the changes
via channels to the Replicas.
*/
package main

import (
	"fmt"
	"sync"
	"time"
)

// Record represents a database row change
type Record struct {
	ID    int
	Value string
}

// Database represents a single DB node
type Database struct {
	Name     string
	Data     map[int]string
	IsMaster bool
	mu       sync.RWMutex
	// Channels for the Master to push WAL (Write-Ahead Log) entries to replicas
	ReplicaStreams []chan Record
}

func NewDatabase(name string, isMaster bool) *Database {
	return &Database{
		Name:     name,
		Data:     make(map[int]string),
		IsMaster: isMaster,
	}
}

// Write only allowed on Master
func (db *Database) Write(id int, value string) error {
	if !db.IsMaster {
		return fmt.Errorf("node %s is a replica and cannot accept writes", db.Name)
	}

	db.mu.Lock()
	db.Data[id] = value
	fmt.Printf("[%s] WRITTEN: id=%d, val=%s\n", db.Name, id, value)
	db.mu.Unlock()

	// Asynchronously replicate to all connected slaves
	record := Record{ID: id, Value: value}
	for _, stream := range db.ReplicaStreams {
		go func(ch chan Record) {
			// Simulate network latency
			time.Sleep(50 * time.Millisecond)
			ch <- record
		}(stream)
	}

	// Master returns success immediately (Asynchronous Replication)
	return nil
}

// Read allowed on any node
func (db *Database) Read(id int) string {
	db.mu.RLock()
	defer db.mu.RUnlock()
	val, exists := db.Data[id]
	if !exists {
		return "<not found>"
	}
	return val
}

// AttachReplica connects a slave to the master's replication stream
func (master *Database) AttachReplica(replica *Database) {
	if !master.IsMaster {
		panic("Cannot attach replica to another replica")
	}
	stream := make(chan Record, 10)
	master.ReplicaStreams = append(master.ReplicaStreams, stream)

	// Start a goroutine to continuously apply the replication stream
	go func() {
		for record := range stream {
			replica.mu.Lock()
			replica.Data[record.ID] = record.Value
			fmt.Printf("[%s] REPLICATED: id=%d, val=%s\n", replica.Name, record.ID, record.Value)
			replica.mu.Unlock()
		}
	}()
}

// ── Test Harness ──

func main() {
	master := NewDatabase("Master", true)
	replica1 := NewDatabase("Replica-1", false)
	replica2 := NewDatabase("Replica-2", false)

	master.AttachReplica(replica1)
	master.AttachReplica(replica2)

	fmt.Println("--- 1. Client Writes to Master ---")
	master.Write(1, "Alice")
	master.Write(2, "Bob")

	fmt.Println("\n--- 2. Immediate Read from Replica (Demonstrating Replication Lag) ---")
	// The replication goroutines take 50ms due to simulated network latency.
	// If we read immediately, the data won't be there yet.
	fmt.Printf("[Client] Read id=1 from %s: %s\n", replica1.Name, replica1.Read(1))

	fmt.Println("\n--- 3. Waiting for replication to catch up ---")
	time.Sleep(100 * time.Millisecond)

	fmt.Println("\n--- 4. Read from Replicas after delay ---")
	fmt.Printf("[Client] Read id=1 from %s: %s\n", replica1.Name, replica1.Read(1))
	fmt.Printf("[Client] Read id=2 from %s: %s\n", replica2.Name, replica2.Read(2))
}

Sample Output

--- 1. Client Writes to Master ---
[Master] WRITTEN: id=1, val=Alice
[Master] WRITTEN: id=2, val=Bob

--- 2. Immediate Read from Replica (Demonstrating Replication Lag) ---
[Client] Read id=1 from Replica-1: <not found>

--- 3. Waiting for replication to catch up ---
[Replica-1] REPLICATED: id=1, val=Alice
[Replica-2] REPLICATED: id=1, val=Alice
[Replica-1] REPLICATED: id=2, val=Bob
[Replica-2] REPLICATED: id=2, val=Bob

--- 4. Read from Replicas after delay ---
[Client] Read id=1 from Replica-1: Alice
[Client] Read id=2 from Replica-2: Bob

8. Complexity

9. Trade-offs

10. Production Evolution

11. Common Bugs

12. Interview Questions

1. What is Replication Lag and how does it impact user experience? Hint: Asynchronous replication takes time. A user might write data and immediately read old data.

2. How do you solve the read-after-write inconsistency problem? Hint: Pin the user's session to the Master for a short time after they execute a write operation.

3. Does adding more Replicas improve write performance? Hint: No! In fact, it often hurts write performance slightly because the Master has to duplicate the WAL to more destinations. It only improves read performance.

4. What is the difference between Synchronous and Asynchronous replication? Hint: Synchronous guarantees zero data loss but is slow and hurts availability. Async is fast but risks data loss if the master crashes before replicating.

13. Used By (Downstream Blocks)

• 020 Database Sharding — Replication only scales reads. Sharding is required to scale writes.
• 021 CAP Theorem — Network partitions force distributed databases to choose between Consistency and Availability, deeply tied to replication choices.
• 022 Leader Election — Required to automatically promote a replica if the master dies.

14. Used In (Case Studies)

15. Related Blocks

16. Try It Yourself

Exercise 1: Semi-Synchronous Replication

Modify the Go implementation to be Semi-Synchronous. The master.Write function should block until it receives an acknowledgment (via a chan bool) from at least one replica, and then return success to the client.

Exercise 2: Master Promotion (Failover)

Add a function PromoteReplica(replicaName string) that removes the current master, sets the chosen replica's IsMaster = true, and re-routes the other replicas' streams to listen to the new master.

Website Metadata