HTTP & TCP/IP Fundamentals

Metadata

1. Overview

What problem does it solve?

Two computers need to exchange data reliably over an unreliable network. TCP solves the transport problem (reliable, ordered byte delivery), and HTTP solves the application problem (a structured request-response protocol for transferring hypertext, APIs, and data).

What breaks without it?

Everything. Every system design — TinyURL, WhatsApp, Uber, Netflix — starts with a client sending an HTTP request. Without understanding HTTP:

• You can't reason about latency (connection setup, TLS handshake, keep-alive)
• You can't design APIs (methods, status codes, headers)
• You can't debug production issues (timeouts, retries, connection pooling)
• You can't compare protocols (HTTP/1.1 vs HTTP/2 vs HTTP/3, REST vs WebSocket)

2. Motivation

Why was TCP invented?

In the 1970s, the early ARPANET used NCP (Network Control Program), which assumed a reliable underlying network. As the network grew, packets got lost, duplicated, and reordered. Vint Cerf and Bob Kahn designed TCP (1974) to provide reliable, ordered, error-checked delivery over an unreliable network (IP).

Why was HTTP invented?

In 1989, Tim Berners-Lee at CERN needed a way for physicists to share documents across the internet. He created HTTP as a simple text-based protocol: send a request with a method and path, get a response with a status code and body. HTTP/0.9 (1991) only supported GET. HTTP/1.0 (1996) added headers, status codes, and content types. HTTP/1.1 (1997) added persistent connections and chunked transfer.

3. Real-World Usage

4. Prerequisites

• Basic programming knowledge (variables, functions, loops)
• Understanding of what a "network" is (machines connected together)
• No previous building blocks required — this is Block 001

5. Visual Explanation

The TCP/IP Layer Model

┌─────────────────────────────────────────────┐
│  Application Layer   (HTTP, DNS, SMTP)      │  ← You write code here
├─────────────────────────────────────────────┤
│  Transport Layer     (TCP, UDP)             │  ← Reliable delivery
├─────────────────────────────────────────────┤
│  Network Layer       (IP)                   │  ← Routing between hosts
├─────────────────────────────────────────────┤
│  Link Layer          (Ethernet, Wi-Fi)      │  ← Physical transmission
└─────────────────────────────────────────────┘

TCP Three-Way Handshake

sequenceDiagram
    participant C as Client
    participant S as Server

    Note over C,S: Connection Establishment
    C->>S: SYN (seq=100)
    S->>C: SYN-ACK (seq=300, ack=101)
    C->>S: ACK (ack=301)
    Note over C,S: Connection Established

    Note over C,S: Data Transfer
    C->>S: HTTP Request (PSH, seq=101)
    S->>C: ACK (ack=201)
    S->>C: HTTP Response (PSH, seq=301)
    C->>S: ACK (ack=501)

    Note over C,S: Connection Teardown
    C->>S: FIN
    S->>C: FIN-ACK
    C->>S: ACK

HTTP Request-Response Flow

Client                                    Server
  │                                         │
  │  ──── TCP Handshake (SYN/SYN-ACK/ACK) ─────
  │                                         │
  │  GET /api/users HTTP/1.1                │
  │  Host: api.example.com                  │
  │  Accept: application/json               │
  │  ─────────────────────────────────────► │
  │                                         │
  │                    HTTP/1.1 200 OK       │
  │                    Content-Type: json    │
  │                    Content-Length: 42    │
  │                                         │
  │                    {"users": [...]}      │
  │  ◄───────────────────────────────────── │
  │                                         │
  │  ──── Connection: keep-alive ──────────
  │       (reuse for next request)          │

6. Internal Working

TCP — Step by Step

Problem: IP (Internet Protocol) delivers packets unreliably. Packets can arrive out of order, get duplicated, or vanish entirely.

TCP's solution — four mechanisms:

6.1 Sequencing

Every byte in the stream gets a sequence number. The receiver reassembles bytes in order, regardless of arrival order.

Sent:    [seq=1, 100 bytes] [seq=101, 100 bytes] [seq=201, 50 bytes]
Arrived: [seq=201, 50 bytes] [seq=1, 100 bytes]   [seq=101, 100 bytes]
Result:  [bytes 1-100] [bytes 101-200] [bytes 201-250]  ✓ Correct order

6.2 Acknowledgment

The receiver sends back an ACK with the next expected sequence number. If the sender doesn't receive an ACK within a timeout, it retransmits.

6.3 Flow Control

The receiver advertises a window size — how many bytes it can buffer. The sender never sends more than the window allows. This prevents a fast sender from overwhelming a slow receiver.

6.4 Congestion Control

TCP starts slow (slow start) and doubles the sending rate each round trip until it detects loss. On loss, it halves the rate (congestion avoidance). This prevents the network itself from being overwhelmed.

HTTP — Step by Step

HTTP is a text-based, stateless, request-response protocol layered on top of TCP.

6.5 HTTP Request Structure

METHOD PATH HTTP/VERSION\r\n       ← Request line
Header-Name: Header-Value\r\n     ← Headers (key-value pairs)
Header-Name: Header-Value\r\n
\r\n                                ← Empty line = end of headers
[optional body]                    ← Request body (POST, PUT)

6.6 HTTP Response Structure

HTTP/VERSION STATUS_CODE REASON\r\n   ← Status line
Header-Name: Header-Value\r\n        ← Response headers
\r\n                                   ← Empty line
[response body]                       ← The actual content

6.7 Key HTTP Methods

6.8 Key Status Codes

7. Implementation

Why Python? Python's socket module exposes raw TCP with minimal boilerplate, making the TCP→HTTP layering visible. No frameworks — just the protocol itself.

"""
001 - HTTP Server from Scratch
Demonstrates TCP socket programming + HTTP protocol parsing.
No frameworks. No libraries. Just sockets.
"""
import socket
import threading
from datetime import datetime, timezone


def parse_http_request(raw: bytes) -> dict:
    """Parse raw bytes into an HTTP request dictionary."""
    text = raw.decode("utf-8", errors="replace")
    lines = text.split("\r\n")

    # Parse request line: GET /path HTTP/1.1
    request_line = lines[0].split(" ")
    method = request_line[0] if len(request_line) > 0 else "GET"
    path = request_line[1] if len(request_line) > 1 else "/"
    version = request_line[2] if len(request_line) > 2 else "HTTP/1.1"

    # Parse headers
    headers = {}
    body_start = 0
    for i, line in enumerate(lines[1:], start=1):
        if line == "":
            body_start = i + 1
            break
        if ":" in line:
            key, value = line.split(":", 1)
            headers[key.strip().lower()] = value.strip()

    # Extract body
    body = "\r\n".join(lines[body_start:]) if body_start < len(lines) else ""

    return {
        "method": method,
        "path": path,
        "version": version,
        "headers": headers,
        "body": body,
    }


def build_http_response(status_code: int, body: str, content_type: str = "text/plain") -> bytes:
    """Build a raw HTTP response from components."""
    reason_phrases = {
        200: "OK", 201: "Created", 301: "Moved Permanently",
        400: "Bad Request", 404: "Not Found", 429: "Too Many Requests",
        500: "Internal Server Error",
    }
    reason = reason_phrases.get(status_code, "Unknown")
    now = datetime.now(timezone.utc).strftime("%a, %d %b %Y %H:%M:%S GMT")
    body_bytes = body.encode("utf-8")

    response = (
        f"HTTP/1.1 {status_code} {reason}\r\n"
        f"Content-Type: {content_type}\r\n"
        f"Content-Length: {len(body_bytes)}\r\n"
        f"Date: {now}\r\n"
        f"Connection: close\r\n"
        f"\r\n"
    ).encode("utf-8") + body_bytes

    return response


# ── Simple Router ──

# In-memory "database" for demonstration
short_urls = {"abc123": "https://sourav-saha.in"}


def handle_request(request: dict) -> bytes:
    """Route requests to handlers — like a tiny TinyURL."""
    method = request["method"]
    path = request["path"]

    # GET / → Homepage
    if method == "GET" and path == "/":
        return build_http_response(200, '{"status": "ok", "service": "building-block-001"}',
                                   "application/json")

    # GET /shorten?url=... → Create short URL (simplified)
    if method == "GET" and path.startswith("/shorten"):
        return build_http_response(201, '{"short_url": "/abc123"}',
                                   "application/json")

    # GET /<code> → Redirect (301) — this is how TinyURL works!
    if method == "GET" and path.startswith("/"):
        code = path[1:]
        if code in short_urls:
            redirect_body = f"Redirecting to {short_urls[code]}"
            resp = (
                f"HTTP/1.1 301 Moved Permanently\r\n"
                f"Location: {short_urls[code]}\r\n"
                f"Content-Length: {len(redirect_body)}\r\n"
                f"Connection: close\r\n"
                f"\r\n"
                f"{redirect_body}"
            ).encode("utf-8")
            return resp

    # 404 for everything else
    return build_http_response(404, '{"error": "not found"}', "application/json")


def handle_client(client_socket: socket.socket, address: tuple):
    """Handle a single TCP connection."""
    try:
        # TCP receive — read up to 4096 bytes
        raw_data = client_socket.recv(4096)
        if not raw_data:
            return

        # Parse HTTP from raw TCP bytes
        request = parse_http_request(raw_data)
        print(f"[{address[0]}:{address[1]}] {request['method']} {request['path']}")

        # Route and respond
        response = handle_request(request)
        client_socket.sendall(response)  # TCP send — reliable delivery guaranteed

    except Exception as e:
        error_resp = build_http_response(500, f'{{"error": "{str(e)}"}}', "application/json")
        client_socket.sendall(error_resp)
    finally:
        client_socket.close()  # TCP FIN — connection teardown


def start_server(host: str = "127.0.0.1", port: int = 8080):
    """
    Start a TCP server that speaks HTTP.

    The layering is visible:
    1. socket()      → Create a TCP socket
    2. bind()        → Attach to an address
    3. listen()      → Mark as a server socket
    4. accept()      → Wait for TCP handshake (SYN → SYN-ACK → ACK)
    5. recv()        → Read bytes from the TCP stream
    6. parse HTTP    → Interpret those bytes as an HTTP request
    7. sendall()     → Write HTTP response bytes to the TCP stream
    8. close()       → TCP teardown (FIN → FIN-ACK → ACK)
    """
    server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    server_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    server_socket.bind((host, port))
    server_socket.listen(128)  # Backlog: max pending connections

    print(f"HTTP Server listening on http://{host}:{port}")
    print(f"Try: curl http://{host}:{port}/")
    print(f"Try: curl http://{host}:{port}/abc123 -v")

    try:
        while True:
            # accept() blocks until a client completes the TCP 3-way handshake
            client_socket, address = server_socket.accept()
            # Handle each connection in a separate thread
            thread = threading.Thread(target=handle_client, args=(client_socket, address))
            thread.daemon = True
            thread.start()
    except KeyboardInterrupt:
        print("\nShutting down.")
    finally:
        server_socket.close()


if __name__ == "__main__":
    start_server()

Sample Output

$ python 001_http_server.py
HTTP Server listening on http://127.0.0.1:8080
Try: curl http://127.0.0.1:8080/
Try: curl http://127.0.0.1:8080/abc123 -v

# Terminal 2:
$ curl http://127.0.0.1:8080/
{"status": "ok", "service": "building-block-001"}

$ curl -v http://127.0.0.1:8080/abc123
< HTTP/1.1 301 Moved Permanently
< Location: https://sourav-saha.in
Redirecting to https://sourav-saha.in

$ curl http://127.0.0.1:8080/nonexistent
{"error": "not found"}

8. Complexity

Scalability Characteristics

• HTTP/1.1: One request per connection at a time (head-of-line blocking). Mitigated by keep-alive and pipelining.
• HTTP/2: Multiplexed streams over a single TCP connection. Solves HOL at HTTP layer.
• HTTP/3: QUIC over UDP. Solves HOL at TCP layer.

9. Trade-offs

When NOT to use HTTP

• Real-time bidirectional: Use WebSockets (Block 003)
• Ultra-low-latency RPC: Use gRPC (HTTP/2 + Protobuf)
• Streaming video: Use HTTP but with DASH/HLS chunking
• Inter-service communication: Consider gRPC or message queues (Block 023)

10. Production Evolution

11. Common Bugs

12. Interview Questions

1. What happens when you type a URL in the browser and press Enter? Hint: DNS → TCP handshake → TLS → HTTP request → server processing → response → rendering

2. Why does HTTP use TCP instead of UDP? Hint: Web pages need every byte delivered correctly and in order

3. Explain the difference between HTTP/1.1 keep-alive and HTTP/2 multiplexing. Hint: Keep-alive reuses the connection but still serializes requests; HTTP/2 interleaves frames

4. What is head-of-line blocking and how does HTTP/3 solve it? Hint: TCP treats all streams as one byte stream — one lost packet blocks everything

5. How would you handle 10,000 concurrent connections on a single server? Hint: Event-driven I/O (epoll) instead of thread-per-connection — this is the C10K problem

13. Used By (Downstream Blocks)

• 002 REST API Design — layered on HTTP
• 003 WebSockets — HTTP Upgrade handshake
• 004 DNS — DNS queries travel over UDP, but zone transfers use TCP
• 007 Load Balancer — distributes TCP/HTTP connections
• 008 API Gateway — terminates HTTP, routes to upstream
• 009 Rate Limiter — operates on HTTP requests
• 010 Caching — HTTP caching headers (ETag, Cache-Control)
• 012 CDN — serves HTTP responses from edge servers
• 027 Heartbeat — health checks over HTTP
• 029 Retry & Backoff — retrying failed HTTP requests
• 035 Observability — HTTP request metrics and tracing

14. Used In (Case Studies)

15. Related Blocks

16. Try It Yourself

Exercise 1: Add POST Support

Extend the server to handle POST /shorten with a JSON body {"url": "https://example.com"}. Generate a random short code, store it in the short_urls dictionary, and return 201 Created with the short URL.

Exercise 2: Connection Keep-Alive

Modify the server to support Connection: keep-alive. Instead of closing the socket after one request, loop and handle multiple requests on the same connection. Add a 5-second idle timeout.

Website Metadata