HTTPS & TLS

Overview

HTTPS is HTTP over TLS.

TLS (Transport Layer Security) is a cryptographic protocol that provides:

• Confidentiality — data is encrypted; eavesdroppers cannot read it.
• Integrity — data cannot be tampered with in transit.
• Authentication — the server proves its identity via a certificate.

Without TLS:

Client ──── plaintext ────► Server
             (anyone can read)


With TLS:

Client ──── encrypted ────► Server
             (only endpoints can read)

Every serious production system uses TLS.

Without it, passwords, tokens, API keys, and personal data travel as plaintext across the network.

Why does it exist?

HTTP was designed as a plaintext protocol.

Anyone on the network path can:

• Read every request and response.
• Modify data in transit (man-in-the-middle attack).
• Impersonate a server.
• Steal credentials, cookies, and sessions.

TLS solves all four problems.

It wraps HTTP in a secure channel so that the conversation between client and server is private, authentic, and tamper-proof.

Real-world Motivation

HTTPS is not optional anymore.

Google Chrome

Marks all HTTP sites as "Not Secure" since 2018.

Let's Encrypt

Issues free TLS certificates. Over 300 million active certificates.

Cloudflare

Terminates TLS for millions of websites at the edge.

AWS / GCP / Azure

All cloud providers offer managed TLS through load balancers and CDNs.

NGINX & Envoy

Handle TLS termination as reverse proxies in production architectures.

Apple & Google

Both require HTTPS for all mobile app API calls (App Transport Security / Android Network Security Config).

HTTP/2 and HTTP/3

Both effectively require TLS. Browsers only support HTTP/2 over TLS.

Why Existing Solutions Fail

Before TLS, systems tried other approaches:

Application-level encryption

Encrypt individual fields (e.g., passwords) before sending.

Problems:

• Metadata (URLs, headers, cookies) still exposed.
• Every application must implement encryption independently.
• Key management becomes a nightmare.
• No server authentication.

VPN

Encrypt the entire network tunnel.

Problems:

• Heavy infrastructure.
• Not suitable for public-facing services.
• Performance overhead.
• Does not authenticate individual servers.

IPsec

Encryption at the IP layer.

Problems:

• Complex to configure.
• Not designed for web applications.
• Cannot distinguish between different services on the same host.

TLS operates at the right layer — between TCP and HTTP — providing encryption transparently to all application data.

Internal Working

TLS establishes a secure connection through a handshake before any application data is exchanged.

TLS 1.3 Handshake

TLS 1.3 (the current standard) uses a single round trip.

sequenceDiagram
    participant Client
    participant Server

    Note over Client: Step 1: ClientHello
    Client->>Server: ClientHello + Key Share + Supported Ciphers

    Note over Server: Step 2: ServerHello
    Server->>Client: ServerHello + Key Share + Chosen Cipher

    Note over Server: Step 3: Server sends encrypted data
    Server->>Client: Certificate + CertificateVerify + Finished

    Note over Client: Step 4: Client verifies
    Client->>Client: Verify Certificate Chain
    Client->>Client: Verify Server Signature

    Note over Client: Step 5: Client confirms
    Client->>Server: Finished

    Note over Client,Server: Secure channel established
    Client->>Server: Encrypted Application Data
    Server->>Client: Encrypted Application Data

Step-by-Step Explanation

Step 1: ClientHello

The client sends:

• Supported TLS versions (e.g., TLS 1.3).
• Supported cipher suites (e.g., TLS_AES_256_GCM_SHA384).
• A key share — the client's half of the key exchange (using Diffie-Hellman).
• A random nonce.

Step 2: ServerHello

The server responds with:

• Chosen TLS version.
• Chosen cipher suite.
• The server's key share — the server's half of the Diffie-Hellman exchange.

At this point, both sides can compute the shared secret independently.

Step 3: Server Certificate

The server sends:

• Its X.509 certificate — proving identity.
• A CertificateVerify message — a digital signature over the handshake transcript using the server's private key.
• A Finished message — a MAC over the entire handshake.

All of this is already encrypted using the shared secret.

Step 4: Client Verification

The client:

• Validates the certificate chain up to a trusted root CA.
• Checks the domain name matches the certificate.
• Checks the certificate is not expired or revoked.
• Verifies the CertificateVerify signature.

If any check fails, the connection is aborted.

Step 5: Handshake Complete

The client sends its Finished message.

Both sides now have a symmetric encryption key derived from the shared secret.

All subsequent data is encrypted with AES-GCM or ChaCha20-Poly1305.

TLS 1.2 vs TLS 1.3

TLS 1.3 is faster and more secure.

Forward Secrecy

TLS 1.3 mandates forward secrecy.

This means: even if the server's private key is compromised in the future, past recorded sessions cannot be decrypted.

This is achieved by using ephemeral Diffie-Hellman (DHE or ECDHE) for every connection.

Each connection generates a unique key pair that is discarded after use.

Session 1: Key A (discarded)
Session 2: Key B (discarded)
Session 3: Key C (discarded)

Server private key leaked →
     Cannot decrypt any past session.

Certificate Chain

Certificates form a chain of trust.

flowchart TD
    A["Root CA<br/>(Trusted by OS/Browser)"]
    B["Intermediate CA<br/>(Issued by Root CA)"]
    C["Server Certificate<br/>(Issued by Intermediate CA)"]
    D["Your Server"]

    A -->|signs| B
    B -->|signs| C
    C -->|installed on| D

Root CA

Pre-installed in operating systems and browsers.

Examples: DigiCert, Let's Encrypt (ISRG Root), GlobalSign.

Intermediate CA

Signed by the Root CA.

Used to issue server certificates.

Keeps the root key offline for security.

Server Certificate

Contains:

• Domain name (e.g., sourav-saha.in).
• Public key.
• Validity period.
• Issuer.
• Digital signature from the Intermediate CA.

Data Structures

X.509 Certificate

Certificate:
    Version: 3
    Serial Number: 04:e3:2b:...
    Signature Algorithm: SHA256withRSA
    Issuer: CN=Let's Encrypt Authority X3
    Validity:
        Not Before: Jan 1 2024
        Not After:  Apr 1 2024
    Subject: CN=sourav-saha.in
    Subject Public Key:
        Algorithm: ECDSA P-256
        Public Key: 04:a1:b2:...
    Extensions:
        Subject Alternative Name:
            DNS: sourav-saha.in
            DNS: www.sourav-saha.in
        Key Usage: Digital Signature
        Basic Constraints: CA:FALSE

Session State

During an active TLS connection, the following state is maintained:

TLS Session:
    Protocol Version: TLS 1.3
    Cipher Suite: TLS_AES_256_GCM_SHA384
    Master Secret: [derived key material]
    Sequence Number: [monotonically increasing]
    Certificate: [server's X.509 cert]

Cipher Suite

A cipher suite defines the algorithms used:

TLS_AES_256_GCM_SHA384

TLS  → Protocol
AES_256_GCM → Symmetric encryption (AES with 256-bit key in GCM mode)
SHA384 → Hash function for key derivation

In TLS 1.3, the key exchange (ECDHE) is negotiated separately.

Algorithms

Diffie-Hellman Key Exchange (Simplified)

The core idea: two parties compute a shared secret over a public channel without ever transmitting the secret.

1. Agree on public parameters: prime p, generator g

2. Alice picks private key: a
   Alice computes: A = g^a mod p
   Alice sends A to Bob

3. Bob picks private key: b
   Bob computes: B = g^b mod p
   Bob sends B to Alice

4. Shared secret:
   Alice computes: S = B^a mod p = g^(ab) mod p
   Bob computes:   S = A^b mod p = g^(ab) mod p

Both arrive at the same S without transmitting it.

In practice, TLS uses Elliptic Curve Diffie-Hellman (ECDHE) on curves like X25519 or P-256 for better performance and security.

Symmetric Encryption (AES-GCM)

After the handshake, all data is encrypted with AES-GCM:

Plaintext + Key + Nonce → Ciphertext + Authentication Tag

Properties:
  - Confidentiality: data encrypted
  - Integrity: authentication tag detects tampering
  - Speed: hardware-accelerated (AES-NI instruction set)

Certificate Verification Algorithm

verify_certificate(cert, trusted_roots):

    1. Check cert.not_before <= now <= cert.not_after
    2. Check cert.subject matches requested domain
    3. Find issuer certificate
    4. Verify signature using issuer's public key
    5. If issuer is a trusted root → VALID
    6. If issuer is intermediate → recurse: verify_certificate(issuer, trusted_roots)
    7. If chain is broken → INVALID

Implementation

Language Choice

Go

Why Go?

• First-class TLS support in the standard library (crypto/tls).
• Widely used for infrastructure and networking tools.
• Minimal boilerplate for TLS server/client.
• Real-world relevance: Caddy, Traefik, and many TLS tools are written in Go.
• Easy certificate generation for development.

Complete Implementation

This implementation creates:

1. A self-signed CA and server certificate.
2. A TLS server.
3. A TLS client that verifies the certificate.

Certificate Generator

// certgen.go — generates self-signed CA + server certificate for development
package main

import (
	"crypto/ecdsa"
	"crypto/elliptic"
	"crypto/rand"
	"crypto/x509"
	"crypto/x509/pkix"
	"encoding/pem"
	"log"
	"math/big"
	"net"
	"os"
	"time"
)

func main() {
	// ── Step 1: Generate CA key pair ──
	caKey, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
	if err != nil {
		log.Fatal(err)
	}

	// ── Step 2: Create CA certificate template ──
	caTemplate := &x509.Certificate{
		SerialNumber: big.NewInt(1),
		Subject: pkix.Name{
			Organization: []string{"Demo CA"},
		},
		NotBefore:             time.Now(),
		NotAfter:              time.Now().Add(365 * 24 * time.Hour),
		IsCA:                  true,
		KeyUsage:              x509.KeyUsageCertSign | x509.KeyUsageCRLSign,
		BasicConstraintsValid: true,
	}

	// ── Step 3: Self-sign the CA certificate ──
	caCertDER, err := x509.CreateCertificate(
		rand.Reader,
		caTemplate,
		caTemplate, // self-signed: parent == template
		&caKey.PublicKey,
		caKey,
	)
	if err != nil {
		log.Fatal(err)
	}

	// Write CA certificate to file
	writePEM("ca-cert.pem", "CERTIFICATE", caCertDER)

	// Parse CA cert for signing server cert
	caCert, err := x509.ParseCertificate(caCertDER)
	if err != nil {
		log.Fatal(err)
	}

	// ── Step 4: Generate server key pair ──
	serverKey, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
	if err != nil {
		log.Fatal(err)
	}

	// ── Step 5: Create server certificate template ──
	serverTemplate := &x509.Certificate{
		SerialNumber: big.NewInt(2),
		Subject: pkix.Name{
			Organization: []string{"Demo Server"},
		},
		NotBefore: time.Now(),
		NotAfter:  time.Now().Add(365 * 24 * time.Hour),

		// Valid for localhost connections
		IPAddresses: []net.IP{net.ParseIP("127.0.0.1")},
		DNSNames:    []string{"localhost"},

		KeyUsage:    x509.KeyUsageDigitalSignature,
		ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},
	}

	// ── Step 6: Sign server certificate with CA ──
	serverCertDER, err := x509.CreateCertificate(
		rand.Reader,
		serverTemplate,
		caCert, // signed by CA
		&serverKey.PublicKey,
		caKey, // signed with CA's private key
	)
	if err != nil {
		log.Fatal(err)
	}

	// Write server certificate and key
	writePEM("server-cert.pem", "CERTIFICATE", serverCertDER)

	serverKeyDER, err := x509.MarshalECPrivateKey(serverKey)
	if err != nil {
		log.Fatal(err)
	}
	writePEM("server-key.pem", "EC PRIVATE KEY", serverKeyDER)

	log.Println("Generated: ca-cert.pem, server-cert.pem, server-key.pem")
}

// writePEM writes DER-encoded data as a PEM file.
func writePEM(filename, blockType string, data []byte) {
	f, err := os.Create(filename)
	if err != nil {
		log.Fatal(err)
	}
	defer f.Close()

	err = pem.Encode(f, &pem.Block{Type: blockType, Bytes: data})
	if err != nil {
		log.Fatal(err)
	}
}

TLS Server

// server.go — HTTPS server with TLS
package main

import (
	"crypto/tls"
	"fmt"
	"log"
	"net/http"
)

func main() {
	// ── Load server certificate and private key ──
	cert, err := tls.LoadX509KeyPair("server-cert.pem", "server-key.pem")
	if err != nil {
		log.Fatalf("Failed to load certificate: %v", err)
	}

	// ── Configure TLS ──
	tlsConfig := &tls.Config{
		Certificates: []tls.Certificate{cert},

		// Only allow TLS 1.3 for maximum security
		MinVersion: tls.VersionTLS13,
		MaxVersion: tls.VersionTLS13,
	}

	// ── Set up HTTP handler ──
	mux := http.NewServeMux()
	mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		// Log TLS connection details
		if r.TLS != nil {
			log.Printf("TLS Version: %x", r.TLS.Version)
			log.Printf("Cipher Suite: %s",
				tls.CipherSuiteName(r.TLS.CipherSuite))
			log.Printf("Server Name: %s", r.TLS.ServerName)
		}

		fmt.Fprintf(w, "Hello over TLS!\n")
		fmt.Fprintf(w, "Protocol: %s\n", r.Proto)
		fmt.Fprintf(w, "Method: %s\n", r.Method)
		fmt.Fprintf(w, "Path: %s\n", r.URL.Path)
	})

	mux.HandleFunc("/health", func(w http.ResponseWriter, r *http.Request) {
		fmt.Fprintf(w, "OK\n")
	})

	// ── Create HTTPS server ──
	server := &http.Server{
		Addr:      ":8443",
		Handler:   mux,
		TLSConfig: tlsConfig,
	}

	log.Println("TLS Server running on https://localhost:8443")

	// ListenAndServeTLS uses the certificates from TLSConfig
	err = server.ListenAndServeTLS("", "")
	if err != nil {
		log.Fatal(err)
	}
}

TLS Client

// client.go — HTTPS client that verifies server certificate
package main

import (
	"crypto/tls"
	"crypto/x509"
	"fmt"
	"io"
	"log"
	"net/http"
	"os"
)

func main() {
	// ── Step 1: Load CA certificate to trust ──
	caCert, err := os.ReadFile("ca-cert.pem")
	if err != nil {
		log.Fatalf("Failed to read CA cert: %v", err)
	}

	// ── Step 2: Create certificate pool ──
	caCertPool := x509.NewCertPool()
	if !caCertPool.AppendCertsFromPEM(caCert) {
		log.Fatal("Failed to add CA cert to pool")
	}

	// ── Step 3: Configure TLS client ──
	tlsConfig := &tls.Config{
		RootCAs:    caCertPool,      // trust our CA
		MinVersion: tls.VersionTLS13, // enforce TLS 1.3
	}

	// ── Step 4: Create HTTPS client ──
	client := &http.Client{
		Transport: &http.Transport{
			TLSClientConfig: tlsConfig,
		},
	}

	// ── Step 5: Make request ──
	resp, err := client.Get("https://localhost:8443/")
	if err != nil {
		log.Fatalf("Request failed: %v", err)
	}
	defer resp.Body.Close()

	body, err := io.ReadAll(resp.Body)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Printf("Status: %s\n", resp.Status)
	fmt.Printf("TLS Version: %x\n", resp.TLS.Version)
	fmt.Printf("Cipher Suite: %s\n",
		tls.CipherSuiteName(resp.TLS.CipherSuite))
	fmt.Printf("\nResponse:\n%s", body)
}

Test Harness

Step 1: Generate certificates

go run certgen.go

Step 2: Start TLS server

go run server.go

Step 3: Run client

go run client.go

Alternative: test with curl

curl --cacert ca-cert.pem https://localhost:8443/

Sample Output

Server log

TLS Server running on https://localhost:8443
TLS Version: 304
Cipher Suite: TLS_AES_128_GCM_SHA256
Server Name: localhost

Client output

Status: 200 OK
TLS Version: 304
Cipher Suite: TLS_AES_128_GCM_SHA256

Response:
Hello over TLS!
Protocol: HTTP/2.0
Method: GET
Path: /

Complexity

Space complexity:

O(sessions × key_material)

Per connection:
  ~500 bytes for TLS session state
  ~2-4 KB for certificate storage

Trade-offs

Advantages

• Confidentiality — all data encrypted.
• Integrity — tamper detection built in.
• Authentication — server identity verified.
• Forward secrecy — past sessions safe even if key leaks.
• Industry standard — universal browser/client support.
• Enables HTTP/2 and HTTP/3.

Disadvantages

• Handshake latency — 1 RTT for TLS 1.3 (2 RTT for TLS 1.2).
• CPU overhead — encryption/decryption costs (mitigated by AES-NI).
• Certificate management — renewal, rotation, revocation.
• Debugging difficulty — encrypted traffic harder to inspect.
• Complexity — more failure modes (expired certs, mismatched domains).

Alternatives

Production Improvements

TLS Termination at Reverse Proxy

Most production systems terminate TLS at the reverse proxy (NGINX, Envoy, Caddy).

Client ── TLS ──► Reverse Proxy ── plaintext ──► Backend

Benefits:
  - Backends don't manage certificates
  - Centralized TLS configuration
  - Better performance (connection reuse)

See: Reverse Proxy for details on TLS termination.

Automatic Certificate Renewal

Let's Encrypt + ACME protocol.

Tools like Certbot and Caddy automatically renew certificates before expiry.

Certificate expires in 30 days
    → ACME client requests renewal
    → New certificate issued
    → Hot-reloaded without downtime

OCSP Stapling

Instead of the client checking certificate revocation with the CA, the server periodically fetches the OCSP response and staples it to the TLS handshake.

Without stapling:
  Client → CA: "Is this cert revoked?" (extra latency)

With stapling:
  Server → CA: "Give me my OCSP status" (background)
  Server → Client: certificate + OCSP response (no extra latency)

Session Resumption

TLS 1.3 supports 0-RTT resumption.

Returning clients can send encrypted data in the first packet.

First connection: 1-RTT handshake
Resumed connection: 0-RTT (data in first packet)

Trade-off: 0-RTT data is vulnerable to replay attacks. Only use for idempotent requests.

Hardware Acceleration

Modern CPUs include AES-NI instructions.

Encryption/decryption throughput:

Without AES-NI: ~500 MB/s
With AES-NI:    ~5 GB/s (10x improvement)

Certificate Transparency

All certificates are logged to public Certificate Transparency (CT) logs.

Allows domain owners to detect unauthorized certificates.

Google Chrome requires CT for all certificates.

Monitoring

Track:

• Certificate expiry dates.
• TLS handshake failure rates.
• Cipher suites in use.
• Protocol version distribution.
• Handshake latency (p50, p99).
• Certificate chain validation errors.

Common Bugs

1. Expired certificates

The most common TLS outage. Set up automated renewal and monitoring alerts.

2. Missing intermediate certificates

Server sends its certificate but not the intermediate CA. Some clients cannot build the chain.

Fix: always send the full certificate chain.

3. Hostname mismatch

Certificate issued for example.com but server accessed as www.example.com.

Fix: use Subject Alternative Names (SANs) with all valid domains.

4. Using TLS 1.0 or 1.1

Deprecated and insecure. Set MinVersion: tls.VersionTLS12 at minimum.

5. Weak cipher suites

Allowing old ciphers like RC4 or DES.

Fix: configure an explicit allowlist of strong cipher suites.

6. Disabling certificate verification

InsecureSkipVerify: true is common in development but catastrophic in production.

It silently disables all authentication.

7. Not rotating private keys

Long-lived private keys increase the blast radius of a compromise.

8. Clock skew

Certificate validation depends on accurate system time. Servers with wrong clocks reject valid certificates.

9. Forgetting to redirect HTTP to HTTPS

Users accessing http:// still send data in plaintext.

Fix: redirect all HTTP traffic to HTTPS. Use HSTS headers.

Interview Questions

1. Explain the TLS 1.3 handshake step by step.

Hint: ClientHello → ServerHello → key exchange → certificate verify → encrypted data. One round trip.

2. What is forward secrecy and why does it matter?

Hint: Ephemeral keys per session. Compromised long-term key cannot decrypt past sessions.

3. How does a client verify a server's certificate?

Hint: Certificate chain validation up to trusted root CA. Check domain, expiry, signature.

4. What is the difference between symmetric and asymmetric encryption in TLS?

Hint: Asymmetric (ECDHE) for key exchange, symmetric (AES-GCM) for bulk data. Hybrid approach.

5. Where should TLS termination happen in a microservices architecture?

Hint: At the edge (load balancer / reverse proxy). Consider mTLS for service-to-service communication.

Used By

This concept is required before understanding:

• ✅ API Gateway
• ✅ Load Balancer
• ✅ Reverse Proxy
• ✅ CDN
• ✅ TinyURL
• ✅ WhatsApp

Related Topics

Prerequisites

• HTTP & TCP Fundamentals
• DNS & Service Discovery
• Reverse Proxy

Next Topics

• LRU Cache

Related Pages

• API Gateway
• Load Balancer
• CDN
• Caching Strategies
• Rate Limiter

Try It Yourself

Exercise

Extend the TLS server to:

• Serve multiple domains using SNI (Server Name Indication).
• Log which cipher suite was negotiated per request.
• Implement an HTTP → HTTPS redirect on port 8080.
• Add the Strict-Transport-Security (HSTS) header.

Advanced Challenge

Build a complete mTLS (mutual TLS) system:

• Generate a client certificate signed by the same CA.
• Configure the server to require client certificates.
• Verify client identity on the server side.
• Reject connections from untrusted clients.
• Log the client certificate's Common Name for each request.
• Implement certificate revocation by serial number.

Website Integration

Suggested URL

/system-design/networking/https-tls

Breadcrumb

System Design

→ Networking

→ HTTPS & TLS

Sidebar Category

Networking

Previous Page

Reverse Proxy

Next Page

LRU Cache

Related Links

• HTTP & TCP Fundamentals
• DNS & Service Discovery
• Reverse Proxy
• REST API Design
• API Gateway
• Load Balancer
• CDN
• TinyURL
• WhatsApp

Hero Illustration Prompt

"A clean technical illustration showing a TLS handshake between a client and server. Depict a padlock forming in the center as encrypted data flows through a secure channel. Show certificate chain icons (Root CA → Intermediate CA → Server Certificate), key exchange symbols, and encryption ciphers. Modern flat engineering style with green and blue security accents, suitable for a backend engineering learning platform."

SEO Meta Description

Learn HTTPS and TLS from first principles. Understand the TLS 1.3 handshake, certificate chains, cipher suites, forward secrecy, and build a complete TLS server and client in Go with certificate generation, verification, and production best practices.