Networking & Protocols: How Bytes Actually Travel
Series: System Design · Networking & Protocols — Pillar 2 of 8 This pillar: 00 — Overview · 01 — OSI Model · 02 — TCP vs UDP · 03 — HTTP vs HTTPS · 04 — TLS/SSL · 05 — DNS · 06 — DNS Load Balancing · 07 — Anycast Routing · 08 — CDN · 09 — Wrap-up
Systems Design
| # | Post | What it covers |
|---|---|---|
| 00 | Networking & Protocols: How Bytes Actually Travel ← you are here | Before you can design systems that scale, you need to understand how bytes actually travel. Eight concepts every backend engineer must know. (148 chars) |
| 01 | The OSI Model: The Map Every Engineer Needs | The OSI model isn't just interview theory — it's the map that tells you exactly where in the stack a network problem lives. Here's how to use it. (152 chars) |
| 02 | TCP vs UDP: Reliability vs Speed at the Transport Layer | TCP guarantees delivery. UDP doesn't look back. Understanding why each exists — and when to reach for each — is fundamental to network design. (150 chars) |
| 03 | HTTP vs HTTPS: The Language of the Web and Its Secure Version | 301 Moved Permanently, 302 Found, 304 Not Modified |
| 04 | TLS/SSL: How HTTPS Actually Works Under the Hood | TLS is what puts the S in HTTPS. Here's how the handshake works, what a certificate actually contains, and why TLS 1.3 matters for performance. (152 chars) |
| 05 | DNS: The Phone Book That Runs the Internet | DNS is the phone book of the internet — and one of the most misunderstood layers in the stack. Here's how it works and how it fails. (133 chars) |
| 06 | DNS Load Balancing: Traffic Distribution at the Name Layer | DNS load balancing distributes traffic before a single packet reaches your servers. Here's how it works, where it excels, and where it falls short. (154 chars) |
| 07 | Anycast Routing: One Address, Everywhere at Once | One IP address, dozens of locations, zero client configuration. Anycast is how the fastest global infrastructure works — here's the mechanism behind it. (158 chars) |
| 08 | CDN: Moving Content Closer to the People Who Need It | A CDN isn't just a cache in front of your server. Here's how content delivery networks work, when they help, and when they add complexity for nothing. (154 chars) |
| 09 | Networking & Protocols: Wrap-Up | A complete recap of the eight core networking concepts — OSI, TCP, HTTP, TLS, DNS, CDN — and how they connect into a complete picture. (135 chars) |
Networking & Protocols: How Bytes Actually Travel
The scenario
You've just deployed a new feature. It works perfectly in your local environment. In production, users in Europe are seeing 800ms response times. Users in Sydney are timing out entirely. Your servers are healthy, your application code is fine, and your database queries are fast.
The problem is the network — but "the network" is not one thing. It's a stack of protocols, each responsible for a different layer of the journey your data makes from server to user. Without knowing how that stack works, you're debugging in the dark: restarting services, tweaking timeouts, and hoping something changes.
The Foundations pillar gave you the vocabulary for what a system should do — availability, consistency, reliability. This pillar is about the medium those properties live in. Every request your system handles travels through a network stack, resolves a domain name, negotiates a secure connection, and may traverse a global content delivery network before reaching a user. Understanding how each of those layers works — and how they fail — is the difference between diagnosing a networking problem in minutes and spending days changing the wrong things.
Eight concepts. By the end of this pillar, you'll be able to read a network trace, understand why your HTTPS handshake is slow, explain what DNS propagation actually means, and design a CDN strategy that doesn't accidentally serve stale content to half your users.
TL;DR: The OSI model gives you the map. TCP and UDP are the two dominant transport choices. HTTP and HTTPS define how your application communicates. TLS is the mechanism that makes HTTPS trustworthy. DNS turns human-readable names into addresses. DNS load balancing and Anycast routing are how large systems direct traffic globally. CDNs move your content closer to users. Know all eight, and the network stops being a black box.
What this pillar covers
The OSI model — the map of the stack
Every network communication in the world is conceptually organised into seven layers — from the physical electrical signals on a wire at the bottom to the application protocols your code speaks at the top. The OSI (Open Systems Interconnection) model is the framework that names and defines each layer.
Most engineers interact with layers 4 (Transport) and 7 (Application) daily — TCP, UDP, HTTP, WebSockets. But knowing what lives below those layers — IP routing at layer 3, Ethernet framing at layer 2 — is what lets you understand why a packet gets dropped, why a firewall is blocking your connection, or why a VPN changes your apparent location.
Best mental model: mailing a letter. The OSI layers are like the envelope, the postal address, the sorting office, the delivery van, and the postal worker — each layer responsible for a different part of the journey, each one handing off to the next.
TCP vs UDP — reliability vs speed at the transport layer
TCP (Transmission Control Protocol) and UDP (User Datagram Protocol) are the two dominant transport protocols. They solve the same problem — sending data across a network — with fundamentally different philosophies.
TCP guarantees delivery: packets are acknowledged, retransmitted if lost, and reassembled in order. It's reliable, but that reliability has overhead. UDP fires packets and doesn't look back: no acknowledgement, no retransmission, no ordering guarantee. It's fast, but you're responsible for handling loss.
The choice between them shapes the character of every protocol built on top: HTTP/2 runs on TCP. DNS queries use UDP. WebRTC uses UDP. Knowing why each made that choice is more valuable than memorising which one does which.
Best mental model: TCP is a recorded delivery letter — tracked, confirmed, redelivered if it goes missing. UDP is dropping a leaflet through every letterbox on the street — fast, high volume, and if one blows away in the wind, nobody's coming back for it.
HTTP vs HTTPS — the application protocol and its secure version
HTTP (Hypertext Transfer Protocol) is the language the web speaks. Every API call, every web page load, every webhook — almost all of it is HTTP. It defines how requests are structured, how responses are returned, how headers communicate metadata, and how status codes signal outcomes.
HTTPS is HTTP with a security layer underneath — TLS. The data is the same. The difference is that on HTTPS, that data is encrypted in transit, the server's identity is verified, and a passive observer on the network sees only ciphertext.
Understanding HTTP deeply — methods, status codes, caching headers, connection management — is a prerequisite for designing APIs, debugging integration failures, and understanding how browsers and servers negotiate communication.
Best mental model: HTTP is a postcard — readable by anyone who handles it. HTTPS is the same message in a sealed, tamper-evident envelope that only the intended recipient can open.
TLS/SSL — how HTTPS actually works
TLS (Transport Layer Security) is the protocol that puts the S in HTTPS. It does three things: encrypts the connection so nobody can read it in transit, authenticates the server so you know you're talking to who you think you are, and verifies integrity so tampering is detectable.
SSL is TLS's predecessor — technically obsolete, but the term persists colloquially. When someone says "SSL certificate," they mean a TLS certificate.
Understanding TLS matters beyond just knowing it exists. The TLS handshake — the negotiation that happens before any application data flows — adds latency to every new connection. TLS 1.3 reduced that significantly. Knowing why is what lets you optimise connection setup, troubleshoot certificate errors, and reason about the security properties your system actually has.
Best mental model: a TLS handshake is like two strangers agreeing to speak in code — first they verify each other's credentials, then they negotiate which cipher to use, then they start the encrypted conversation. It takes a few exchanges to set up, but after that it's seamless.
DNS — how names become addresses
DNS (Domain Name System) is the phone book of the internet. When you type api.example.com into a browser or make an API call to a hostname, DNS resolves that name into an IP address — the actual numerical address your network stack can route to.
DNS is so fundamental and so invisible that engineers rarely think about it until it breaks — and when it breaks, it breaks in ways that are slow, confusing, and affect everything simultaneously. DNS propagation delays, TTL misconfiguration, DNS caching at unexpected layers, and resolver failures are all production incidents waiting to happen for teams that treat DNS as someone else's problem.
Best mental model: DNS is like asking for directions. You don't know the address of the restaurant; you know its name. The DNS resolver is the local who knows the neighbourhood — they translate the name into an address you can navigate to.
DNS load balancing — directing traffic at the name layer
Standard load balancing happens at the network or application layer — a load balancer device receives traffic and distributes it across servers. DNS load balancing happens earlier: at name resolution time, different clients receive different IP addresses for the same hostname, distributing traffic before a single packet is sent to your infrastructure.
It's simpler than application-layer load balancing and requires no additional infrastructure — but it has meaningful limitations around health checking, TTL-driven stickiness, and geographic precision. Understanding where DNS load balancing excels and where it falls short is what prevents over-relying on it for use cases it isn't suited to.
Best mental model: instead of one restaurant address, the phone book gives different callers different branch locations based on where they're calling from. Simple, scalable — but the phone book can't know if a particular branch is full or closed.
Anycast routing — one address, many locations
Anycast is a network addressing technique where the same IP address is announced from multiple geographic locations simultaneously. When a client sends a packet to an Anycast address, the internet's routing infrastructure automatically delivers it to the nearest location announcing that address.
This is how Cloudflare, Google's public DNS (8.8.8.8), and major CDNs serve users globally from a single IP with minimal latency — without the client knowing which physical location handled the request. Understanding Anycast explains a class of global infrastructure design that would otherwise seem like magic.
Best mental model: Anycast is like a national helpline number — you dial one number, but your call is automatically connected to the nearest call centre. The number is the same everywhere; the destination varies by location.
CDN — moving content closer to users
A Content Delivery Network is a geographically distributed network of servers that caches and serves content from locations close to users — reducing latency, offloading traffic from origin servers, and providing resilience against traffic spikes.
CDNs are one of the highest-leverage architectural decisions available to most teams. A well-configured CDN can reduce page load times by hundreds of milliseconds for global users, absorb DDoS traffic before it reaches origin, and dramatically reduce egress costs. A poorly configured CDN can serve stale content, create cache poisoning vulnerabilities, and add complexity with no benefit.
Best mental model: a CDN is like a chain of local warehouses for an online retailer. Rather than shipping every order from one central depot, stock is pre-positioned in warehouses near customers. Most orders ship from the nearest warehouse — faster and cheaper — with the central depot only involved for items not stocked locally.
The running example: a URL shortener
From this pillar onwards, each concept is grounded in a concrete evolving system: a URL shortener — think bit.ly or tinyurl. It's simple enough to explain quickly, complex enough to stress-test every concept we'll cover.
Here's the starting point:
A user submits a long URL, gets back a short code (
sho.rt/x7Kp2)Anyone visiting the short URL is redirected to the original
The system needs to work globally, handle traffic spikes, and be fast
By the end of this pillar alone, this simple description will raise questions about every concept we cover: which DNS record type should the domain use? Should redirects be served from a CDN edge node or origin? How does TLS termination affect latency? We'll answer each one as we go.
The decision framework
When a networking problem lands on your desk, these eight concepts give you the layers to reason through:
Where in the stack is the failure?
└─ Physical/data link (layer 1-2)? → Infrastructure / hardware issue
└─ Network (layer 3)? → IP routing, firewall rules
└─ Transport (layer 4)? → TCP vs UDP, connection management
└─ Application (layer 7)? → HTTP, TLS, DNS
Is it a latency problem?
└─ DNS resolution slow? → TTL too low, resolver issues
└─ TLS handshake slow? → TLS version, session resumption
└─ Geographic distance? → CDN, Anycast, edge caching
Is it a traffic distribution problem?
└─ Single origin overwhelmed? → CDN, DNS load balancing
└─ Users in multiple regions? → Anycast, GeoDNS, multi-region
Is it a security problem?
└─ Data in transit exposed? → TLS configuration
└─ Certificate errors? → Cert validity, chain of trust, SNI
Common traps at this stage
Treating DNS as someone else's problem. DNS is the first thing that happens for every user request and one of the last things engineers understand deeply. TTL misconfiguration, missing health checks on DNS-load-balanced records, and resolver caching surprises are responsible for a disproportionate number of hard-to-diagnose production incidents.
Assuming HTTPS means secure. TLS encrypts the connection and authenticates the server. It says nothing about the security of the application running on top of it. A perfectly valid TLS certificate on a site that stores passwords in plaintext is still a security disaster.
CDN as a silver bullet. CDNs work brilliantly for static and cacheable content. They add complexity with minimal benefit for highly dynamic, personalised, or uncacheable responses. Cache invalidation on a CDN is one of the genuinely hard problems in production infrastructure — not because it's technically complex, but because cache inconsistency bugs are subtle and slow to surface.
Ignoring TCP connection overhead. Every TCP connection requires a handshake before data flows. On a high-latency connection, this overhead compounds with the TLS handshake and adds hundreds of milliseconds before your first byte. Connection pooling, keep-alive, and HTTP/2 multiplexing all exist because this cost is real and avoidable.
Key takeaways
The OSI model is the map — seven layers from physical signals to application protocols, each responsible for a different part of communication.
TCP vs UDP is the transport choice — reliability and ordering vs speed and simplicity.
HTTP and HTTPS are the application layer — the language your services speak, with TLS providing encryption and authentication for the secure version.
TLS is the mechanism behind HTTPS — handshake, certificate, cipher negotiation, and the latency cost of getting security right.
DNS turns names into addresses — and its failure modes are subtle, global in impact, and frequently underestimated.
DNS load balancing distributes traffic at name-resolution time — simple and scalable with meaningful limitations.
Anycast routes one IP address to the nearest physical location — the technique behind global low-latency infrastructure.
CDN moves content closer to users — the highest-leverage latency optimisation available to most teams, with real cache management complexity to match.
Up next
Part 1 → The OSI Model: The Map Every Engineer Needs
We start at the foundation of all networking — the seven-layer model that explains why HTTP, TCP, and IP coexist without stepping on each other, and what each layer is actually responsible for. Once the map is in your head, every other concept in this pillar has a home.
Part of the System Design series. Tags: #systemdesign #networking #distributedsystems #backend #softwarearchitecture #engineering
→ Next: The OSI Model: The Map Every Engineer Needs — The OSI model isn't just interview theory — it's the map that tells you exactly where in the stack a network problem...
→ Next: The OSI Model: The Map Every Engineer Needs — The OSI model isn't just interview theory — it's the map that tells you exactly where in the stack a network problem...
