Networking for System Design

The life of a request

Typing a URL and getting HTML involves, in order:

1. DNS resolution — hostname → IP. Cached at the browser, OS, and resolver; a cold lookup can take 20–120 ms and may itself be several queries (root → TLD → authoritative).
2. TCP handshake — SYN, SYN-ACK, ACK. One round trip before any data.
3. TLS handshake — TLS 1.3 needs one round trip; TLS 1.2 needed two. Session resumption can make it zero.
4. The HTTP request — one more round trip to first byte.

So a cold connection to a far-away server costs roughly 3–4 RTTs before your application code runs. At 8 ms (Zurich→Frankfurt) that's ~32 ms and nobody notices. At 160 ms (Zurich→Singapore) it's 640 ms and everybody does.

DNS, and what it's really for

DNS is a lookup, but in system design it's also a routing layer:

• TTL controls how long resolvers cache an answer. Low TTL (30–60 s) gives fast failover but more lookups; high TTL (hours) is efficient but means a bad record lingers. Note that many resolvers and client libraries ignore short TTLs, so DNS failover is best-effort — never your only failover mechanism.
• GeoDNS returns different IPs by client location, sending users to a nearby region.
• Anycast advertises the same IP from many locations and lets BGP route to the nearest. This is how CDNs and public resolvers work, and it fails over faster than DNS because it's the network doing the routing.

TCP, UDP, and QUIC

TCP's slow start matters more than people expect: a new connection doesn't begin at full speed, it ramps. For short responses the connection is finished before it reaches full throughput — another reason connection reuse beats connection setup.

UDP gives you nothing but datagrams, which is exactly right when a late packet is worthless: a 200 ms-old voice sample, a superseded player position, a metrics sample you'll re-send in a second anyway.

QUIC is TCP's guarantees rebuilt on UDP in userspace, which lets it fix two things TCP can't: it merges transport and crypto setup into one round trip, and it removes connection-wide head-of-line blocking. It also survives a network change — switching from Wi-Fi to cellular keeps the connection, because QUIC identifies connections by an ID rather than by the IP/port four-tuple.

HTTP versions, and what actually changed

HTTP/1.1 — one request at a time per connection. Browsers work around this by opening ~6 connections per host, which is why "domain sharding" used to be a performance trick. A slow response blocks everything behind it on that connection.

HTTP/2 — multiplexes many streams over one TCP connection, adds header compression and server push (since largely abandoned). It removes HTTP-level head-of-line blocking, but not TCP-level: one lost packet stalls every stream, because TCP must deliver bytes in order. On a lossy mobile network, HTTP/2 can be worse than HTTP/1.1.

HTTP/3 — HTTP/2's semantics over QUIC. Because each stream is independently ordered, a lost packet stalls only its own stream. Plus 0-RTT resumption for repeat visitors.

Choosing a communication style

REST over HTTP — the default. Cacheable, debuggable, universally supported. Weak spots: over- and under-fetching, and no built-in streaming.

gRPC — HTTP/2 plus Protocol Buffers. Binary and compact, generates typed clients from a schema, supports bidirectional streaming. Excellent for service-to-service; awkward from browsers (needs grpc-web and a proxy) and harder to inspect with ordinary tools.

GraphQL — the client specifies the shape of the response. Solves over-fetching for diverse clients, at the cost of hard-to-cache queries, and a server that can be asked to do something ruinously expensive unless you add query depth and cost limits.

For server→client push, three options and one obsolete one:

SSE is underrated. It's plain HTTP, it reconnects automatically, and it works through proxies. If the client only needs to receive, it's simpler than a WebSocket in every way.

Practical latency reductions

Ordered by how much they typically buy you:

1. Move the endpoint closer — CDN or edge PoP. Turns 160 ms RTTs into 10 ms ones. Nothing else comes close.
2. Reuse connections — keep-alive, HTTP/2 multiplexing, connection pools between services. Eliminates handshakes entirely.
3. Cut round trips — TLS 1.3, 0-RTT resumption, avoiding redirect chains. Each redirect is a full extra round trip, and they're easy to accumulate accidentally.
4. Compress — Brotli for text, and correctly-sized modern image formats. Bandwidth is cheap but mobile radios are not.
5. Parallelise dependent calls — if a request fans out to five services sequentially, you pay five RTTs. Fan out concurrently and you pay one, plus the slowest.

That last point is where server-side latency usually hides. A handler making six sequential internal calls at 3 ms each is 18 ms of nothing but waiting.

What to take away

• A cold request costs 3–4 RTTs before your code runs. Count round trips, not milliseconds.
• Latency is bounded by physics; the only real fixes are proximity and fewer round trips.
• HTTP/2 removed HTTP head-of-line blocking; only HTTP/3 removes TCP's.
• SSE for one-way push, WebSockets when genuinely bidirectional — and WebSockets make your servers stateful.
• Sequential internal calls are the most common source of self-inflicted server latency.