Case Study — Netflix Open Connect

The economics of not renting

Netflix began Open Connect in 2011, for two published reasons: it had grown into a significant fraction of the traffic on consumer ISP networks, and a delivery system built for one workload could be made far more efficient than a general-purpose, demand-driven one.

A commercial CDN sells capacity per gigabyte delivered, because it is amortising a network across thousands of customers with wildly varied traffic. Netflix's workload is the opposite: enormous byte volume, tiny request volume, one catalogue, a nightly peak you could set a watch by. Renting generic capacity for that is paying for flexibility you do not use.

Open Connect deploys appliances (OCAs) two ways:

At internet exchange points, peering settlement-free with any mutually-present ISP.
Embedded inside the ISP, free of charge to qualifying partners. Netflix supplies the hardware; the ISP supplies rack space, power and connectivity, and controls which of its customers route to it.

The embedded case is the structurally unusual one. The ISP's largest single slice of evening traffic stops crossing paid transit and is served from inside its own network. Netflix's cost becomes hardware plus operations rather than a per-gigabyte bill, and the path to the viewer gets shorter.

	Commercial CDN	Open Connect
Cost shape	Per GB delivered	Capital + operations
Cache fill	On demand, on miss	Pushed ahead of demand
Catalogue	Unbounded, many tenants	One, small, immutable
Placement	Provider's PoPs	Inside the ISP, or at an IX
Who pays for the box	You, indirectly	Netflix, directly

All of it rests on a property most companies do not have: a small, immutable, highly concentrated catalogue.

Video is an unnaturally good cache object

Everything that ruins hit rate in CDN & the edge is structurally absent here.

The files are immutable. An encoded episode never changes. There is no invalidation problem, no purge propagation delay, no surrogate keys — the entire invalidation apparatus of a normal CDN is simply not needed. New content gets a new file.

They are pre-encoded, not generated. Each title becomes a ladder of resolution/bitrate pairs (Netflix tunes that ladder per title rather than using one fixed ladder), across several codecs, plus audio and subtitle files. That multiplies the object count per title — but every one of those objects is byte-identical for every viewer who requests it.

There is no personalisation in the bytes. The cookie that destroyed the hit rate in the CDN lesson has no analogue: the cache key is effectively the file name. Personalisation lives entirely in which file the client is told to fetch, never in its contents.

One specialised workload also means you can tune the whole stack for it. OCAs run FreeBSD with NGINX, serving via sendfile with kernel TLS so encrypted video never leaves the kernel on its way to the NIC. Netflix engineers publicly demonstrated 400 Gb/s of TLS video from a single server in 2021 — a number only reachable because the box does exactly one thing.

The cache that is never cold

Here is the part worth stealing conceptually even if you never touch a CDN.

A normal CDN is demand-driven. The first request for an object misses, goes to a shield, goes to origin. Caching spends a lot of effort on the consequences: cold start, stampedes, thundering herds, request coalescing.

Open Connect is push-filled. Control-plane services predict regional popularity, decide what each appliance should hold, and copy it there during configurable off-peak fill windows — every night, before the demand exists. Appliances can also fill from each other, keeping update traffic off long-haul links. Netflix's own description of the benefit is that a proactive, directed cache reduces demand on upstream network capacity by several orders of magnitude versus a demand-driven one.

Note what replaces the miss path. Eviction stops being a reactive LRU decision made at request time and becomes a planning decision made nightly: what should be resident on this box, given this region's predicted viewing and this much disk. Misses still exist — an appliance is finite — but the client is simply steered to a larger or more distant appliance that holds the file. A miss is a routing outcome, not a latency cliff.

If your demand is user-generated, long-tailed and unpredictable, none of this transfers. That contrast is the point of teaching it.

Control plane in AWS, data plane on the appliances

Netflix runs its application in AWS, having completed that migration in early 2016. Open Connect is not in AWS. The split is clean and documented:

OCAs report health, the BGP routes they have learned, and which files they hold to a cache control service in AWS.
A steering service uses that data to pick appliances for a given client.
Playback application services check authorisation and licensing, decide which files this device and this network need, and hand the client a set of OCA URLs.
The client then fetches bytes directly from the OCA. AWS is not in the byte path.
OCAs hold no member data — no viewing history, no DRM material, no account state.

	Control plane (AWS)	Data plane (Open Connect)
Traffic	Small requests, high fan-out	Enormous sequential byte streams
State	Per-member, mutable	Immutable files, no user state
Deploys	Many per day	Slow, physical, in others' racks
Failure impact	New play-starts fail	Bytes stop
Scaling unit	Instances and regions	Appliances per ISP

Two systems with almost nothing in common except that they cooperate, and building them separately is the correct call rather than an accident of history. It also buys a resilience property: a stream already playing does not depend on the control plane at all. Control-plane trouble degrades starting something new while bytes in flight keep flowing — and the multi-region failover exercises that move traffic between AWS regions do not move the video.

Degradation changes the arithmetic

Playback authorisation is a hard dependency. Recommendation ranking, personalised artwork and row ordering are not. When those fail, the sane behaviour is a generic row, a default image, a slightly worse home screen — and playback that still works.

This is the availability math argument made concrete. Every service you move out of the critical path leaves the serial product entirely; five 99.9% dependencies multiply to roughly 99.5%, and making three optional cuts downtime several-fold with no new hardware. Netflix's stated purpose for its microservice-level chaos experiments is precisely to build confidence that a service degrades gracefully when non-critical downstream services fail — an admission that degradation is a claim, and claims decay unless tested.

Injecting failure on purpose

Chaos Monkey (2011) terminated production instances at random, on the theory that if instance death is routine, nobody ships a service that cannot survive it. The Simian Army added siblings — latency injection, availability-zone failure — and Chaos Kong exercised full regional evacuation.

The trajectory afterwards matters more than the origin story. Failure Injection Testing (2014) moved from killing machines to injecting request-level faults into chosen components for chosen user cohorts. ChAP runs experiments on live traffic against a small canary, watches the metrics automatically, and aborts on deviation. Blast radius shrank as the tooling matured; the random-instance-killer is the least sophisticated thing in the family, not the state of the art.

The reasoning is worth repeating: untested failover is a hypothesis. The fallback that throws on its first real invocation, the retry storm nobody modelled — these get found in production or at 03:00, and production during office hours is the better option.

What transfers and what does not

Almost nobody should build a CDN. The appliance programme requires traffic volume that gives you standing with ISPs, a catalogue small enough to fit on a box, and content you own.

What does transfer: split the control plane from the data plane when their requirements diverge; pre-position rather than fill on demand wherever demand is predictable — nightly report builds, model artefacts, game assets before a launch; make peripheral dependencies optional and measure the fallbacks; and test failure deliberately, small first.

Worth saying plainly: per-ISP offload ratios, the popularity algorithm and Netflix's delivery cost per gigabyte are not public. Quoting a figure for any of them is a good way to be wrong out loud.

In interviews

"How would you design Netflix?" — lead with the control-plane/data-plane split. Almost every candidate designs one system; naming two with different requirements is the signal.
"How do you avoid a thundering herd on a cold cache?" — coalescing and shielding first, then offer pre-positioning as the stronger answer when demand is predictable.
"Why not just use a commercial CDN?" — one catalogue of immutable files at extreme volume makes specialised hardware inside the ISP beat rented generic capacity. Add that the reasoning does not hold for most products.
"Would you run Chaos Monkey here?" — the senior answer is usually not yet: name the preconditions, then propose a scoped game day instead.
"What if the recommendation service is down?" — playback continues, the home screen degrades, and the dependency leaves the availability product. Say the arithmetic.

What to take away

Open Connect works because the catalogue is small, immutable and owned — not because owning a CDN is generally smart.
Video is a near-perfect cache object: no invalidation, no personalisation in the bytes, perfect sharing.
Off-peak pre-positioning removes the cold-cache and thundering-herd problems rather than mitigating them.
Pre-positioning needs predictable demand; user-generated long-tail workloads cannot use it.
Control plane in AWS, data plane on appliances — different requirements, correctly different systems.
Chaos engineering is only responsible with observability, automatic aborts and a contained blast radius.

Check yourself

Why can Open Connect pre-position content while a general-purpose commercial CDN cannot?
Pre-positioning is a prediction problem before it is a storage problem. Netflix owns the catalogue, schedules its own launches and sees a strong daily viewing cycle, so it can decide tonight what each appliance should hold tomorrow. A CDN serving thousands of tenants has no basis for that prediction and must fill on demand.
A client presses play. Which component serves the actual video bytes?
The AWS control plane authorises playback, decides which files are needed and hands back OCA URLs. The client then fetches bytes straight from the appliance, so AWS is entirely out of the data path. That separation is why a stream already playing does not depend on the control plane.
What is the strongest argument that video files are unusually well suited to caching?
Immutability removes the invalidation problem entirely — changed content simply becomes a new file — and the absence of per-user bytes means the cache key is effectively the file name. The cookie and Vary problems that wreck hit rates on ordinary sites have no analogue here.
A team wants to adopt chaos engineering after reading about Chaos Monkey. What should they establish first?
Netflix's own tooling moved from random instance kills towards precisely targeted, canary-scoped experiments with automated analysis and abort. Without a measurable hypothesis, fast observability and a way to stop, injecting failure into production is just an outage you caused and cannot see.