Availability Math

The table you should know cold

The jump from 99.9% to 99.99% is where cost explodes, because 4 minutes a month is less than the time it takes a human to read a page, open a laptop, and understand what's happening. Below about 99.95%, humans can be in the recovery path. Above it, recovery must be automatic. That's the real architectural boundary, not the digits.

Components in series multiply

If a request must pass through several components, and any one failing fails the request, availabilities multiply:

A_total = A₁ × A₂ × A₃ × …

Five services at 99.9% each:

0.999^5 = 0.995  →  99.5%,  which is 43 hours of downtime per year

Five very reliable components produce a distinctly unreliable system. This is the single most important piece of arithmetic in the lesson, and it has a blunt consequence:

Redundancy in parallel — and its lie

If two components are redundant and either can serve, the system fails only if both fail:

A_total = 1 − (1 − A₁) × (1 − A₂)

Two 99% replicas give 1 − 0.01² = 99.99%. Three give 99.9999%. Redundancy looks like magic, and this is where designs go wrong, because the formula assumes independent failures.

They are almost never independent:

• Both replicas are in the same availability zone, and the zone goes down.
• Both received the same bad config push.
• Both run the same code, and the same poisoned input crashes both.
• Both depend on the same database, the same DNS, the same certificate authority.
• The failover mechanism itself is broken — a very common discovery, made at the worst time.

A more honest model splits failures into independent and correlated:

A ≈ 1 − [ (1−A_ind)ⁿ + (1−A_corr) ]

The correlated term does not shrink with n. Once your independent failures are rare, adding a fourth replica changes nothing — the remaining risk is entirely in the shared parts. That's why serious designs spend their effort on decorrelation: separate AZs, staged config rollouts, different instance types, canary deploys.

MTBF, MTTR, and where to spend

Availability = MTBF / (MTBF + MTTR)

MTBF is mean time between failures; MTTR is mean time to recovery. There are two ways to raise availability, and they are not equally priced.

Doubling MTBF means making the system fail half as often — hard, open-ended, and it fights every change you ship. Halving MTTR means detecting and recovering twice as fast — bounded, concrete, and achievable with tooling you already understand: better alerting, fast rollback, feature flags, automated failover, good runbooks.

Beating the serial math

You can't remove dependencies, but you can stop them being critical:

Make them asynchronous. If the recommendation service is a queue write rather than a blocking call, its downtime doesn't fail the request.

Degrade gracefully. Recommendations unavailable → show popular items. Avatar service down → show initials. Each fallback converts a multiplied failure into a cosmetic one.

Cache the dependency. A stale response beats no response. stale-if-error at the edge and a last-known-good cache in the service both do this.

Set aggressive timeouts and shed load. A dependency that is slow is worse than one that is down, because slowness consumes your threads and spreads the failure back to you. Fail fast, then fall back.

Bulkhead. Separate connection pools and thread pools per dependency, so one saturated downstream cannot exhaust the resources of unrelated paths.

Applied together, five 99.9% dependencies with graceful degradation can yield a service more available than any of them — because the request no longer requires all five to succeed.

Availability is not durability

Two different promises, often conflated:

• Availability — can I reach the data right now?
• Durability — will the data still exist later?

S3-class storage advertises eleven nines of durability and about four of availability. Your data is essentially never lost, but you may be briefly unable to read it. A cache is the opposite: highly available and not durable at all.

Say which one you mean. An interviewer who asks "how do you make this durable?" is not asking about load balancers.

SLI, SLO, SLA

• SLI — the measurement. "Proportion of requests served in under 300 ms."
• SLO — your internal target. "99.9% of requests under 300 ms, monthly."
• SLA — the contractual promise, with penalties. Always looser than the SLO, because you want to breach your own target long before you breach a contract.

The error budget falls out of the SLO: 99.9% monthly allows 43 minutes of failure. That budget is a resource to spend deliberately — on risky deploys, migrations, load tests. When it's exhausted, you stop shipping features and fix reliability. It converts an argument about feelings into an argument about a number.

What to take away

• Convert nines to minutes before agreeing to them. 99.99% means recovery cannot involve a human reading a page.
• Serial dependencies multiply. Five 99.9% services in a chain give 99.5%.
• Parallel redundancy assumes independence, and independence is usually false. Spend on decorrelation, not on more replicas.
• A = MTBF/(MTBF+MTTR). MTTR is the cheaper lever, and it's why fast rollback matters.
• Async calls, graceful degradation, timeouts, and bulkheads break the serial multiplication.
• Availability ≠ durability. Error budgets turn reliability into a number you can spend.