Daily active users
30 M
our chosen baseline
How a service like bit.ly or tinyurl turns a long link into seven characters, hands billions of redirects per day, and keeps it all from being abused — pulled apart layer by layer.
Nail the scope before drawing boxes. A URL shortener sounds trivial — until you ask how many, how fast, and what happens when the input is hostile.
/launch) when it's still free.Pick numbers you can defend, then multiply. The point is the order of magnitude — does this fit on one box or do we need a fleet?
Read this aloud: a hundred thousand redirects per second, multi-petabyte over a decade. One Postgres won't do it; a shard plus an aggressive cache will.
Two endpoints carry the whole product. Keep the surface small — every extra knob is something to version, document, and defend.
| Endpoint | Purpose | Request | Response |
|---|---|---|---|
POST /api/v1/shorten |
Create a short link |
{ "long_url": "https://...", "alias": "launch", "expires_at": "2027-01-01" }
Bearer token if the user wants ownership / analytics; anonymous otherwise.
|
201 { "short_url": "https://sho.rt/aB3xK9q", "expires_at": "..." }
409 if the alias is taken, 400 on malformed URL, 422 on blocked target. |
GET /:short_code |
Redirect to long URL |
Path only. The client is a browser following a link.
No auth — short URLs are bearer tokens themselves; whoever holds the link follows it.
|
302 Found with Location: <long_url>
404 if unknown, 410 Gone if expired, 451 if blocked. |
The same long URL submitted by the same user returns the same short code — don't burn a new code per retry.
/api/v1/ lives under a separate hostname from the redirect domain so the short host stays tiny and fast.
POST /shorten is gated per IP and per API key. Redirects are not gated — they're just reads.
One main table does almost everything. Lean on the primary key for the only query that matters on the hot path: lookup by short code.
| Column | Type | Notes |
|---|---|---|
short_code | VARCHAR(10) | Primary key. 7 base62 chars in practice. |
long_url | TEXT | Up to ~2 KB; we don't normalise or rewrite. |
owner_id | BIGINT, nullable | FK to users. Null for anonymous links. |
created_at | TIMESTAMPTZ | Defaults to now(). Used for analytics & cleanup. |
expires_at | TIMESTAMPTZ, nullable | NULL means never. Indexed for the reaper job. |
is_blocked | BOOLEAN | Set by abuse scanner — short-circuits redirect. |
click_count | BIGINT | Async, eventually consistent. Real numbers in the analytics store. |
Sharded by hash of short_code. Secondary index on owner_id for a user's "my links" page. Analytics (per-click rows) live in a separate columnar store, never in this table.
Short codes are drawn from base62: digits 0-9, lowercase a-z, uppercase A-Z. Each character carries log₂(62) ≈ 5.95 bits.
n characters → 62ⁿ codes. We need at least one code per link we'll ever issue, plus a safety margin so collisions stay rare under random generation.
URL-safe without percent-encoding, case-sensitive (more bits per character than base36), no ambiguous punctuation. Some services strip 0/O and 1/l/I for human-readability and lose a little of that headroom.
Trillions of codes is overkill for one product's lifetime — and that's the point. We never want to run out, never want to grow the URL, never want users to retype it.
Six characters fits today's traffic but corners us in five years. Adding a character later means a flag day for every cached link.
Run the long URL through a cryptographic hash, truncate to 7 base62 characters, hope it's free. If it isn't, perturb and try again.
SHA-256 the URL, take the first ~42 bits, base62-encode them into 7 characters. Look it up. If the row is missing, insert. If it already maps to the same URL, return it. If it maps to a different URL — collision.
Append a salt (random suffix, user-id, attempt counter) to the input and hash again. Retry until you find a free slot. Loop bounded by a small max — practical collisions are exceedingly rare at 42 bits.
Stateless workers — any server can shorten without coordinating with peers. Re-shortening the same URL naturally returns the same code (idempotent without an extra index).
Every write does a read first to check uniqueness. At high write rates and a fuller key space, the retry rate creeps up. Codes look pseudo-random — fine for privacy, awkward for debugging.
Hand each new link a unique integer ID, then base62-encode it. No collisions, no retries, by construction.
A central ID service hands out monotonically increasing integers. The shortener encodes the integer in base62 (5–7 chars) and stores the row.
Zero collision logic. One write per shorten. Compact codes — early users get 1–2 character URLs (in theory). Easy to range-scan for batch operations.
The ID service is a single point of contention. Solve with sharded counters, ticket batches (each worker fetches 1 000 IDs at a time), or a distributed ID generator like Snowflake.
Sequential IDs reveal traffic volume and adjacency — competitors can scrape neighbouring codes. Mitigate by hashing the ID through a fixed bijection (e.g. XOR with a secret, multiply mod 2ⁿ) before base62-encoding.
Redirects follow a power law — a handful of viral links serve most of the traffic. Push those rows as close to the user as possible.
Frequently-hit short URLs cache the redirect response itself. With a 301 + far-future Cache-Control, the request never touches our servers. Effective for the long-lived viral tail.
Hash-partitioned by short_code. Stores code → long_url plus is_blocked. LRU eviction, ~24 h TTL — long enough to ride viral spikes, short enough to pick up edits and revocations.
Cache 404s for a minute or two. Stops scrapers probing random codes from torching the DB.
When a link is blocked or edited, push a delete to Redis and a purge to the CDN. The TTL is the safety net if either delivery fails.
Both redirect the browser. The difference is who remembers — and that decides whether you can ever see another click.
The textbook answer is "302 for shorteners." The honest answer is "302 unless you've actually measured server load and decided you'd rather lose analytics than scale the redirect tier."
A shortener is a redirection oracle for the open web. Phishing kits, malware drops, and scam funnels all want to hide behind your domain. Defence is layered, never single-shot.
On POST /shorten, check the target against Google Safe Browsing, PhishTank, internal blocklists. Reject outright if it's a known bad. Async re-scan in the background as feeds update — yesterday's clean URL can be today's compromised host.
Token bucket per IP, per API key, per user — for /shorten only. A residential IP that submits 10 000 URLs in a minute is automation. Redirect endpoint stays unrated; throttling reads punishes legitimate viral content.
Newly-registered domains, free hosting providers, and IDN-homoglyph URLs (Cyrillic 'а' for Latin 'a') get extra scrutiny — sometimes an interstitial warning page before forwarding.
Every short URL has a ?report path. Three independent reports trip a soft-block (interstitial); a human moderator confirms hard-block. Audit log captures who set is_blocked and why.
When confidence is medium ("looks suspicious but not confirmed"), show a "this link may be unsafe" page that requires a click-through. Buys time and protects the cautious.
The product is simple. The interesting choices are about traffic shape, identifier strategy, and where to draw the line on trust.
At 100:1 read-to-write, every microsecond on the redirect dominates total cost. Design the lookup first; the writer can be slower if it has to be.
Hashing buys you stateless workers and pays in collision retries. Counters give you guaranteed uniqueness and pay in a contended ID service. Pick the failure mode you'd rather operate.
CDN catches the viral tail, Redis catches the warm body, the DB sees only the cold misses. Each layer needs its own TTL and its own invalidation story.
Temporary redirects keep analytics honest and let you revoke abusive links. The "load saving" of a 301 is hypothetical until your dashboards prove otherwise.
Scan on create, scan again in background, rate-limit creators, take user reports, interstitial the borderline. No single layer catches everything.
Anyone holding the URL can follow it. Don't lean on obscurity; assume scrapers exist; randomise enough that adjacent codes don't reveal each other.