Chapter 4 · Hinglish · Sunne-wala Sabak
Rate Limiter — Buckets aur Windows
Aapka pehla real system design — ek rate limiter. Niche "Play all" dabaiye aur browser pura chapter padh kar sunayega. Kisi bhi section par "Suniye" se wahin se shuru karein.
▶ Companion slides kholेंTip: jo voice sabse natural lage wahi chuniye — ek Hindi (hi-IN) ya Indian-English (en-IN) voice aam taur par best chalti hai.
Rate limit kyun Slide 2
Chapter chaar, design a rate limiter. Rate limiter ek chhota lekin tej tool hai jo har request par decide karta hai ki use aage jaane do ya rok do. Iske chaar kaam hain. Ek, abuse rokna, jaise credential stuffing, scraping, ya brute force. Do, cost cap karna, kyunki pay per use APIs mein ek atki hui retry loop seedha bill badha deti hai. Teen, fairness, yaani ek tenant baaki sabki capacity na kha jaaye. Aur chaar, stability, yaani system ko uske safe envelope ke andar rakhna. Har ek kaam akele bhi limiter ko justify karta hai.
Limiter kahan rehta hai Slide 3
Ab dekho limiter kahan baithta hai. Teen jagah ho sakti hain. Pehli, client side, yaani SDK pehle se throttle karta hai. Yeh polite hai par untrusted, kyunki ek hostile client ise hata sakta hai. Doosri, edge ya API gateway, jo primary control point hai. Yeh har request ko dekhta hai compute kharch hone se pehle, aur yahi ek jagah hai jahan ek tenant ka traffic saari services ke aar paar dikhta hai. Teesri, service side, jo last line of defense hai aur ek specific resource ko bachata hai, jaise ek paid API ya database. Best practice hai inhe layer karna, ek ke upar ek.
Token bucket Slide 4
Ab pehla algorithm, token bucket. Socho ek bucket hai jo zyada se zyada capacity B tokens rakh sakta hai, aur yeh r tokens per second ki rate se bharta rehta hai. Har request ek token nikalti hai. Agar bucket khaali hai to request reject ho jaati hai. State sirf do cheezein hain, kitne tokens hain aur last refill ka time. Iska sabse bada faayda yeh hai ki bursts first class hain. Ek idle client dheere dheere poore B tokens jama kar leta hai, to quiet phir achanak burst wala pattern theek se handle hota hai, aur long run average phir bhi r par bandha rehta hai. Trade off yeh hai ki agar B bada hai to ek starved client milliseconds mein saara credit kharch kar sakta hai.
Leaky bucket Slide 5
Doosra, leaky bucket. Yahan requests ek fixed size FIFO queue mein aati hain aur ek constant rate L par bahar nikalti hain. Jo extra aati hain woh drop ho jaati hain. Token bucket credit ko shape karta hai, jabki leaky bucket output ko shape karta hai, ekdum smooth. Iska faayda yeh hai ki downstream ko hamesha constant arrival rate milti hai, chahe upstream kitna bhi spiky ho. Yeh tab perfect hai jab aap kisi aisi cheez ko bacha rahe ho jo bursts se nafrat karti hai, jaise ek database with fixed connection pool. Trade off yeh hai ki backlog ke dauraan har accepted request par latency badhti hai, aur bade bursts chup chaap kho jaate hain.
Fixed window counter Slide 6
Teesra, fixed window counter. Time ko fixed slots mein baant do, maan lo ek minute, aur har slot per key ek integer rakho. Har request par counter ko ek se badhao, aur jab count limit se zyada ho to reject karo. Banana bilkul trivial hai, ek single increment Redis mein kaafi hai. Lekin iski badi kamzori hai boundary par. Maan lo limit hai das per minute. Client baarah baj kar zero minute unsaath second par das requests maar de, aur phir baarah baj kar ek minute par phir das. Yeh do second mein bees requests ho gayin. Letter of the limit toot-ta nahi, lekin spirit toot jaata hai. Isliye fixed window ko kabhi apni akeli line of defense mat banao.
Sliding window log Slide 7
Chautha, sliding window log. Yahan aap last W seconds ki har request ka timestamp ek sorted set mein rakhte ho. Jab nayi request aaye, pehle un entries ko hatao jo now minus W se purani hain, baaki ko count karo, aur agar count limit ke neeche hai to allow karo. Iska faayda yeh hai ki yeh perfectly accurate hai, koi boundary artifact nahi, kyunki window sach mein slide karti hai. Lekin trade off bhaari hai. Memory traffic ke saath badhti hai. Ek million RPS client ko ek minute window ke liye har waqt saath karod timestamps store karne padenge. Isliye ise sirf tab use karo jab accuracy waqai matter karti ho, jaise billing ya security sensitive flows.
Sliding window counter Slide 8
Paanchva, sliding window counter, jo high volume APIs ka workhorse hai. Sirf do fixed window counters rakho, current aur previous, aur interpolate karo. Pehle overlap nikaalo, yaani aap current slot mein kitne aage ho. Phir estimate banao, jo hai previous count guna one minus overlap plus current count. Yaani purane slot ka contribution linearly ghat kar zero ho jaata hai. Isse storage aur CPU fixed window jitne hi rehte hain, lekin boundary doubling attack khatam ho jaata hai. Trade off yeh hai ki yeh approximate hai, exact nahi, kyunki yeh maanta hai ki previous window mein traffic uniform tha. Real traffic uniform nahi hota, par error aam taur par sirf kuchh percent rehta hai. Yeh tradeoff lagbhag hamesha worth it hai.
Sahi algorithm chuno Slide 9
To paanchon algorithms ka short recap. Token bucket general purpose API throttling ke liye achha hai aur bursts ko allow karta hai. Leaky bucket fragile downstream ke liye, jab output smooth chahiye. Fixed window sabse sasta hai par boundary par edge effect deta hai, isliye sirf coarse internal limits ke liye. Sliding log exact hai par bhaari, isliye billing aur security critical kaam ke liye. Aur sliding counter, jo do integers mein lagbhag pachanbe percent accuracy deta hai, high volume public APIs ka default hai. Rule of thumb yaad rakho, hamesha sliding window counter se shuru karo, aur sliding log tab utha-o jab koi audit ya invoice exactness par depend karta ho.
Ek limiter, kai machines Slide 10
Ab distributed case. Ek akela edge node sirf apna traffic ka hissa dekhta hai. Ek global limit enforce karne ke liye aapko shared state chahiye, aam taur par Redis. Lekin naive version toota hua hai. Naive limiter pehle counter ko GET karta hai, limit se compare karta hai, aur agar allowed ho to INCR karta hai. Problem yeh hai, GET aur INCR ke beech mein doosra node bhi yahi kar sakta hai, dono sochte hain ki woh under the limit the, aur dono request ko aage jaane dete hain. Fix hai decision ko atomic banao. Poora check aur increment ek single Redis Lua script mein daalo, jise server bina interleaving ke chalata hai. Aur operationally, ek key ke saare writes ek shard par pin karo, hot keys ke liye decision ko kuchh milliseconds locally cache karo, aur pehle hi decide karo ki outage mein fail open karna hai ya fail closed.
Client ko sach batao Slide 11
Ab wire protocol. Ek throttled client ko bina guesswork ke gracefully recover karna chahiye. Status code chaar sau untees bhejo, yaani too many requests, na ki paanch sau teen. Chaar sau untees ka matlab hai tum bahut zyada bhej rahe ho, jabki paanch sau teen ka matlab hai server unhealthy hai. Inhe mix karna retry logic ko confuse karta hai. Saath mein ek Retry-After header bhejo, jo bataye ki kitne seconds baad wapas aana hai. Aur sabse achhi practice, X-RateLimit headers, yaani Limit, Remaining aur Reset, har response par bhejo, sirf denials par nahi. Isse clients deewar se takraane se pehle khud ko throttle kar lete hain, aur aapka support load kam ho jaata hai.
Monitoring aur abuse Slide 12
Ab observability. Ek limiter bina instrumentation ke tab tak invisible rehta hai jab tak woh toot na jaaye. Graph karo throttled requests per second, allow deny ratio per route, limiter ki apni added latency, aur top keys by denial rate. Phir abuse ke signals dekho. Jab kuchh hi keys zyadatar denials produce karein, jab requests perfectly even spaced ho, jo ek bot ki nishaani hai insaan ki nahi, jab ek hi subnet se achanak cross key correlation aaye, ya jab login aur password reset endpoints par denial spike ho. Phir loop close karo, jaise repeat violations par limit auto tighten karna, chronic offenders ko block list mein daalna, aur sirf systemic spikes par page karna, single key par nahi.
Edge cases Slide 13
Do edge cases jo production mein kaatte hain. Pehla, clock skew. Agar har node apni local clock se timestamps likhe, to do nodes jo kuchh sau milliseconds alag hain woh disagree karenge ki window ke andar kya aata hai. Fix hai store ki clock use karo, yaani Redis ka TIME command Lua script ke andar se pass karo, ek key ke writes ek shard par pin karo, aur NTP drift par alert lagao. Doosra, thundering herd at reset. Jab ek fixed window reset hota hai, har throttled client theek us hi instant par retry karta hai, jisse ek synchronized spike banta hai jo kabhi unthrottled traffic se bhi bada ho sakta hai. Fix hai Retry-After ko jitter karo, sliding windows ko prefer karo kyunki unmein global reset moment hi nahi hota, aur hard limit se pehle soft shed karna shuru karo.
Chhah principles Slide 14
To chhah principles yaad rakho. Ek, default mein sliding window counter uthao. Do, har decision ko atomic banao, kyunki distributed system mein check then act limiter ek toota hua limiter hai. Teen, apni limits ko layer karo, edge par coarse, service par fine, SDK mein advisory. Chaar, wire par saaf bolo, yaani chaar sau untees, Retry-After aur X-RateLimit headers. Paanch, jo bhi reset hota hai use jitter karo, kyunki synchronized clients synchronized spikes banate hain. Aur chhah, denial ko sirf ek outcome nahi, ek signal samjho, kyunki per key denials ka graph aapki sabse pehli abuse detection feed hai.
Aage kya
Bas, yahi hai chapter chaar. Ek kaam karo, do endpoints lekar algorithm choose karne ki practice karo, ek login endpoint aur ek public read API. Login ke liye chhoti limit, abuse sensitive, aur boundary par doubling nahi chahiye. Public read ke liye high volume, thoda burst tolerant, aur sasta storage. Har choice ko accuracy, burst tolerance aur memory se justify karo. Agle chapter mein hum consistent hashing dekhenge.