Chapter 11 · Hinglish · Sunne-wala Sabak
News Feed System — Push, Pull, aur Hybrid
Ek social feed kaise banti, store hoti, aur serve hoti hai. Niche "Play all" dabaiye aur browser pura chapter padh kar sunayega. Kisi bhi section par "Suniye" se wahin se shuru karein.
▶ Companion slides kholiyeTip: jo voice sabse natural lage wahi chuniye — ek Hindi (hi-IN) ya Indian-English (en-IN) voice aam taur par best chalti hai.
News feed kya hai Slide 2
Chalo chapter gyaarah shuru karte hain, ek news feed system design karna. Dekhne mein yeh ek simple list lagti hai, par andar se yeh ek merge problem hai. Har viewer ke liye aapko un sabhi accounts ke recent posts ikatthe karne hain jinhe woh follow karta hai, unhe order karna hai, aur ek page lautana hai, bhi kuch sau milliseconds ke andar. Functional cheezein choti hain, post karo, follow karo, feed padho. Asli mushkil non-functional hai, latency, social graph ki shape, aur ek read-to-write ratio jo bahut hi lopsided hai. Log post kam karte hain, scroll bahut zyada, to reads writes se sau guna ya usse bhi zyada hote hain. Aur ek long tail hai, kuch accounts ke paas millions of followers hote hain, jo har naive design ko tod deta hai.
Do halves — write path aur read path Slide 3
Poora system ek hi sawaal par tika hai. Feed ka merge aap tab karte ho jab koi post likhta hai, ya tab jab koi feed padhta hai. Isse do halves ban jaate hain. Write path tab chalta hai jab koi post karta hai. Yahan aap post ko validate aur store karte ho, aur decide karte ho ki yeh post kis kis ki feed mein jaani chahiye, aur kya use abhi pre-compute karna hai. Read path tab chalta hai jab koi app kholta hai. Yahan aap pehle se bani feed uthate ho, jo missing hai use merge karte ho, phir rank aur paginate karke lautate ho. Merge ka kaam in dono halves mein se kisi ek mein hi rehna hai, aur saari tradeoff isi baat par hai ki yeh cost kaun sa half uthata hai.
Fanout on write — push model Slide 4
Pehla approach, fanout on write, yaani push model. Yahan merge eagerly hota hai. Jaise hi koi post banti hai, ek fanout worker author ke saare followers dekhta hai, aur us post ki ID har follower ki personal feed cache mein daal deta hai, jise inbox keh lo. Jab koi follower baad mein app kholta hai, uski feed pehle se ready hoti hai, aur read sirf ek cache lookup ban jaata hai. Ek write se N inserts hote hain, par read ki cost lagbhag zero ho jaati hai. Push tab achha hai jab reads writes se bahut zyada hon aur zyadatar accounts chote hon.
Fanout on read — pull model Slide 5
Doosra approach, fanout on read, yaani pull model. Yahan merge lazily hota hai. Jab koi post karta hai, kisi aur ki feed ko kuch nahi hota, bas ek insert hota hai. Kaam tab tak rukta hai jab tak koi viewer app na khole. Tab feed service dekhti hai ki viewer kise follow karta hai, un sabhi accounts ke latest posts query karti hai, aur unhe on the fly merge karti hai, ek k-way sort se, time ya score ke hisaab se. Writes bahut saste, par reads saara kaam karte hain. Pull tab achha hai jab kuch accounts bahut bade hon aur zyadatar users kam padhte hon.
Push aur pull ki alag cost Slide 6
Dono mein se koi outright winner nahi hai, dono ek hi cost ko ulte chhor par le jaate hain. Push fast reads deta hai par writes mehnge aur storage zyada. Pull writes saste karta hai par reads dheere aur kaam se bhare. Sabse bada issue celebrity problem hai. Jab kisi account ke chaar crore followers hon aur woh ek post kare, to pure push ko ek hi action ke liye chaar crore inserts karne padte hain. Yeh write amplification queue ko atka deta hai aur baaki sabki fanout late kar deta hai. Isi ek skew ki wajah se koi bhi bada system akela push use nahi karta.
Hybrid — many ke liye push, famous ke liye pull Slide 7
Isi liye real systems dono ko mix karte hain. Aam accounts, jinke followers kam hote hain, push use karte hain, unke posts pehle se inboxes mein pahunch jaate hain. Ek chota set high-fanout accounts ka hota hai, jinke followers kisi threshold se upar hain. Inhe fanout se chhoot mil jaati hai, inke posts sirf store hote hain, aur read time par pull karke viewer ki pehle se bani inbox mein merge kar diye jaate hain. Feed ka zyadatar hissa pehle se compute hota hai, sirf kuch viral authors ko demand par jodte hain. Isse write amplification cap ho jaati hai, theek unhi accounts ke liye jinki fanout warna ruinous hoti.
System ke pieces Slide 8
Ab pieces. Kuch chote services hain, har ek ka ek hi kaam, jo queues aur caches se jude hote hain taaki slow path fast path ko block na kare. Post service posts ko accept, validate, aur store karti hai, aur ek post-created event nikalti hai, use followers ki parwaah nahi. Fanout service us event ko queue se uthati hai, author ke followers load karti hai, aur hybrid rule ke hisaab se decide karti hai ki push kare ya pull ke liye chhode. Feed cache ek fast key-value store mein har user ki recent post IDs ki list hai, read path sabse pehle isi ko chhuti hai. Aur news feed service reads handle karti hai, cache ki list uthati hai, pull sources merge karti hai, IDs ko full posts mein hydrate karti hai, rank aur paginate karke page lautati hai.
Feed cache mein kya hota hai Slide 9
Ek important baat, cache mein posts nahi hoti, sirf post IDs hoti hain. Yeh ek bounded, ordered list hai, newest first, kisi N par capped, maan lo aakhri ek hazaar. Post ke bodies, author info, aur attachments kahin aur rehte hain, aur unhe tabhi hydrate kiya jaata hai jab page actually padha jaata hai. Sirf IDs store karne se har user ki entry choti rehti hai, aur yahi cheez millions of users ke liye personalized list ko memory mein rakhna affordable banati hai. Ek quick estimate se pata chalta hai ki poori cache aaram se memory mein aa jaati hai, aur yahi wajah hai ki push practical hai.
Cursor se paginate karo, offset se nahi Slide 10
Feed ke top par lagatar naye posts add hote rehte hain, aur yahi offset pagination ko chupke se tod deta hai. Agar page ek aur page do ke beech kuch naye posts aa jaayein, to har item neeche khisak jaata hai, aur page do wahi items dohra deta hai jo user pehle dekh chuka, ya kuch skip kar deta hai. Iska solution cursor hai. Har page ek opaque cursor lautata hai jo aakhri item ke theek baad point karta hai, aur agla request kehta hai, is cursor se purane items do. Top par naye posts aane se cursor hilta nahi, na duplicates, na gaps. Aur cursor ek rank score bhi encode kar sakta hai, sirf time nahi. Yaad rakho, list ke head par jo bhi mutable ho, use ek order-stable pointer chahiye, integer offset nahi. Isi se smooth infinite scroll bhi chalta hai.
Chronological ya ranked Slide 11
Candidate posts ikatthe hone ke baad, system decide karta hai unka order, aur yeh choice user behaviour ko lagbhag har doosre decision se zyada shape karti hai. Reverse chronological, yaani newest first, ek simple timestamp sort hai. Predictable, sasta, aur debug karna aasaan. Par yeh signal kho deta hai jab followed accounts alag alag rate par post karte hain, ek prolific account baaki sabko dabaa sakta hai. Doosra option ML-ranked hai. Ek model har candidate post ko score deta hai, features se jaise recency, author affinity, predicted engagement, aur viewer history. Yeh uneven posting ko smooth karta hai aur relevance badhata hai, par iske liye ek feature pipeline, ek low-latency model server, aur gaming se bachne ke guardrails chahiye.
Jo cases naive design todte hain Slide 12
Happy path par feed aasaan hai, par asli complexity messy cases mein hai. Common thread yeh hai, write-time ka snapshot purana pad jaata hai, isliye read path ko yeh decide karna hai ki abhi kaun kya dekh sakta hai. Delete, post ID ab bhi millions of inboxes mein pada hai, to post ko tombstone karo aur hydration par missing IDs drop karo. Block, blocked user ke purane posts feed mein reh jaate hain, to read time par block list se filter karo. Privacy change, account private ho jaata hai ya audience badal jaati hai, to visibility har viewer ke liye hydration par re-check karo. Repost, share aur original dono dikh jaate hain, to content se dedupe karo. Unfollow, ab-unfollowed author ke cached IDs dikhte rehte hain, to unfollow par purge karo ya read time par current follow set se filter karo. Aur cache drift, caches sync se bahar ho jaati hain, to ek background reconciliation job source of truth se rebuild karta hai.
Chhah principles Slide 13
To chhah principles yaad rakho. Ek, write aur read path alag karo, decide karo merge kahan hota hai. Do, push aur pull ko mix karo, default push, aur pull sirf un accounts ke liye jinki fanout write path ko melt kar de. Teen, IDs cache karo aur late hydrate karo, per-user list choti rakho. Chaar, cursors se paginate karo. Paanch, read time par filter karo, kyunki blocks, privacy, aur unfollows caches se tez badalte hain. Aur chhah, repair ke liye plan karo, ek background reconciliation job design ka hissa hai, baad ki soch nahi.
Aage kya
Bas, yahi hai chapter gyaarah. Ek kaam karo, socho ki ek user paanch sau aam accounts aur teen celebrities ko follow karta hai. Hybrid model mein uski feed kaise banti hai, kaun se posts push hote hain aur kaun se pull, yeh loud bolkar samjhao. Phir socho, wahi user fanout ke turant baad un paanch sau mein se ek ko block kar deta hai, kaun sa layer use catch karta hai aur pre-built cache par bharosa kyun nahi kar sakte. Agle chapter mein hum ek chat system design karenge.