Tier 3 · Patterns & Assembly

Stream Processing

Event time, watermarks, and deciding when an unbounded computation is allowed to answer

⏱ 18 min patternsstreamingevent-time

Batch and stream are one spectrum

A batch job reads a bounded input and terminates; a stream processor reads an unbounded input and never does. Shrink the batch interval towards one record and you slide continuously from one to the other.

Batch size is not the real difference. A bounded input tells you when the computation is done: SELECT sum(amount) ... WHERE day = '2026-07-27' finishes because the table ends. A stream has no end, so Monday's total has no moment at which it becomes final. You have to invent that moment; windows, watermarks and late-data policy are how.

Event time versus processing time

Event time is the timestamp inside the event, set when the thing happened. Processing time is the wall clock when your operator saw it. They diverge constantly: broker retries (seconds), consumer lag or a rebalance (minutes), a mobile client offline in a tunnel (hours), and backfills, where a month of history arrives in twenty minutes.

Processing time therefore makes results a function of your infrastructure's mood — a 90-second lag spike at 23:59 moves money across a day boundary and nothing records it — and makes reprocessing disagree with the original run, since every processing timestamp differs.

Client clocks lie, though (clocks): a phone set four hours ahead dates events in the future and blows past every watermark. Keep both timestamps, client and broker-append, and clamp client values far ahead of ingest.

Windows

Window Shape State cost Models
Tumbling Fixed, non-overlapping; each event in one One window per key Hourly billing
Hopping / sliding Length L, advancing every S Multiplied by L/S, output too Moving averages
Session Dynamic, closed by an inactivity gap Unbounded, merges out of order User behaviour

The sliding multiplier is where people get hurt: a 1-hour window hopping every minute puts each event in 60 windows — 60x state and 60 output rows per key per hour.

Sessions are the only windows derived from the data rather than the clock, and the only ones that merge: an event landing between two open sessions joins them, so their state cannot be finalised locally.

Watermarks

A watermark is a claim: I expect no more events with event time earlier than T. When it passes a window's end, the window closes and emits — the only mechanism that turns an unbounded stream into finite answers. It is a heuristic, not a fact: nothing stops an event stamped T − 5min arriving afterwards, and the engine honours your bet either way.

It is usually max event time seen − a fixed lateness per input partition, combined as the minimum across partitions — which is why one idle partition freezes the job's watermark and every window stops emitting, a direct consequence of partitioning.

Delay Latency Completeness State
5 s Real time Loses retries and lagging consumers Minimal
5 min Dashboard-grade Covers rebalances and short lag 60x
6 h No use for alerting Covers offline mobile clients Enormous

Measure ingest time − event time at p99 and p99.9 rather than guessing: telemetry might be 8 s at p99 and 6 hours at p99.99, so no watermark is cheap and complete at once.

When late data arrives anyway

Drop it. Legitimate — but emit a counter per window and alert on the rate; this is the silent correctness loss observability exists to surface.

Side output. Route past-watermark events to a separate topic for nightly reconciliation.

Let the window update. Hold state past the watermark for a grace period and re-emit when a late event lands: Flink's allowed lateness, Beam's panes.

The third option's cost lands downstream. Consumers now receive updates, not appends: the sink must upsert by window identity, and any downstream aggregation needs retraction semantics — a negative for the old value beside the new — or its sums drift. Teams pick drop-and-count not for correctness but because correction costs three teams a schema change.

State and what it costs

Windowed aggregation is stateful: a partial aggregate per open window per key. Two million users, 24 hourly windows open across a day, 200 bytes each is about 10 GB per operator; larger windows mean proportionally more state.

State must survive a crash, so engines checkpoint: snapshot operator state to durable storage with the offsets it corresponds to, then restore, rewind and replay on recovery. That works only because the source is a replayable log, not a queue that deletes on read (queues and logs).

Checkpointing is also the main source of pauses: barriers stall the faster inputs and a multi-gigabyte RocksDB state means a multi-gigabyte upload, giving a p99 spike exactly one checkpoint interval apart. Levers: incremental or unaligned checkpoints, a longer interval, or less state.

Exactly-once stops at your process boundary

Exactly-once means output and consumed offsets commit atomically, so a replay after failure cannot double-apply — a Kafka transaction spanning produce and offset commit, or Flink's two-phase commit at the checkpoint. Effectively once, inside the pipeline.

It covers nothing outside it. A Stripe charge, an email, an INSERT into external Postgres: the replay does them again. Safe side effects need idempotency keys (idempotency).

Backfill, and the Lambda/Kappa argument

The test that matters: replay 30 days of log through the current code. Do you get the same numbers as the original run?

On event time, yes: events carry their own timestamps, so windows fall on the same boundaries. On processing time, no — that bucketing was a function of one Tuesday's consumer lag, which no longer exists. A pipeline that cannot be reproduced cannot be audited or safely changed, which is a stronger argument for event time than day-boundary accuracy.

Lambda ran two pipelines — streaming for fresh approximate results, batch recomputing from raw — at the price of one computation implemented twice and drifting. Kappa keeps one: start a second instance from offset zero, let it catch up, switch reads. With 30 to 90 days of retention that covers nearly every backfill, so the separate batch layer is usually unnecessary complexity.

What to take away

Check yourself

  1. A live dashboard and a nightly recomputation of the same day, from the same log, disagree on daily revenue. What is the most likely cause?

  2. Which statement about watermarks is correct?

  3. You enable allowed lateness so windows re-emit corrected results when late events arrive. What must change downstream?

  4. A Flink job with exactly-once enabled calls a payment API inside an operator. After a failure and restart, can a customer be charged twice?