Caching | Interview Dojo

Example context

Facebook-like Home Feed

Read path

GET /feed → fetch candidate postIds → hydrate posts/media → rank → rules → cursor

Write path

POST /post → validate → store → async fanout/index/counters/notifications

Engagement

POST /like | /comment → idempotent write → counters → ranking features update

Caching goals for this system

Reduce p95/p99 latency for GET /feed (hydration is expensive)
Protect DB and downstream services from read amplification
Avoid cache stampedes for hot users and hot posts
Keep privacy boundaries (don’t leak personalized feed to wrong user)
Lower origin egress for media via CDN hit ratio

A) Where to cache

Think in layers: client → edge → gateway/proxy → service → database. Each layer has different risk and payoff.

Template — choose cache layers safely

Template

1) Start with the edge (CDN) for bytes: • static assets + media • biggest win: egress + tail latency 2) Add a service cache (Redis/Memcached) for hot objects/lists: • reduces DB QPS and read amplification 3) Add in-proc cache only for micro-hot data: • small, bounded (Caffeine/Guava) • protects p95 but must be conservative 4) Use client cache for static/semi-static: • Cache-Control + ETag • content-hash filenames for perfect caching Rule of thumb: • The closer to user, the bigger the byte savings • The closer to DB, the bigger the QPS savings • The more personalized the data, the higher the risk

Example — Home Feed cache layers

Example

Client: • App caches static JS/CSS bundles (hashed) and some images CDN/Edge: • All media (images/video segments) cached with long TTL + versioned URLs • Origin shield enabled so spikes don’t hit origin Gateway/Proxy: • Coalesce duplicate requests for same user feed rebuild (if supported) Service (Redis): • feed:head:<user> (top N postIds) short TTL • post:<id> and author/profile fragments TTL minutes • negative cache for missing post IDs briefly Database: • rely on buffer pool, but do not treat DB cache as your “app cache”

Common pitfalls

Caching personalized responses at CDN without correct Vary → privacy leak
Skipping CDN and trying to “Redis your way” out of media egress
Using DB query cache as a scaling plan (often unpredictable and invalidates broadly)
Letting in-proc cache grow unbounded (OOM or GC pain)

Interview line

I use CDN for bytes, Redis for hot objects/lists, and small in-proc caches for micro-hot reads—personalized responses are carefully scoped to avoid leaks.

↑ Back to top

B) What to cache

Cache what is reused, expensive, and safe to serve slightly stale—with a clear invalidation story.

Template — pick cache candidates

Template

Pick items that are: • High reuse (many reads per write) • Expensive to compute or fetch • Small enough to store • Safe if briefly stale Typical categories: 1) Objects by ID: user, post, profile 2) Lists / heads: top-N feed, trending IDs 3) Derived results: permissions, feature flags, exchange rates 4) Rendered fragments: precomposed JSON/HTML (high risk: vary rules) 5) Negative cache: 404s briefly to prevent hot-miss storms Avoid caching: • Anything that is user-specific unless key includes identity + policy • Highly volatile correctness-critical values (unless very short TTL / write-through)

Example — Home Feed: what to cache

Example

Cache objects: • post:<postId> (text + metadata) TTL 5–15m • user:<userId>:profile TTL 15–60m Cache list heads: • feed:head:<userId> TTL 30–120s (soft TTL), hard TTL 10m • comments:top:<postId> TTL 30–120s Derived: • permissions:<viewerId>:<ownerId> TTL short + version token Negative cache: • post:missing:<postId> TTL 30–60s (prevents repeated DB misses)

Common pitfalls

Caching query results with unbounded keys (key explosion)
Caching giant payloads (serialization + memory blowup)
Caching authorization decisions without including relevant policy version
No negative caching → hot-miss floods when IDs are probed

Interview line

I cache objects by ID and the feed head; I keep TTLs aligned to freshness needs and I negative-cache misses briefly to stop hot-miss storms.

↑ Back to top

C) Invalidation (TTL, event busting, versioned keys)

Cache invalidation is hard—so use a small set of proven patterns and keep them consistent.

Template — invalidation patterns that scale

Template

1) TTL as the default: • simplest, works for semi-static and lists • jitter TTL ±10–20% to avoid synchronized expiry 2) Explicit busting (event-based): • on write, emit “entity changed” event • consumers delete or update cache keys • use Outbox/CDC so you don’t miss events 3) Versioned keys / tokens: • key contains version token (v7) • bump token to instantly invalidate many keys • old generations expire naturally via TTL Recommended combo: • TTL + event busting for objects • soft/hard TTL + refresh-ahead for hot lists

Example — Home Feed invalidation story

Example

Post updated: • DB write succeeds • Outbox publishes PostUpdated(postId) • Cache consumer deletes post:<postId> and related fragments • Feed head remains valid (it contains IDs), hydration pulls fresh post on next read Feed head: • Use soft TTL 60s and refresh-ahead • Keep hard TTL 10m as safety net • If ranking logic changes: bump feed version token and rebuild gradually

Common pitfalls

Dual-writes without outbox/CDC (cache and DB drift)
No TTL on versioned keys (old generations never leave)
Aggressive global purges during peak (self-inflicted outage)
Treating TTL as correctness guarantee (it’s a freshness knob, not truth)

Interview line

TTL is my default; correctness-sensitive objects also use event-based busting via outbox/CDC, and hot lists use soft/hard TTL with refresh-ahead.

↑ Back to top

D) Consistency patterns

Choose the pattern per data type: correctness vs latency vs complexity.

Template — map data → cache pattern

Template

Cache-aside (most common): • Read: cache → miss → DB → set • Write: DB → invalidate/update cache Read-through: • cache layer loads DB on miss (good for coalescing) Write-through: • write cache + DB synchronously (good for counters if correctness matters) Write-behind / write-back: • write cache first, DB async (fast but risky unless cache is durable and replayable) Refresh-ahead: • refresh hot keys before they expire to protect p95/p99 Rule: • Use cache-aside unless you have a strong reason not to.

Example — Home Feed: which pattern where

Example

Cache-aside: • post:<id>, user:<id>:profile Read-through + coalescing: • feed:head:<userId> (many concurrent reads) Write-through: • like_count and comment_count (if you need fast reads + strong-ish correctness) Refresh-ahead: • top feed head for active users (prevents p99 spikes when TTL expires)

Common pitfalls

Write-behind on correctness-critical data without durability (data loss risk)
Write-through everywhere (hurts write latency and availability)
Refresh-ahead without bounds (background refresh becomes a thundering herd)

Interview line

Cache-aside for safety, write-through for some counters, and refresh-ahead for hot lists—pattern depends on correctness and tail-latency needs.

↑ Back to top

E) Eviction & sizing

Eviction policy + memory sizing decides whether your cache is stable or constantly thrashing.

Template — eviction & sizing plan

Template

1) Pick eviction/admission: • LRU: good for recency • LFU/TinyLFU: protects from one-hit wonders • segmented policies (ARC/2Q): resist scans 2) Measure working set: • plot hit ratio vs memory (curve) • aim for where extra memory gives diminishing returns 3) Headroom: • keep 20–40% free memory to avoid sudden evictions and latency spikes 4) Hybrid caches: • small in-proc cache for micro-hot reads • distributed cache for shared reuse 5) Watch cardinality: • list caches can explode key count; add bucketing/versioning and TTLs

Example — Home Feed cache sizing intuition

Example

Data shape: • post objects are reused across many users (good cache reuse) • feed heads are per-user (high cardinality; keep TTL short) Policy: • Use TinyLFU/LFU-like admission for post objects • Keep per-user feed head small (top N IDs only) and short-lived • Reserve memory for shared objects; cap per-user keys to avoid key explosion

Common pitfalls

Caching per-user large blobs (memory gets eaten by cardinality)
LRU without admission control (one-hit wonders push out hot keys)
No headroom (eviction storms at peak)
Oversized values (serialization + network overhead dominates)

Interview line

For mixed workloads I prefer TinyLFU-style admission with LRU eviction, keep headroom, and cap high-cardinality per-user caches.

↑ Back to top

F) Stampede protection

A cache miss can amplify into an outage. Stampede protection is non-negotiable for hot keys.

Template — stop the thundering herd

Template

1) Request coalescing (single-flight): • one request rebuilds the value • others wait or serve stale 2) Locks with TTL: • Redis SET NX PX=... to acquire lock • use fencing tokens for safety 3) Jitter TTL: • randomize expiry times 4) Soft/hard TTL: • serve stale within hard TTL while refreshing in background 5) Backpressure: • cap concurrent rebuilds • shed optional work (degrade gracefully)

Example — Home Feed stampede prevention

Example

Problem: • many clients open app at the same time → feed head expires → stampede Solution: • soft TTL 60s: serve stale + trigger background refresh • hard TTL 10m: never hammer DB during refresh failures • single-flight per user: only one refresh builds the feed head • cap refresh concurrency globally to protect downstream services

Snippet — simple Redis lock for refresh (conceptual)

Snippet

// Acquire lock (10s). Only the winner rebuilds.
SET lock:feed:head:<userId> <token> NX PX 10000

// Winner rebuilds then writes cache
SET feed:head:<userId> <value> PX <ttl_ms>

// Release lock (verify token to avoid deleting someone else's lock)
if GET lock == token then DEL lock

Common pitfalls

No coalescing (every request rebuilds on miss)
Locks without TTL (deadlocks)
Locks without fencing tokens (stale owner deletes new lock)
No backpressure (refresh job floods DB under load)

Interview line

Hot keys use single-flight and soft/hard TTL. We would rather serve slightly stale than stampede the DB.

↑ Back to top

G) Metrics & alarms

If cache health isn’t visible, you’ll learn about it from an outage or a bill.

Template — cache observability checklist

Template

Core: • hit ratio (overall + per namespace) • byte hit ratio (bytes served from cache vs origin) • latency: cache get/set p50/p95/p99 Stability: • evictions/sec, memory used, fragmentation • key cardinality per namespace • error rates (timeouts, connection pool saturation) Downstream impact: • DB QPS and p95 latency • upstream request latency when cache degrades Alarms: • hit ratio drop > 10% for 5m • eviction surge • cache latency p95 jumps • connection pool > 80% or timeouts spike

Example — Home Feed: alarms that matter

Example

Key alarms: • feed:head hit ratio drop → likely TTL misconfig or stampede • post:<id> miss spike → invalidation bug or cache flush • Redis p95 latency ↑ → network saturation or CPU pressure • Origin egress ↑ + CDN hit ratio ↓ → cache key/headers regression

Interview line

We alert on hit ratio drops, eviction storms, and cache latency, plus downstream DB QPS—cache metrics must correlate to user SLOs.

↑ Back to top

H) Practical examples (copy/paste patterns)

These are the most reusable interview patterns across many systems.

Example — Post object cache (safe cache-aside)

Example

Key: post:<postId>:v1 TTL: 5–15 minutes + jitter Invalidate: PostUpdated event deletes post:<postId>:* Notes: keep payload small; store compressed if large; avoid over-fetching

Example — Feed head cache (hot list with soft/hard TTL)

Example

Key: feed:head:<userId>:v7 Value: [postId1, postId2, ... postIdN] Soft TTL: 60–120s (serve stale and refresh) Hard TTL: 10m (safety) Stampede: single-flight per user + global refresh cap

Example — CDN media caching (huge egress saver)

Example

URL: /media/<contentHash>/w=640.webp Headers: Cache-Control: public, max-age=31536000, immutable Notes: • versioned URLs so no purges needed • origin shielding on • signed URLs only when private access is required

↑ Back to top

I) Key design & HTTP headers

Cache keys define correctness. If the key doesn’t capture identity/policy, you risk leaks.

Template — safe key structure

Template

Key format: • <namespace>:<id>[:subscope][:version] Include in key when relevant: • user identity (viewerId) for personalized data • locale / device class if response differs • policy version for authorization/visibility • experiment bucket for A/B variants

Example — Home Feed keys (good vs dangerous)

Example

Good: • feed:head:<userId>:v7 • post:<postId>:v1 • perms:<viewerId>:<ownerId>:policyV3 Dangerous: • feed:head:v7 (no user identity → leaks) • profile:<userId> (no version/policy; invalidation pain) • search:<q> without filters (key misses important dimensions)

Snippet — HTTP caching header patterns

Snippet

Static assets (hashed filenames):
Cache-Control: public, max-age=31536000, immutable

Shared dynamic (CDN can cache):
Cache-Control: public, s-maxage=60, stale-while-revalidate=120
Vary: Accept-Encoding

Personalized:
Cache-Control: private, no-store
# or bypass edge and cache in service with user-scoped keys

Interview line

I design cache keys like API contracts: keys include identity and policy versions to prevent leaks, and version tokens make global invalidation safe.

↑ Back to top

J) One-liners to use in the interview

“I cache in layers: CDN for bytes, Redis for shared hot objects/lists, and small in-proc cache for micro-hot paths.”
“Invalidation is TTL by default, event-based for correctness, and version tokens for fast mass purges—with jitter to avoid herds.”
“Hot lists use soft/hard TTL + refresh-ahead; single-flight prevents stampedes.”
“Cache keys include identity and policy version—correctness and privacy come first.”

↑ Back to top