Caching Strategies
Cache patterns, eviction policies, and consistency trade-offs.
Why Cache?
Caching stores frequently accessed data closer to the consumer, reducing latency and backend load. A well-placed cache can turn a 100ms database query into a 1ms memory lookup.
Cache Patterns
Cache-Aside (Lazy Loading)
The application manages the cache explicitly:
- Check cache for data
- On hit, return cached data
- On miss, fetch from database, store in cache, return
Pros: Only caches what's actually requested. Cons: First request is always slow (cold cache). Risk of stale data.
Write-Through
Every write goes to both cache and database:
- Write to cache
- Cache writes to database
- Return success
Pros: Cache is always consistent. Cons: Write latency increases. May cache data that's never read.
Write-Behind (Write-Back)
Write to cache immediately, asynchronously flush to database:
- Write to cache, return success
- Cache batches writes to database in background
Pros: Lowest write latency. Cons: Risk of data loss if cache crashes before flushing.
LRU Cache Simulation
Here's a step-by-step simulation of an LRU cache with capacity 3:
| Step | Action | Key | Cache State | Result |
|---|---|---|---|---|
| 1 | GET | A | A:1 | MISS |
| 2 | GET | B | A:1B:2 | MISS |
| 3 | GET | C | A:1B:2C:3 | MISS |
| 4 | GET | A | B:2C:3A:1 | HIT |
| 5 | GET | D | C:3A:1D:4 | MISS |
| 6 | GET | C | A:1D:4C:3 | HIT |
| 7 | GET | B | D:4C:3B:2 | MISS |
| 8 | GET | E | C:3B:2E:5 | MISS |
| 9 | GET | B | C:3E:5B:2 | HIT |
| 10 | GET | A | E:5B:2A:1 | MISS |
Notice how when the cache is full, the least recently used entry is evicted to make room for new entries. Accessing an existing entry moves it to the "most recently used" position.
Eviction Policies
| Policy | Evicts | Best For |
|---|---|---|
| LRU | Least recently used | General purpose, temporal locality |
| LFU | Least frequently used | Stable popularity patterns |
| FIFO | Oldest entry | Simple, predictable |
| TTL | Expired entries | Time-sensitive data |
| Random | Random entry | When access patterns are uniform |
Cache Invalidation
"There are only two hard things in Computer Science: cache invalidation and naming things." — Phil Karlton
Strategies for keeping caches fresh:
- TTL (Time-To-Live) — entries expire after a fixed duration
- Event-driven invalidation — publish events on data changes, subscribers clear cache
- Version tagging — cache keys include a version; bump version to invalidate
Distributed Caching
For multi-instance deployments, use a shared cache like Redis or Memcached:
- Redis: rich data structures, persistence, pub/sub, clustering
- Memcached: simpler, pure key-value, multi-threaded, slightly faster for basic ops
Layer caches: L1 (in-process, fastest) → L2 (distributed, shared) → Database (source of truth).