Redis

In-memory data structure store. Cache, queue, rate limiter, session store, pub/sub, and stream broker — all in one binary.

At a glance

Why Redis

Redis is a single-threaded, in-memory key/value server with a surprisingly rich set of value types. Every operation is O(1) or O(log n) on structures that already fit in RAM, so latency is measured in microseconds. The tradeoff: everything must fit in memory, and durability is best-effort.

You don’t pick Redis because it’s a database. You pick it because it’s the lowest-latency place to put small, hot data that’s expensive to compute.

Data structures that matter

Strings. Plain bytes up to 512 MB. Use for: cached HTML, JSON blobs, counters (INCR), simple tokens. SET key val EX 300 for a 5-minute TTL.

Hashes. Field/value maps. Store small objects field-by-field — cheaper than serializing JSON on every update.

HSET user:42 name "Ada" plan "pro" credits 1200
HINCRBY user:42 credits -10

Lists. Doubly-linked lists. LPUSH / RPOP give you an FIFO queue with blocking reads (BLPOP). Good for simple work queues.

Sets. Unique unordered collections. SADD, SISMEMBER, and the set algebra commands (SINTER, SUNION, SDIFF) are O(N) on small sets.

Sorted sets (ZSets). The workhorse. Items with a numeric score, sorted. Leaderboards, time-ordered event buffers, sliding-window rate limiters, priority queues — all ZSets underneath.

Streams. Append-only log with consumer groups. Closer to Kafka than to pub/sub. Good for intra-service event fan-out when you don’t want to run Kafka.

Caching patterns

Cache-aside (lazy load). The app checks Redis first; on miss, reads the primary DB, writes the result back with a TTL. Simple, safe, and the default. Downside: first read after eviction is slow.

def get_user(user_id):
    key = f"user:{user_id}"
    if (hit := redis.get(key)) is not None:
        return json.loads(hit)
    row = db.query_one("SELECT ... WHERE id = %s", user_id)
    redis.set(key, json.dumps(row), ex=300)
    return row

Write-through. Every write updates the DB and the cache synchronously. Strong consistency, higher write latency. Use when reads outnumber writes by a huge margin and stale data is unacceptable.

Write-behind (write-back). Write to cache, flush to DB asynchronously. Lowest write latency, but a cache crash loses data. Rarely the right call unless you deeply understand the failure modes.

Cache invalidation. Either TTL everything and accept staleness, or delete the key on write. “There are only two hard things in computer science…” — still true. When in doubt, use a short TTL.

Pub/Sub vs Streams vs Kafka

• Redis Pub/Sub — fire-and-forget broadcast. No persistence. If a subscriber is offline, it misses the message. Use for ephemeral fan-out (cache invalidation broadcasts, live UI pings).
• Redis Streams — durable log with consumer groups and acknowledgments. Good up to low millions of events per day per stream, all-in-one operational footprint.
• Kafka — built for high throughput, long retention, multiple consumer groups replaying history, and cross-team integration. Higher ops burden.

Rule of thumb: if you already run Redis and your throughput is modest, Streams is free. If you’re building a data platform, pay the Kafka tax.

Rate limiting recipe

A robust sliding-window rate limiter in a few lines using a sorted set:

def allow(key, limit, window_seconds):
    now = time.time()
    cutoff = now - window_seconds
    pipe = redis.pipeline()
    pipe.zremrangebyscore(key, 0, cutoff)
    pipe.zadd(key, {str(uuid4()): now})
    pipe.zcard(key)
    pipe.expire(key, window_seconds)
    _, _, count, _ = pipe.execute()
    return count <= limit

Each request adds a timestamp, old entries are pruned, and we count what’s left. Cheap and correct. In high-traffic paths replace with a Lua script so it’s atomic in one round-trip.

Persistence and durability

• RDB snapshots — periodic fork-and-dump to disk. Fast recovery, loses data since last snapshot.
• AOF (append-only file) — every write is appended. appendfsync everysec is the sane default: durable to within 1 second.
• Both together — what we run in production: RDB for fast restart, AOF for minimal data loss.

Treat Redis as a cache first, a database second. If losing the whole thing would be a company-wide incident, you’re using it wrong.

Operational gotchas

• KEYS * in production will block the server. Use SCAN.
• Big keys (multi-MB strings, million-element lists) stall the server on access. Break them up.
• Eviction policy. Set maxmemory and maxmemory-policy (usually allkeys-lru) — otherwise the server OOMs instead of evicting.
• Cluster gotcha. In Redis Cluster, multi-key commands only work if all keys live in the same hash slot. Use hash tags ({user:42}:profile) to force co-location.