Redis with Python

What happens when you stop using Redis and start building one?

I built a Redis-compatible in-memory data store from scratch in Python as part of CodeCrafters' Build Your Own Redis challenge.

The project started with a simple PING / GET / SET server and grew into a small Redis-like system with its own RESP implementation, persistence, replication, transactions, streams, pub/sub, sorted sets, geospatial operations, authentication, and bit operations.

View the source on GitHub →

What I built

⚡ Networking & RESP

• TCP server built directly with Python's socket API
• RESP request parsing and response encoding
• Multiple commands and clients
• Command dispatch and connection handling
• Blocking operations using threads and condition variables

🧱 Data Structures

Implemented Redis-style operations for:

• Strings
• Lists
• Streams
• Sorted sets
• Bitmaps
• Geospatial data

Including commands such as:

SET · GET · INCR · RPUSH · LPUSH · LRANGE · LPOP · BLPOP

XADD · XRANGE · XREAD

ZADD · ZRANGE · ZRANK · ZSCORE · ZREM

GEOADD · GEOPOS · GEODIST · GEOSEARCH

🔄 Transactions & Optimistic Locking

Implemented:

• MULTI
• EXEC
• DISCARD
• WATCH
• UNWATCH

Keys maintain versions so a transaction can detect whether a watched key changed before EXEC.

This turned a simple command queue into a small concurrency-control problem.

📡 Replication

Built a basic master/replica system including:

• Replica connection and handshake
• PING
• REPLCONF
• PSYNC
• Full resynchronization
• RDB transfer
• Command propagation
• Replica acknowledgements
• WAIT
• Replication offsets

The replica communicates with the master through the same RESP-based command stream used by clients.

💾 Persistence

Implemented both major persistence paths:

• RDB loading — parsing RDB data and restoring keys with expiry information
• AOF — writing mutating commands to an append-only file and replaying them on startup
• AOF manifests and incremental AOF files
• Configurable appendfsync behaviour

📺 Streams & Blocking Commands

Implemented Redis Streams with:

• XADD
• XRANGE
• XREAD
• Blocking XREAD
• $ stream IDs

I also implemented blocking list retrieval with BLPOP, using Python condition variables to wake waiting clients when data becomes available.

📣 Pub/Sub

Implemented:

• SUBSCRIBE
• UNSUBSCRIBE
• PUBLISH

Connections enter subscription mode and are restricted to the commands Redis allows in that state.

🔐 Authentication & ACL

Added a small ACL system supporting:

• ACL WHOAMI
• ACL GETUSER
• ACL SETUSER
• AUTH
• SHA-256 password hashing
• Per-connection authentication state

🗺️ Geospatial Operations

Implemented:

• GEOADD
• GEOPOS
• GEODIST
• GEOSEARCH

Coordinates are encoded using bit interleaving, while distance calculations use the Haversine formula.

🔢 Bit Operations

Implemented:

• SETBIT
• GETBIT
• BITCOUNT
• BITOP
• STRLEN

This required treating stored values as raw bytes and manipulating individual bits rather than simply working with Python strings.

What I learned

The interesting part wasn't implementing individual Redis commands. It was seeing how the pieces interact.

A command like BLPOP requires synchronization.

WATCH requires tracking mutations to individual keys.

Replication requires a protocol stream and byte offsets.

Persistence requires reconstructing state from serialized data.

Streams require ordering and blocking semantics.

And all of them ultimately have to pass through the same command-processing and RESP machinery.

Building these pieces from scratch made the boundary between "a database API" and "the machinery underneath the API" much easier to see.

Tech

Python · TCP sockets · RESP · threading · synchronization · persistence · replication · Redis data structures

The implementation itself does not use Redis or redis-py.

Challenge

This project is based on CodeCrafters' Build Your Own Redis, a hands-on systems challenge that progressively tests more advanced Redis behaviour.

Try the challenge yourself →

Read the implementation →