Spindle

GitHub Repository →

Abstract: Spindle is a blazing fast, multithreaded in-memory Key-Value database engineered in Modern C++20 for microsecond latency and lock-free concurrency. It implements a highly optimized Sharded Hash Map, lock-free wait structures, a robust Write-Ahead Log (WAL), and an epoll-based Reactor for unparalleled throughput.

1. Architecture

Sharded Hash Map

Lock contention is eliminated by sharding the main Hash Map into hundreds of independent segments. Writers only lock the specific shard they modify, allowing multiple threads to read and write simultaneously across different shards.

Lock-Free Reads

Readers bypass heavy Mutexes completely. Spindle utilizes advanced atomic primitives and SeqLocks (Sequence Locks) to guarantee read consistency without blocking or acquiring traditional locks.

Write-Ahead Log (WAL)

All mutations are appended to an AOF (Append-Only File) for durability. To prevent I/O blocking, WAL writes are asynchronously batched using a high-throughput MPSC (Multi-Producer Single-Consumer) Ring Buffer.

Multi-Reactor Network

Spindle uses a modern Event Loop built on Linux epoll (Edge Triggered). With SO_REUSEPORT, incoming TCP connections are distributed automatically across a pool of Reactor threads by the kernel.

2. Performance Benchmarks

Tested locally on an 8-core CPU using 100 concurrent connections with Pipelining (batch=100) and tiny payloads. Designed to measure theoretical max engine throughput.

Spindle 4 Threads
6,812,298 ops/sec
P50: 0.53ms P99: 4.79ms
Redis 7.0 1 Thread
1,514,800 ops/sec
P50: 2.51ms P99: 8.54ms

3. Protocol & Commands

Spindle uses a simple, human-readable text protocol over TCP. Every command must be terminated with a newline (\n).

SET

SET <key> <value> [EX seconds | PX milliseconds]

Stores a value with the specified key. If the key already exists, it is overwritten. You can optionally provide an expiration time.

Returns:

Integer: (integer) 1 if successful, or ERR INVALID_FORMAT if the syntax is incorrect.

Example TCP Socket:
SET mykey John (integer) 1 SET mykey active EX 3600 (integer) 1

4. Installation

$ git clone https://github.com/DoanTrungHuy/Spindle.git $ cd Spindle $ mkdir build && cd build $ cmake .. $ make -j4 $ ./spindle_app --port 8888 --threads 4