# Architecture
> Inside the Iggy server: how work is scheduled across cores, how requests are routed, and how a message reaches disk.
Rendered page: https://iggy.apache.org/docs/introduction/architecture/
Source: https://github.com/apache/iggy-website/blob/main/content/docs/introduction/architecture.mdx
This page covers the internals of the Iggy server: how work is scheduled across CPU cores, how requests are routed, and how data ends up on disk. If you're looking for the domain model (streams, topics, partitions, consumer groups, message format), see [concepts](/docs/introduction/concepts) first.
## How a message flows through Iggy
Before diving into the architecture details, here's the complete journey of a message from client to disk:
## Thread per core (shared nothing) + io_uring
Iggy uses a **thread-per-core shared nothing architecture** combined with `io_uring` for maximum performance. This design has been proven by systems like ScyllaDB and Redpanda, and is inspired by the Seastar framework.
### How it works
Each CPU core runs its own **shard** (an instance of `IggyShard`), pinned to a specific core via `sched_setaffinity` on Linux. Each shard has its own single-threaded `compio` async runtime, which means there is no cross-thread synchronization needed within a shard. Memory is bound to the NUMA node of the core via `hwlocality` for optimal memory access latency.
### Shard roles and connection distribution
Shard 0 has a special role: it binds **every listener** - the replica plane and all client transports (TCP, QUIC, WebSocket, HTTP). Connections are then spread across shards at accept time:
- **Plaintext TCP and WebSocket** connections are handed off round-robin to peer shards. Shard 0's coordinator duplicates the socket's file descriptor, ships a connection-setup frame to the target shard, and drops its own handle, so the owning shard serves the connection from then on - before a single byte is read.
- **QUIC and TLS-wrapped TCP** connections terminate on shard 0, because their per-connection state cannot be moved between shards.
All shards, **including shard 0**, own partitions and serve partition requests.
### Request routing
Requests are routed between shards using **message passing** (via `crossfire` bounded mpsc channels), which avoids locking entirely. The routing logic splits operations into two planes:
- **Metadata operations** (create/delete stream/topic/user etc.) always execute on **shard 0** - it is the only shard that commits metadata
- **Partition operations** (send_messages, poll_messages, store_consumer_offset) are routed to the shard owning that partition via a lock-free concurrent map lookup. A request that lands on a non-owning shard rides the inter-shard message bus to the owner
Partition-to-shard assignment is **deterministic**: the packed `IggyNamespace` is hashed with `Murmur3`, and the upper 16 bits of the hash are taken modulo the shard count (the upper bits are used because Murmur3 has weak lower bits for small integer inputs).
### IggyNamespace
The `IggyNamespace` packs stream_id (20 bits), topic_id (12 bits), and partition_id (20 bits) into a single `u64` for efficient hashing and routing. This gives maximums of 1,048,576 streams, 4096 topics per stream, and 1,000,000 partitions per topic.
### CPU allocation modes
The sharding system supports multiple allocation modes via the `cpu_allocation` config:
- `"all"` - one shard per available CPU core
- A numeric value (e.g. `4`) - exactly N shards pinned to cores 0..N
- A range (e.g. `"5..8"`) - shards on specified core range
- `"numa:auto"` - automatically detect NUMA topology and bind accordingly
- `"numa:nodes=0,1;cores=4;no_ht=true"` - fine-grained NUMA control per node with hyperthread avoidance
### io_uring and compio
Traditional async runtimes like tokio use `epoll` which is **readiness-based** - you ask the kernel "is this file descriptor ready?" and then perform the I/O yourself. The Linux kernel considers regular files "always ready" for epoll, which means tokio has to outsource file I/O to a blocking thread pool (up to 512 threads). This does not scale well.
`io_uring` is **completion-based** - you submit I/O requests to a submission queue (SQ), and the kernel completes them and places results in a completion queue (CQ). Both queues are lock-free ring buffers shared between user space and kernel. This is fundamentally better for disk I/O.
Iggy uses **compio** as its async runtime, which provides a driver-disaggregated architecture on top of io_uring (Linux) and IOCP (Windows). Each shard gets its own compio executor configured with:
- Capacity: 4096 concurrent I/O operations (by default)
- Event interval: 128 events per loop iteration
- Cooperative task running enabled
### Performance: Tokio vs Thread-per-Core
The migration from Tokio to thread-per-core with compio delivered significant latency improvements across the board:
### Left-right metadata
Shared metadata (streams, topics, consumer groups) uses a **left-right** concurrent data structure. Shard 0 holds the single write handle and is the only shard that commits metadata mutations, while every shard holds a read handle. This provides:
- Lock-free reads on all shards (no contention on the hot path)
- Strongly consistent writes serialized through shard 0
## Clustering
The server is built on the **Viewstamped Replication (VSR)** consensus protocol. Metadata operations are committed through the VSR write-ahead log **even on a single node**, which is the default deployment. Multi-node clustering is enabled via the `[cluster]` configuration (a node roster plus `--replica-id` to pick the local entry). See [clustering](/docs/clustering/vsr) for the details and maturity caveats.
## Storage structure
Having in mind that stream consists of topics, which might have one or more partitions assigned, and each partition consists of segments, we can visualize the structure of the Iggy data directory as follows:
```bash
local_data/
├── metadata/
│ └── journal.wal
├── runtime/
│ └── current_config.toml
├── logs/
│ └── iggy-server.log
├── state/
└── streams/
└── 0/
└── topics/
└── 0/
└── partitions/
└── 0/
├── 00000000000000000000.index
├── 00000000000000000000.log
└── offsets/
```
The stream, topic and partition directories are named after their numeric IDs, **assigned from 0**. The `metadata/journal.wal` file is the VSR write-ahead log that persists all metadata operations (stream/topic/user creation, etc.). The `runtime/current_config.toml` file captures the configuration the server actually booted with. Segment files are named by the 20-digit start offset of their first record. The `.index` file is created automatically and speeds up searches by keeping track of the offsets and timestamps of the records. The `offsets/` directory holds the server-side consumer and consumer group offsets.
## Memory pool
Iggy uses a custom memory pool with 28 buckets holding buffer sizes from 4 KiB to 512 MiB. The default pool size is 4 GiB with up to 8192 buffers per bucket. This eliminates allocation overhead on the hot path and enables zero-copy message passing between components. The pool is page-aligned (4096-byte multiples) and requires a **minimum of 512 MiB**.
## Write pipeline
Messages flow through a multi-stage write pipeline:
1. Messages arrive on the owning shard and are buffered in the partition journal
2. A flush is triggered when either the message count threshold or the size threshold is reached - both are **per-topic options** set at topic creation (defaults: 1024 messages, 1 MiB)
3. The `MessagesWriter` uses **vectored I/O** with up to 1024 buffers per syscall
4. `fsync` per write is a per-topic option (`enforce_fsync`) for durability guarantees
5. When a segment reaches the topic's segment size (default 1 GiB), it is **sealed** and a new segment is created