Welcome
Apache Iggy documentation: how Iggy organizes data into streams, topics, partitions and segments, and where to start.
Hyper-Efficient Message Streaming
at Laser Speed
High-performance, persistent message streaming platform written in Rust, capable of processing millions of messages per second with ultra-low latency.
Named after the Italian Greyhound - small yet extremely fast
Why Iggy?
How Iggy compares to traditional message streaming platforms
How Iggy organizes data
Iggy uses a hierarchical model: streams contain topics, topics contain partitions, and partitions store messages in segments on disk. This gives you flexible multi-tenancy, parallel processing, and efficient storage all in one.
Stream Hierarchy
The append-only log
Every partition is an append-only log. Messages are immutable once written, consumers can start from a chosen offset, and retained messages can be replayed. Retention removes old data, and offsets can contain gaps.
Append-Only Log
Messages are appended sequentially. Consumers track their position independently via offsets.
Click any message block to move the consumer pointer. New messages appear from the right.
Key capabilities
| Capability | Details |
|---|---|
| Transport | TCP, QUIC, WebSocket (custom binary protocol), HTTP (REST) |
| Performance | Thread-per-core + io_uring on Linux, binary message views, configurable memory pool with a default 4 GiB budget |
| Security | TLS on all transports, Argon2id hashing, AES-256-GCM message encryption, granular permissions with per-stream/per-topic permissions |
| Connectors | 16 sink and 4 source plugins including PostgreSQL, MongoDB, Elasticsearch, ClickHouse, Apache Iceberg, S3, with data transforms |
| AI Integration | MCP server with 40+ tools for LLM-driven message streaming management |
| Management | Web UI dashboard (embedded or standalone), CLI with shell completions, Prometheus metrics; connector OpenTelemetry export (broker export limits) |
| Clustering | Built on Viewstamped Replication (VSR) consensus; single node by default, multi-node via [cluster] configuration |
| Deployment | Single server process, Docker & Helm, configurable CPU/NUMA affinity on Linux |
Ecosystem
Iggy is more than just a server. The project includes a full ecosystem of tools:
- Connectors Runtime - dynamically loaded Rust plugins for data integration: 16 sinks and 4 sources. Ingest from PostgreSQL, Elasticsearch or InfluxDB into Iggy, or forward to MongoDB, Elasticsearch, ClickHouse, Apache Iceberg, Quickwit, S3 and more. Built-in data transforms and Prometheus metrics.
- MCP Server - Model Context Protocol server exposing 40+ tools for LLM integration. Works with Claude Desktop via stdio and HTTP transports.
- Web UI - SvelteKit dashboard for stream/topic management, message browsing with JSON/string/XML decoders, and user management.
- CLI - full-featured command-line interface with named connection contexts, session-based login, and shell completions.
- SDKs - client libraries for 8 languages (Rust, Python, Java, Go, Node.js, C#, C++, PHP), most with runnable examples.
- Benchmarking - built-in
iggy-benchtool with a Yew/WebAssembly dashboard for performance testing.