Apache Iggy 0.8.0 Release

By piotr

Release 0.8.0

Less than two months after 0.7.0, we're excited to release Apache Iggy 0.8.0 - a release focused on deep internal restructuring and hardening the path to clustering.

While 0.7.0 laid the foundations for Viewstamped Replication (VSR) consensus, 0.8.0 is about unifying the architecture: a complete wire protocol rewrite that eliminates three redundant serialization paths, a new shard crate with proper request routing, a persistent WAL journal for metadata durability, a WAL-backed ClientTable with view-change safety, new memory primitives for zero-copy message handling, initial A2A (Agent-to-Agent) protocol support for agentic workloads, and significant security improvements including user header encryption. On the SDK side, the C++ client is being revived with low-level bindings and Bazel infrastructure, while the Go SDK gained full TCP/TLS support and the Java SDK shipped async connection pooling.

Iggy Server

Wire Protocol Rewrite

The largest refactoring effort in this release is the complete unification of the wire protocol. The binary protocol layer previously had three redundant serialization paths: ServerCommand enum with a hand-rolled binary_mapper, the BytesSerializable trait for domain type serialization, and WireEncode/WireDecode for wire protocol types - all doing the same job with different APIs.

This release collapses them into one:

  • Wire protocol codec and types - new WireEncode/WireDecode types in the binary_protocol crate
  • Wire types for all commands - wire types and zero-copy message primitives for every command
  • Sans-IO frame codec - frame codec and command dispatch table, decoupled from I/O for better testability
  • ServerCommand/BytesSerializable removal - the ServerCommand enum, binary_mapper, 45+ command structs, and BytesSerializable trait are all deleted; wire types flow directly from transport through shard dispatch to handlers
  • Partitions rework - partitions refactored around Message<H, B> with explicit ownership model, removing the old prefix/tail copy-on-write design and its aliasing problems

Clustering & VSR Consensus

Building on 0.7.0's VSR foundations, this release introduces the structural primitives needed to wire consensus into the server:

  • Shard crate & IggyShard - new dedicated shard crate with a generic IggyShard that dispatches incoming network messages to the correct consensus plane via PlaneIdentity
  • Shard router - routes requests within shards to the appropriate handlers
  • Plane trait - a unified abstraction for dispatching requests to different subsystems based on plane demux logic
  • MuxPlane handles - metadata and partition handles for MuxPlane, wiring subsystems into the consensus layer
  • Namespaced pipelines - independent commit progress across namespaces with per-namespace VecDeques under one global operation sequence and hash chain
  • Loopback queue - loopback queue for primary self-addressed messages in consensus
  • STM response threading - state machine responses now propagated back through consensus Reply body, so clients receive assigned IDs without a second query
  • VSR types consolidation - consolidated VSR types into binary_protocol
  • server-ng binary - new iggy-server-ng binary crate for the next-generation server with clustering support
  • ClientTable with WAL-backed commit path - client request tracking with at-most-once semantics, durably persisted through the WAL and safe across view changes

The simulator also advanced significantly with a PacketSimulator for deterministic testing, poll_messages support, Network wired into the tick loop with per-replica outboxes and replica crash simulation, and the view change protocol wired end-to-end so leader failover can now be exercised deterministically.

Persistent WAL Journal

Persistent WAL journal with recovery and compaction - metadata changes are now durably persisted to a write-ahead log with recovery and compaction support, a critical building block for cluster state durability. The partition journal was also refactored to use the Storage trait as backing storage rather than inline data.

Memory & Buffer Primitives

New memory management primitives for high-throughput message handling:

  • Aligned buffer memory pool - pre-allocated aligned buffers for efficient I/O operations
  • TwoHalves buffer - a split buffer using Vec<u8> instead of Bytes, with mutable and immutable halves that share the same allocation initially and diverge on first clone - the mutable part is copied while the immutable part uses thread-safe reference counting for cheap sharing

Agentic Workloads

  • A2A (Agent-to-Agent) protocol support - initial server-side support for the A2A protocol, backed by JWKS so agents from different tenants can authenticate with their own public keys, with key rotation handled without server restarts. This lands the first piece of the agentic story outlined in 0.7.0's roadmap.

Security Improvements

  • User header encryption - both server-side and client-side encryption now encrypt user-defined headers alongside the message payload; previously headers were left in plaintext on disk (BREAKING)
  • Random JWT secrets - empty JWT secret defaults now trigger secure random generation at startup, preventing all instances from sharing the same signing key
  • SecretString credentials - credentials wrapped in SecretString to prevent accidental leaks in logs and debug output
  • Consensus header validation - fixed undefined behavior where untrusted bytes from the network were reinterpreted as Rust enum discriminants without validation
  • Plaintext secrets regression test - ensures secrets are never persisted as plaintext
  • Dependabot security fixes - swept up outstanding Dependabot-flagged vulnerabilities across the workspace

Notable Bug Fixes

Other Improvements

Breaking Changes

Docker images are available on Docker Hub.

SDKs

Rust

C++

The C++ SDK is being actively revived with foundational work:

Java

C#

Go

Python

Node.js (TypeScript)

CLI

Connectors

New connectors and runtime improvements:

Learn more in the documentation.

Benchmark Dashboard

Web UI

Helm Charts

CI/CD & Infrastructure

What's Next

Our primary focus remains completing the clustering implementation with VSR consensus. The iggy-server-ng binary introduced in 0.8.0 will become the cluster-ready server, integrating the shard crate, plane abstractions, persistent WAL journal, and namespaced pipeline work into a fully functional replicated cluster.

Other upcoming efforts:

  • Server core optimizations: kTLS, DirectIO, and continued NUMA tuning
  • SDK evolution with sans-IO architecture - the frame codec and wire types shipped in 0.8.0 are the foundation for transport-agnostic, fully testable client implementations
  • C++ SDK completing its low-level bindings and reaching feature parity
  • More connector integrations and runtime hardening
  • Web UI enhancements - building on the embedded dashboard
  • Agentic AI - building on the initial A2A protocol support in 0.8.0

Thanks to our amazing community and contributors for making Apache Iggy better with every release.

Join us on Discord and help shape the future of the project!