Apache Iggy 0.7.0 Release

By piotr

Release 0.7.0

Less than three months after 0.6.0, we're excited to release Apache Iggy 0.7.0 — a massive step toward production-ready clustering.

While 0.6.0 was about the io_uring rewrite, 0.7.0 is about what comes next: replication. We've built the foundations for Viewstamped Replication (VSR) consensus, redesigned how the server manages metadata, added NUMA awareness and socket migration, embedded the Web UI directly into the server binary, and shipped important improvements across all SDKs and connectors.

New Committer

We're thrilled to welcome Łukasz Zborek as a new Apache Iggy committer! Łukasz has been a core contributor to our C# SDK and is the #8 contributor in our repository. His work goes far beyond building a top-notch C# SDK — he has helped find and fix issues across many areas of the project. His contributions span CI, BDD testing, core features like TLS support, reconnection handling, and the high-level consumer/publisher APIs. Congratulations, Łukasz!

Iggy Server

Clustering & VSR Consensus

The primary focus of 0.7.0 is laying the groundwork for clustering with Viewstamped Replication (VSR) — a critical milestone for fault-tolerant, replicated deployments.

We've implemented the core consensus building blocks that will power the upcoming cluster mode:

  • VSR view_change — the view change protocol from the VSR-revisited paper, enabling leader election and fault recovery
  • Replication pipeline — core replication machinery including on_ack, send_prepare_ok, and replicate, with on_replicate for data propagation between nodes
  • Deterministic tick-based timeouts — precise, deterministic timeout mechanism essential for correct consensus behavior
  • Metadata snapshots — a three-layer snapshot system enabling state transfer between replicas for recovery and bootstrapping
  • Partitions module — extracted partition and segment logic into a dedicated crate, with a unified Partitions trait that abstracts both single-node and replicated operations behind the same interface
  • Pipeline decoupling — made VsrConsensus generic over the Pipeline trait, so implementations can be swapped and memory usage is reduced by keeping only headers in the pipeline
  • Message bus — internal message bus for inter-component communication
  • Simulator — initial version of a distributed system simulator for testing VSR behavior in a controlled, deterministic environment

LeftRight-Based Global Metadata

We've replaced the distributed slab ECS approach with a centralized LeftRight-based global metadata store. This is one of the largest refactoring efforts in this release — shards now read from a shared, lock-free snapshot while shard 0 exclusively handles writes. The result is consistent metadata across all shards without per-read synchronization, and a much simpler concurrency model that's easier to reason about and extend for clustering.

NUMA Awareness

The server now features NUMA awareness with configurable CPU affinity and memory binding per NUMA node. On multi-socket machines, this keeps each shard's memory local to its CPU, reducing cross-socket latency and improving cache efficiency — an important optimization for high-throughput deployments.

Socket Migration Across Shards

TCP socket migration eliminates cross-shard forwarding overhead. When a client connects to a shard that doesn't own its target partition, the socket is migrated to the correct shard instead of forwarding every request. This significantly reduces latency for workloads spread across multiple shards.

Embedded Web UI

The Web UI can now be embedded directly into the server binary, served at the /ui endpoint. No separate deployment needed — just start the server and open the dashboard. The feature is enabled by default and can be toggled via http.web_ui in the server config.

Cooperative Partition Reassignment

Cooperative partition reassignment for consumer groups eliminates message duplicates during rebalancing. Previously, partitions were moved instantly on join — the old consumer could still be mid-processing while the new one polled the same uncommitted offset. The new approach introduces a "pending revocation" phase where partitions are held until the committed offset catches up or a timeout is reached, ensuring clean handoff without duplicates.

Log Rotation

Configurable log rotation based on file size and retention policies prevents unbounded log growth in long-running deployments.

Breaking Changes

Security Improvements

  • Hardened deserializers — all command deserializers now use checked reads instead of direct byte indexing, preventing malformed packets from crashing the server
  • Password hashing before state log — fixed a critical bug where plaintext passwords were persisted to the state log, causing login failures after server restart
  • Auth required for stats — the /stats endpoint now requires authentication
  • Permission inheritance fixes — consolidated permissions into the metadata crate and fixed broken inheritance (e.g., manage_stream now correctly implies read_stream)

Notable Bug Fixes

Docker images are available on Docker Hub.

SDKs

All SDKs now have auto-publishing enabled — edge and RC versions are automatically published to their respective package registries on every merge.

  • Rust — improved high-level producer/consumer with ordered sharding, graceful shutdown(), and message loss prevention; fixed max_in_flight permit acquisition and client reconnection after disconnect
  • C# — .NET 10 support, DeleteSegments and GetSnapshot methods, TimeSpan-based expiry handling, source link for debugging
  • Java — fluent builder API (Iggy.tcpClientBuilder().blocking()), semantic exception hierarchy, BDD tests with Cucumber, user headers support, TLS for HTTP, memory leak detection in CI
  • Python — AsyncIterator interface for IggyConsumer, message_expiry parameter in create_topic, Windows build support
  • Node.js (TypeScript) — comprehensive JSDoc comments for improved API documentation and IDE support
  • Go — leader redirection for cluster failover handling

Connectors

New connectors and runtime improvements:

Learn more in the documentation.

Web UI

  • Embedded into the server binary (see above)
  • User headers now display decoded values based on their type (string, int, float, bool, etc.) with expandable rows for type details
  • Checksum displayed as hex alongside raw value
  • Fixed root user ID (now 0 instead of 1)
  • Fixed logout side-effect on page load
  • Refactored from SSR to SPA with client-side authentication
  • Benchmark dashboard upgraded to Yew 0.22

Helm Charts

CI/CD & Infrastructure

  • ARM64 builds — native ARM64 runners with shared Docker cache, aarch64 builds in pre-merge and release workflows
  • Edge releases — rolling pre-releases with versioned Docker tags (e.g., 0.6.1-edge.2) and binary tarballs
  • Auto-publish — all SDKs (Rust, C#, Java, Python, Node.js, Go) automatically publish edge/RC versions on merge
  • Code coverage — Codecov integration for Rust and Java
  • Pre-commit hooks — local validation for all SDKs matching CI tools and versions
  • Test harness — iggy_harness proc macro for consistent, boilerplate-free test infrastructure with crash detection

What's Next

Our primary focus remains completing the clustering implementation with VSR consensus. The foundations laid in 0.7.0 — view change, replication pipeline, metadata snapshots, partitions module — will come together into a fully functional replicated cluster in the upcoming releases.

Other upcoming efforts:

  • Server core optimizations: kTLS, DirectIO, and continued NUMA and socket migration tuning
  • SDK evolution with synchronous clients and sans-io architecture for better testability and portability
  • More connector integrations and runtime hardening
  • Web UI enhancements — building on the newly embedded dashboard
  • Agentic AI exploration for streaming-based agent coordination

Thanks to our amazing community and contributors for making Apache Iggy better with every release. A special welcome to our new committer Łukasz Zborek!

Join us on Discord and help shape the future of the project!