Apache Iggy 0.9.0 Release

Release 0.9.0

We are proud to announce the release of Apache Iggy 0.9.0. This release marks a historical milestone for the project: it is our very first as an official Apache Top-Level Project following our graduation from the Incubator on August 19, 2026. This isn't just another version increment; it is the flagship release we have been relentlessly building toward over the past year. And we can’t wait for you to try it out.

The official release is the source archive distributed by the Apache Software Foundation, with its PGP signature, SHA-512 checksum and the project KEYS. The downloads page explains how to verify it. The GitHub release carries the full list of merged pull requests.

Iggy now runs as a cluster. 0.9.0 ships Viewstamped Replication Revisited (VSR) consensus in the standard iggy-server binary. Start three nodes, lose one, and the cluster keeps serving reads and writes. There is no separate build, feature flag, or clustered edition. The same server runs single-node or replicated, and cluster.enabled in the configuration decides which one you get.

This is the biggest release in the project's history: over 600 merged pull requests from 60 contributors, 34 of them contributing to Iggy for the first time. Alongside the server, we're releasing the Rust SDK 0.11.0, the CLI 0.14.0, the connectors runtime 0.5.0, the MCP server 0.5.0, the Web UI 0.4.0, the bench dashboard 0.8.0, and the Python 0.9.0, Node.js 0.10.0, Java 0.9.0, C# 0.9.0 and Go 0.9.0 SDKs, all from the same repository. Beyond clustering, 0.9.0 brings per-topic durability policies, an extensible topic options catalog, a new PHP SDK, eight new connectors and an HTTP webhook source, the foundation of a Kafka protocol gateway for existing Kafka clients, a redesigned benchmarks dashboard, cluster-aware Helm charts, and hundreds of fixes and performance improvements across the server, SDKs and connectors.

Before the changelog, two topics deserve their own space: clustering and performance.

Clustering with VSR consensus

For three releases the roadmap section of these posts has ended the same way: complete the clustering implementation with VSR consensus. 0.7.0 laid the consensus foundations. 0.8.0 added the shard crate, the plane abstractions, the persistent WAL journal, and the experimental iggy-server-ng binary. In 0.9.0 that work became the server. server-ng was promoted to the Apache Iggy server, the vsr feature flag is gone, the legacy server and its wire format were deleted, and every SDK and CI lane runs against the replicated server. Single-node deployment remains the default, and clustering is opt-in through the [cluster] section of the configuration.

What a cluster gives you

Apache Iggy replicates with Viewstamped Replication Revisited. VSR keeps an ordered state machine consistent across replicas and elects a new primary when the current one fails:

• Normal operation: the primary assigns an operation number, sends Prepare, and commits after a quorum answers PrepareOk.
• View change: when replicas stop hearing from the primary, they exchange StartViewChange and DoViewChange, and the new primary sends StartView. Failover needs no operator.
• Recovery: a restarted or lagging replica probes the current view and repairs the missing WAL ranges from its peers before it serves current state.

Three replicas tolerate one unavailable replica, five tolerate two. Three nodes is the recommended minimum for a highly available deployment. A two-node cluster is useful for development, but it cannot make progress after either node fails.

Replication is split by namespace rather than run as one global log:

Each group runs its own consensus and elects its own primary. Metadata writes route through the metadata primary, and partition writes route through the primary of that partition. Reads are served from the local replicated state on any node, followers included.

How it was built

The consensus work spans well over a hundred pull requests. The milestones, in roughly the order they landed:

• Consensus per partition and the consensus group mechanism made every partition its own replicated state machine
• Session registration with combined login and register, later reworked so sessions resume across restarts, backed by the client table for at-most-once request handling
• Async fire-and-forget message bus for consensus traffic, plus QUIC, TCP-TLS, WebSocket and WSS transports for clients
• Replica bootstrap and multi-shard cross-shard communication, so all shards of a node take part in consensus
• Partition reconciliation loop, partition background tasks, and replica rejoin via view probe and message repair
• Metadata VSR state persisted in a durable superblock, state transfer for the metadata snapshot and client table, and state transfer for partitions, which let a wiped replica refill itself from its peers
• Committed sends confirmed with partition and offset, so producers learn exactly where their batch landed
• Deduplication of partition writes with per-group client table slices, so a produce retried after a timeout does not write twice
• Follower HTTP requests forwarded to the primary, so any node can answer any control-plane request
• Inbox split into consensus and client-reply lanes, so a burst of client traffic cannot starve replication
• Metadata reads served at or above the client's own writes

Then came the correctness fixes found by the simulator, the crash tests and real cluster runs:

• View change no longer discards committed ops
• A replica is kept off a hole in its committed prefix
• Repaired prepares can't rewind the WAL parent
• The dedup fence survives capacity eviction
• Writes recover after a fast primary rejoin
• A late partition materialiser can't wedge its group
• Partition and metadata repair are driven from the tick
• An unservable header counts as a nack
• Frames no plane claims are refused
• Torn segment tails are truncated instead of resurrected
• Spurious heartbeat elections are prevented, the last change before the tag

Durability you choose per topic

Replication forced a precise answer to a simple question: what does an acknowledgement mean? 0.9.0 answers it with two create-only topic options, introduced in let topics require durable acks:

durability governs message production and consumer_offset_durability governs explicit offset stores and deletes. They default independently and neither inherits the other. Both policies write data to disk. With persisted, each multi-replica partition keeps a bounded on-disk prepare WAL, sized by partition.wal_bytes_max (256 MiB by default), and a replica cannot release its PrepareOk until its message bodies, WAL history and durable frontier are recoverable. The old enforce_fsync topic option and the server-wide consumer_offset_enforce_fsync setting are gone. Every SDK, the CLI, the HTTP API and iggy-bench expose both options.

The details are in Durability and Cluster Durability.

Secure replica traffic

The replica-to-replica port is authenticated and, optionally, encrypted:

• PSK + BLAKE3 handshake: every peer proves possession of a cluster-wide pre-shared key of at least 32 bytes through a three-message keyed-MAC handshake before it can register as a replica. A previous_shared_secret enables rolling key rotation without an authentication outage.
• TLS 1.3 on the replica plane with PSK channel binding: the opt-in [cluster.tls] table wraps replica connections in TLS 1.3 (ALPN iggy-replica) with CA and self-signed modes. The PSK MAC absorbs the TLS exporter value, so a relay that terminates both legs fails the handshake.
• Cluster metadata is auth-gated and registration is forwarded to the primary: unauthenticated clients can no longer read the roster, and a client that dials a backup completes its login there. Credentials never cross the replica interconnect.
• Incompatible clients are rejected at login via protocol version

See Cluster Security.

Client failover

Every SDK speaks the VSR wire protocol and can connect to any node. Leader-aware clients fetch cluster metadata after login, reconnect to the advertised leader when they landed on a follower, poll for a new leader through an election, and cap the number of redirects so a flapping roster cannot bounce them forever:

• Rust: fail over to a surviving node when the current one dies, rejoin consumer groups after membership loss, heartbeat handle
• Java: cluster metadata and leader redirection in the TCP client
• C#: leader redirection BDD scenario
• Rust, Go, C# and Java run the shared leader_redirection BDD suite against a real cluster

Writes return after the required VSR commit, or return a retryable error while a replica is changing view or catching up. TransientNotAccepted proves the request was never admitted and permits a retry on another node. TransientNotCommitted has an uncertain outcome and is replayed with the same session and request identity, and the server-side dedup takes care of the rest. Details in Client Failover.

Per-node advertised addresses and per-client-network address selectors let a node publish different endpoints to clients on different networks, which matters for containers and cloud deployments.

Running a cluster

Every node loads the same configuration and is told apart only by --replica-id. Save this configuration as iggy-vsr.toml:

[cluster]
enabled = true
name = "iggy-vsr-dev"

[cluster.auth]
enabled = true
shared_secret = ""

[[cluster.nodes]]
name = "iggy-node-1"
ip = "127.0.0.1"
replica_id = 0
ports = { tcp = 8090, quic = 8080, http = 3000, websocket = 8093, tcp_replica = 9090 }

[[cluster.nodes]]
name = "iggy-node-2"
ip = "127.0.0.1"
replica_id = 1
ports = { tcp = 8091, quic = 8081, http = 3001, websocket = 8094, tcp_replica = 9091 }

[[cluster.nodes]]
name = "iggy-node-3"
ip = "127.0.0.1"
replica_id = 2
ports = { tcp = 8092, quic = 8082, http = 3002, websocket = 8095, tcp_replica = 9092 }

Set the shared configuration path, root credentials and replica authentication secret in each terminal. Use the same credentials and secret for all three nodes. Root credentials are required on the first boot, and the secret must contain at least 32 bytes:

export IGGY_CONFIG_PATH="$PWD/iggy-vsr.toml"
export IGGY_ROOT_USERNAME=iggy
export IGGY_ROOT_PASSWORD="replace-with-your-root-password"
export IGGY_CLUSTER_AUTH_SHARED_SECRET="replace-with-at-least-32-random-bytes"

Start each node in a separate terminal:

IGGY_PATH=local_data/node-0 iggy-server --replica-id 0
IGGY_PATH=local_data/node-1 iggy-server --replica-id 1
IGGY_PATH=local_data/node-2 iggy-server --replica-id 2

Every roster field can also be set through typed IGGY_CLUSTER_* environment variables, which is what the new Helm chart cluster mode renders. The Deploy a Cluster guide walks through the whole setup, and the Configuration reference covers every [cluster] key, including consensus timing (heartbeat_timeout, view change and repair intervals, all on a fixed 10ms tick).

Testing a consensus protocol

We trust this implementation because of how much effort went into breaking it. Most of that effort is deterministic simulation:

• Seed-based workload fuzz harness and the workload fuzzer with consensus oracles: randomized, fully seed-deterministic workloads run against a predictive shadow, crash replicas, and assert consensus invariants on every tick and at quiesce. Every failure reproduces from its seed.
• Deterministic future executor and server dispatch shell: the real server dispatch code runs inside the simulator
• Workloads under network and replica faults: message loss, delays, partitions, crashes with stability windows, client resends with target rotation. This one alone found two real bugs before any user could.
• Durability failures no existing test could reach: a storage model injects crashes, power loss and torn writes into the WAL tests
• SIGKILL crash-durability cluster tests and client table durability across node restart on real processes
• On-disk storage format breaks caught pre-merge: a baseline data directory boots against every pull request
• Miri undefined behavior detection for the binary_protocol and consensus crates
• Cross-SDK BDD suites running against a real two-node cluster

The Viewstamped Replication page is the entry point to all of this.

Performance

Efficiency has always been the point of Iggy: thread-per-core, shared-nothing, io_uring, and no copies where the format allows it. 0.9.0 carried that into the replicated server and shipped a series of hot-path improvements:

• Poll replies served without copying record bytes: a poll reply used to be three copies plus a memset, roughly four times the payload in memory traffic. It is now a 272-byte head fragment followed by the journal or disk fragments as they are, written with a single writev.
• Persisted-mode disk traffic per acknowledgment cut: publishing the WAL frontier took three barriers on three inodes per batch. It is now two slots in one file, overwritten in place, and the body and WAL barriers land in one journal window.
• Log and index persisted concurrently: the two serialized fdatasync calls per flush became one join. On the fsync-gated benchmark used in that pull request, throughput went from 13.4k to 26.1k msg/s and p50 from 5.83 to 3.01 ms.
• Batch checksum computed in a single produce pass, user headers decoded without copying the blob, message header written directly into the target buffer
• Sealed-segment read handles capped with a per-partition LRU, and the active segment no longer pays one openat per poll
• 16 MiB per active segment index no longer reserved, consumer offset state bounded per partition
• Unnecessary backpressure under concurrent clients eliminated, produce and poll overhead reduced in cluster mode
• TCP-TLS and WSS connections distributed across shards
• mimalloc restored as the global allocator
• CPU affinity and cgroup limits respected in shards and stats, so a container gets the shard count it was given
• Foreign SDKs generate message ids from random bytes instead of UUIDs, and the Go SDK pools its request-path wire payload

Benchmarks: 0.9.0 and what comes next

We're not done. A dedicated performance effort started right after the 0.9.0 code freeze, and the first results are in. The tables below compare the 0.9.0 release against a development snapshot, both measured with iggy-bench on the same host with identical workload parameters. Each version ran its own build of iggy-bench, because the development client sends the new deferred poll. These results therefore compare the server and client together. Each cell reads 0.9.0 → next, where next is the development snapshot. Latencies are in milliseconds. p50 is the median, p99 the 99th percentile, and p999 the 99.9th percentile.

Durability legend

• Replicated (durability=replicated, the default for producer rows without a persisted label): a successful write waits for quorum commit and local application, but does not wait for stable storage. Acknowledged data can still be in memory.
• Persisted (durability=persisted): a successful write also requires recoverable copies on stable storage at the replication quorum. On a three-node cluster, that means at least two replicas.

Both policies write data to disk. On a single node, the quorum is one: replicated provides no extra copy, while persisted waits for local stable storage. These labels describe producer acknowledgements. Consumer rows measure reads. See Durability for the full guarantees.

Workload legend

The comparisons below cover rate-limited workloads.

• Pinned producer / consumer: each actor owns its own stream and partition. Actors counts concurrent producers or consumers.
• 800 MB/s / 500 MB/s: the configured aggregate rate limit. Achieved MB/s is the measured throughput, which can be lower. Compare latency alongside achieved throughput.
• Cold / warm: consumers read uncached data or re-read cached data, respectively.

The tails are where the work shows. On a three-node cluster with persisted topics, the p99 of a 20-producer workload dropped from 7.07 ms to 2.94 ms and the p999 from 12.4 ms to 3.9 ms. The single-producer p999 on the same cluster went from 10.9 ms to 1.8 ms. On a single node, the 20-producer p999 fell from 11.1 ms to 3.0 ms. Cold consumer reads improved on both setups. In this development snapshot, 20 persisted producers on a three-node cluster achieved 800 MB/s with a median latency of 2.243 ms.

Three changes behind the numbers

Three changes are in flight, and the tables above measure them together against 0.9.0. These runs do not isolate how much each change contributes. A fourth is on the drawing board.

Consumers stop asking for nothing

A polling consumer pays a round trip for nothing on an idle topic, and on a busy one its interval trades log latency against empty replies.

pollMessagesDeferred (commands 105 and 106, tracked in #3470) appends a 24-byte trailer to the poll request: a maximum wait, a minimum message count, a byte cap for the reply, and a total request budget. The server parks the request and answers when the minimum is readable, the cap is reached, or the wait expires. The Rust client speaks it on every transport, the Java client over TCP and HTTP, and the Rust consumer now defaults to bounded long polling with manual commits. It answers a community request for deferred responses.

File I/O leaves the shard loop

A shard owns many partitions and used to wait for each one's file writes and durability barriers before serving the next, so one slow disk delayed every partition on the shard.

Now each file job runs as its own task, up to 16 in flight by default, and its result returns to the shard for identity matching and ordered publication. Storage work now overlaps with network work.

Replica links read ahead

Every pair of nodes shares one TCP connection for consensus traffic, owned by a single link shard. Reading one frame took two io_uring reads, one for the 256-byte header and one for the body, and frames already in the kernel receive queue waited for the next call. Under load that shard bottlenecked the cluster write path.

A 256 KiB read-ahead buffer (#4224) now serves a whole burst with one socket read. A frame at least as large as the buffer skips it and lands in its own allocation, so large messages pay no extra copy.

The next one: more replica links

The read-ahead makes each read cheaper, but every partition group a pair of nodes replicates still goes through one link shard.

The next change gives a peer several links, spread over shards. An operator picks the link count per peer, each link carries a contiguous range of shards, and the default stays at today's one. The upgrade rolls, but changing the link count takes a coordinated restart, and nodes that disagree on it refuse each other.

This one is a design, not code, and the read-ahead has to justify it first. More links only make sense when the link shard stays saturated behind the buffer. Otherwise the cheaper fix has already won.

All three measured changes are still in development, and every number above will move before they ship. A driver for the OpenMessaging Benchmark is in progress as well, and its results will follow.

Beyond these, in the order we expect them to land:

• Explicit group commit: several acknowledgements share one disk barrier by design, rather than by whatever happened to queue up during the previous one
• Direct I/O for the storage engine, writing past the page cache instead of paying for fsync barriers
• Multi-leader clusters, further out: a leader per partition, with leaders spread across all nodes of the cluster, so write traffic and replication work are shared by every machine instead of concentrating on one

And a long tail of smaller changes in the same direction. Efficiency is the reason Iggy exists, and it stays the priority now that clustering is in.

A like-for-like comparison of 0.8.0 against 0.9.0 with this parameter set is planned for the benchmarking platform, together with the final numbers for the next release. The benchmarking guide has the commands to reproduce these workloads on your own hardware, and the Linux tuning guide covers the host preparation.

Iggy Server

One server, one wire protocol

• server-ng promoted to the Apache Iggy server: the thread-per-core, io_uring server is now the only server. The iggy-server binary, the server package, the configs tree and the systemd integration are the former server-ng, and all transports are kept: TCP with TLS, QUIC, WebSocket and HTTP. The legacy server, its compat lane and the classic TCP, QUIC and WebSocket framing were deleted (BREAKING).
• Legacy message wire format and v2 bridge APIs removed from the SDK. The new message wire format is encoded on the client side (BREAKING).
• Namespace removed from client headers, a smaller header on every request
• Server-only types extracted into server_common, shard allocator extracted into cpu_allocation, compio executor and io_uring diagnostics extracted into a shared crate, the server's module graph turned into an enforced DAG, and the dispatch spine split into plane-named leaves

Topic options

• Extensible key-value options for topics, streams and users (BREAKING): retention, durability and segment layout are per-topic decisions made at creation. segment_size, messages_required_to_save, size_of_messages_required_to_save and the new preallocate_segments moved from server-wide [system.*] keys to the topic. Unknown keys are rejected at the edge, never silently ignored, and iggy options topic (or GET /options/topic) lists what the server accepts. The server refuses to boot on the relocated configuration keys. See Topic Options.
• Durability policies as described above (BREAKING: enforce_fsync is rejected, [system] is flattened into the root, and IGGY_SYSTEM_* variables lose the SYSTEM_ segment)

Storage and recovery

• Log and index persisted concurrently, recovery hardened: an index is derived from the log and is never evidence about the log. Recovery drops a divergent index whole and rebuilds it from a byte-0 walk, checksumming every batch it accepts. A local persist failure now stops the whole server instead of silently killing one shard.
• Torn segment tails truncated instead of resurrected, WAL suffix truncated above the state-transfer floor, recovery slot collisions rejected
• Offsets reserved before they are confirmed, polls completed on the owning shard, producer batches preserved across replication
• Data persisted on VSR backups, with a data integrity test
• Deleted partitions and topics rolled out of parent stats, namespace caps enforced at admission
• Expired personal access token cleaner

HTTP, security and operations

• Cluster-correct HTTP listener with per-operation RBAC, HTTPS on the REST listener, CORS honored, Web UI served by the server
• PAT limits enforced and Prometheus metrics exposed
• Wildcard bind without an advertised address rejected (BREAKING): a 0.0.0.0 bind says where a node listens, not where a client can reach it. Deployments that bind a wildcard must set node.advertised_address. The Helm chart derives it from the Service DNS name.
• Trusted-issuer A2A JWT verification with SDK token refresh, message encryption in the replicated server
• read_servers permission required for the system snapshot, get_user self-read restored without read_users
• systemd watchdog integration for the server and the MCP server
• Listener, auth, shutdown and disk-poll paths hardened, configuration surface hardened, and the server config cleaned up after a full field audit
• Sibling IGGY environment variables allowed, deterministic environment variable name suggestions
• io_uring panic from an unsupported opcode diagnosed, shard startup kept off blocking fallbacks
• Partitions distributed evenly in cooperative rebalancing
• Non-numeric filenames in consumer offset directories skipped instead of panicking
• Iggy banner on server startup
• Rust toolchain moved to 1.98, with an MSRV of 1.95 declared and enforced in CI

Breaking changes

This is a major upgrade. Read this list before moving an existing deployment:

• Data directory: the on-disk format changed, and there is no migration of 0.8.x data directories. Until 1.0.0, an upgrade means starting from a fresh data directory.
• Wire protocol: VSR framing is the only binary wire protocol. 0.8.x clients cannot talk to a 0.9.0 server and vice versa. Upgrade the server and the SDKs together.
• Legacy message wire format and v2 bridge APIs removed from the Rust SDK
• Server configuration: the [cluster] and sharding sections were reworked. Review your config.toml.
• Topic options: segment_size, messages_required_to_save, size_of_messages_required_to_save and related keys are per-topic. The server refuses to boot on the old [system.*] keys.
• Durability: enforce_fsync and consumer_offset_enforce_fsync are rejected. [system] is flattened into the root of the config and IGGY_SYSTEM_* variables drop SYSTEM_. The HTTP Iggy-Durability header reports replicated or persisted.
• Cluster metadata requires authentication: PING is the only pre-auth command
• Wildcard bind requires an advertised address, and cluster.nodes[*].ip must be a literal IP
• Connector SDK ConnectivityConfig replaced by RetryPolicy (Rust API only, built plugins are unaffected)

The release is available as a signed source archive from the Apache Software Foundation. Docker images are available on Docker Hub: apache/iggy:0.9.0 for the server and apache/iggy-connect:0.5.0 for the connectors runtime.

SDKs

Every SDK migrated to the VSR wire protocol and runs its test suite against the replicated server only. All of them expose the new durability options and a raw command API for custom commands.

Rust

• Rust client bumped to 0.11.0
• VSR header framing, the client side of the new protocol
• Fail over to a surviving node when the current one dies, rejoin consumer groups after membership loss, late offset stores no longer hit a left group
• Raw requests for custom commands and unknown command codes forwarded instead of rejected, so the protocol can be extended without forking the client
• Consumer group polling and commits corrected in IggyConsumer, consumer progress and error handling preserved, stream terminates after consumer shutdown
• max_buffer_size respected when merging batches, error callback awaits shard flush on producer shutdown
• NonZeroIggyDuration for retries and heartbeats
• Documentation for IggyClient, IggyProducer and IggyConsumer

PHP (new)

A brand new SDK. The PHP SDK is a native PHP extension built in Rust with ext-php-rs. It wraps the Rust SDK and exposes a synchronous Iggy\Client covering streams, topics, messages and consumer groups. It maps Iggy errors to typed exceptions, ships a consumer message iterator and examples, and runs a BDD suite with TLS tests in CI. It is defined as the apache/iggy-php composer package, is not yet published to a package manager, and is built from source today. See the PHP SDK documentation.

Python

Python received more pull requests than any other SDK in this release and closed most of its API gap with the Rust client:

• High-level producer API and message partitioning strategies
• TCP, QUIC, HTTP and WebSocket transport configuration
• Stream and topic management: stream listing, update, delete, purge, topic listing, update, delete, purge, remaining Topic fields and partitions, partition management
• Consumer groups: create and get, delete, join and leave, poll_messages takes a consumer
• Users: user management, permissions and remaining auth methods, update_user options
• User headers and origin timestamp, get_stats
• ruff configured, modern coding standards, pyrefly type checking, Windows wheels restored

Java

• TCP client migrated to the VSR wire protocol
• Cluster metadata and leader redirection
• TCP clients run one I/O thread or share a group
• Wire strings sized by UTF-8 byte length
• Java benchmark: CLI and resource provisioner, actor and data batch generator, reporting suite
• Pinot connector E2E coverage

C#

• Migrated to the VSR wire protocol, with the TCP connection cleaned up afterwards
• Heartbeat mechanism
• Rented message polling and rented payloads for fewer allocations
• Encryption moved to the raw client, missing metrics in StatsResponse, integer properties made unsigned
• UTF-8 byte count for string validation, OS default socket buffer size

Go

• Migrated to the VSR wire protocol
• context.Context on client methods
• Transport state separated from session state, structured logger aligned with the Rust SDK
• Request-path wire payload pooled, delete segments
• Fixed bounds checking for payloads over 64 KB, S2 decode returning an error instead of panicking, teardown of only the failed connection, MaxPayloadSize raised to 64 MB

Node.js (TypeScript)

• VSR framing and classic framing dropped
• Connection string syntax
• TLS with VSR, scoped permissions wire codec, tokens no longer dropped in list responses
• npm prereleases published under a channel tag, not latest
• Node.js client bumped to 0.10.0

C++

The low-level bindings from 0.8.0 grew into a client:

• Messaging FFI functions, system functions, consumer groups and topics, offsets, client lifecycle and message headers
• User management and an initial high-level client, extended with stream, topic and partition functions
• BDD tests for basic messaging, Bazel upgraded to 9.1.1

CLI

• CLI bumped to 0.14.0, with VSR support
• context show and session status commands
• Pure-Rust zbus keyring instead of libdbus, one system dependency fewer
• Bounded ping retries, CLI input validated

Connectors

The connectors runtime, now 0.5.0, gained eight new sinks, a new source, a new payload format, and a long list of reliability fixes.

New connectors

• S3 sink
• ClickHouse sink
• Delta Lake sink
• Apache Doris sink, with in-request retries and an opt-in CSV output format
• Meilisearch sink
• SurrealDB sink
• Redshift sink
• RabbitMQ sink
• HTTP source webhook gateway: Iggy can now receive webhooks directly. One listener serves many providers, each routed to its own topic, with bearer or HMAC authentication, secret URL endpoints, and a token-guarded admin API to register, re-key and revoke endpoints at runtime. Delivery is best-effort, and the connector's README says so up front.
• InfluxDB v2 and v3 connectors rebuilt as separate source and sink crates

Runtime and SDK

• Avro payload support with separate encoder, decoder and transform crates
• unwrap_envelope transform and envelope detection, fixing source-to-sink format mismatches
• Source batch acknowledgments: a source's checkpoint is saved only after Iggy accepted the batch. Source checkpoints are deferred and sink failures surfaced instead of swallowed, and the PostgreSQL source defers progress until ack.
• Source state stored on an HTTP state server as an alternative to a local file
• Agent docs, per-batch observability and atomic state
• Shared retry_async helper, replacing seven hand-rolled retry loops that all waited twice the configured delay before the first retry (BREAKING for the Rust connector SDK: ConnectivityConfig became RetryPolicy)
• Per-connector init failures isolated, duplicate iggy_sink_open and iggy_source_open rejected, connector key validated before it becomes a path, warning when the runtime API is exposed without a key
• Sink writes and startup readiness validated, failures reported and format conversion corrected
• Distinct error variants instead of an overloaded InvalidRecord, FlatBuffer conversions no longer fail silently, protobuf field lengths validated consistently
• Fixed PostgreSQL CDC producing no messages, Iceberg sink partitions on iceberg 0.10, InfluxDB v3 cursor rollback, doubled _total on counter metrics
• Flink sink uses the TCP client
• CPU and cgroup limits respected in connector stats

Learn more in the connectors documentation.

Kafka gateway

One of the questions we hear most often is some form of "can I point my Kafka clients at Iggy?". Our answer has two parts. Iggy is not built on Kafka and will not reimplement the Kafka protocol inside the server: the Iggy wire protocol, storage engine and replication are their own design, and that is where the performance comes from. Interoperability instead lives in a separate gateway process that speaks the Kafka wire protocol on one side and Iggy's protocol on the other, so existing Kafka producers and consumers can reach Iggy streams without code changes and without touching the server's own wire surface.

0.9.0 ships the first layer of that gateway. The Kafka wire protocol gateway is a new gateways/kafka workspace crate: a TCP listener on the Kafka port that decodes requests, enforces a version firewall, and answers ApiVersions, Metadata, Produce, Fetch, ListOffsets and CreateTopics, backed by 184 regression tests over golden wire fixtures. The community-driven rollout plan is public in the Kafka to Iggy bridge discussion and tracked in the protocol parity issue: the bridge from Produce and Fetch into Iggy streams comes first, then consumer groups and offset management, then the admin and authentication APIs. Transactions will answer with an unsupported error until Iggy itself has transactional writes.

To be clear about the current state: in 0.9.0 this is the foundation layer, not a working bridge yet. No API persists or reads Iggy data. Produce, Fetch and ListOffsets return a retriable error so Kafka clients keep their data and retry, and CreateTopics does not create topics. The work has not stopped at the tag, though. The bridge core, an Iggy SDK client with stream and topic mapping, provisioning, high watermarks and an Iggy-to-Kafka error map, merged the day after the 0.9.0 code was cut, and pull requests for mapping Kafka records to Iggy messages and SASL/PLAIN authentication are in review. Wiring Produce and Fetch through the bridge is next, the work is happening in the open, and contributions are very welcome.

Benchmark Dashboard

• Redesigned dashboard with a landing page and compare mode: the benchmarking platform now opens on a showcase pick and compares any two runs side by side
• Unified benchmarks list and best-pick selection, mobile layout and compare URLs fixed
• Cluster topology recorded in reports and surfaced in the UI, so a result says whether it came from one node or three
• iggy-bench exposes both durability policies as --durability and --consumer-offset-durability flags
• Bench dashboard bumped to 0.8.0

Web UI and MCP

• Web UI bumped to 0.4.0 and served directly by the server
• Explicit CSS import for the latest Vite build, frontend dependencies updated
• MCP server bumped to 0.5.0, runs against the replicated server and supports the systemd watchdog

Helm Charts

• Cluster deployment with one release per node: server.cluster mirrors the server's [[cluster.nodes]] roster one for one and renders into IGGY_CLUSTER_* variables plus --replica-id. Each node is its own release overriding only selfReplicaId, with a three-node example to start from. Roster mistakes fail at render time. server.replicaCount > 1 and autoscaling are now refused, because three independent servers behind one Service sharing a PVC was never a cluster.
• Fixed QUIC published as TCP, a missing WebSocket port, Kubernetes service links leaking IGGY_* variables into the server config, and the default image moved off 0.7.0
• Gateway API and Envoy Gateway replace ingress-nginx in the smoke cluster

CI/CD & Infrastructure

• Incubating references removed after TLP graduation
• SECURITY.md with the ASF reporting process, package tokens replaced with OIDC, read-only permissions declared on reusable workflows
• Miri undefined behavior detector for binary_protocol and consensus, build fails on rustdoc and rustc warnings
• Integration tests run when a binary they launch changes, a plugin's integration suite runs when it changes, Docker :edge refresh gated by the cargo-rail DAG
• HawkEye replaces addlicense for license header checks, later migrated to v7
• PR triage automation: review state labels via slash commands, /pin and /unpin, welcome comment, auto-assign volunteering issue authors, fork-PR commands via workflow_run handoff, and a stale bot that leaves PRs waiting on review alone
• AI assistance expectations added to CONTRIBUTING and a repo-wide team-review agent skill
• Devcontainer for reproducible development
• Most triage and status jobs moved to ARM runners, runner disk cleanup shared across PHP, Rust, BDD and coverage, apt-get bounded so a dead mirror cannot wedge a job
• BDD runners fail on undefined steps, stream CRUD coverage added, integration suites duplicating existing coverage removed
• Source release tarball includes gateways, DEPENDENCIES.md dropped per ASF policy

What's Next

Clustering is in. The next releases are about making it faster and complete:

• Performance: the effort behind the benchmark tables above continues. Deferred polling so consumers stop paying for empty round trips, a task per read and write for more I/O concurrency, more replica links per node pair, explicit group commit, Direct I/O instead of fsync, and more copy-free paths of the kind 0.9.0 started. Further out, multi-leader clusters with a leader per partition spread across all nodes. A full 0.8.0 vs 0.9.0 vs next comparison is planned for the benchmarking platform.
• Kafka gateway: the bridge from the protocol listener into Iggy streams, so Kafka producers and consumers can talk to Iggy through the gateway. Consumer groups follow, as laid out in the rollout discussion.
• SDK parity: leader redirection coverage for Python, Node.js and C++, and the PHP SDK on its way to a published package
• Connectors: more sources, bounded backpressure end to end, and runtime hardening
• Agentic AI: building on the A2A support and the MCP server

Thanks to our amazing community and contributors for making Apache Iggy better with every release. Sixty people contributed to 0.9.0, and 34 of them landed their first Iggy pull request in this cycle. Welcome aboard.

Join us on Discord and help shape the future of the project!