Cluster Durability

Iggy uses Viewstamped Replication to commit partition operations. The topic's durability and consumer_offset_durability options select the storage guarantee required for message production and explicit offset changes, respectively. Both default independently to replicated. Both continue to write data to disk.

For the creation options and single-node behavior, see Durability.

What an acknowledgement means

For an operation that waits for completion:

The policy changes what a replica must retain before its acknowledgement can count toward commit. It does not replace quorum commit with a primary-only disk write.

Iggy's replication quorum is not a strict majority for every group size:

Except for two replicas, the replication quorum is min(ceil(n / 2), 3) and the view-change quorum is n - replication_quorum + 1. Two replicas require both for either quorum. The quorums therefore intersect. These are the implemented quorum rules, not a configurable acknowledgement count.

Replicated completion

With both policies set to replicated, partition prepares stay off the disk prepare-WAL path. Replicas retain the prepares in memory, and committed messages reach segment files through ordinary flush scheduling.

Success therefore does not prove that the operation reached stable storage on any replica. Replication protects against failures while sufficient peers retain the history and can form the quorums needed for recovery and progress. For example, a healthy three-replica group can continue after one replica fails.

Failure independence matters. A shared power loss, an OOM cascade, or a crash triggered on every replica can destroy multiple volatile copies at once. A process-only failure can lose messages still in the processes' journals, even while the kernels and their page caches remain alive. Replication count alone does not establish a bound on that loss.

Persisted completion

When either policy is persisted, each multi-replica partition uses a bounded on-disk prepare WAL. The WAL records full prepares, including message payloads. A prepare requiring persistence cannot release its PrepareOk until its history and durable frontier are recoverable.

Prepares can be forwarded while local persistence is pending. Once enough replicas have met the required barrier, the operation can commit and the primary can apply it and reply. An acknowledged message may still be in the prepare WAL rather than in a segment file; its recovery does not depend on first reaching a segment flush threshold.

The WAL also retains predecessors across message and offset operations. Consequently:

• durability=persisted does not make an offset response persisted when consumer_offset_durability=replicated.
• consumer_offset_durability=persisted with replicated messages still journals message payloads. A durable offset's predecessor history must remain recoverable.
• A shared barrier can also persist co-batched operations with the weaker policy. That incidental persistence does not strengthen what their earlier replicated acknowledgements promised.
• Only the replicated / replicated combination avoids the disk prepare WAL entirely.

WAL history is reclaimed only after the materialized segment and offset state needed to replace it has been synchronized. The server's [partition] wal_bytes_max setting, default 256 MiB, bounds active WAL and queued/in-flight prepare bytes per partition. Capacity pressure causes checkpointing and backpressure; it does not downgrade a persisted operation. Temporary rewrites need additional disk space.

Recovery and failure limits

A restarting replica loads segment and offset state and, when enabled, reconciles the durable prepare WAL with that state. It also restores durable consensus state and follows the recovery protocol before serving as a healthy replica. It does not treat every locally recovered prepare as committed.

Missing required history, checksum failures, or contradictory materialized state must not become an empty healthy partition. Recovery can fence a partition and repair it from peers. Storage errors withhold successful completion; a locally failed application of a committed operation fences the partition and initiates server shutdown.

persisted protects acknowledged operations across process and power failures when the required storage copies remain intact and the storage honors synchronization. Recovery and availability still require the protocol's quorums. It cannot protect against destruction of all durable copies, nor does it prevent configured retention or explicit deletion.

Choosing the policies

Use replicated when the workload accepts the risk of losing acknowledged operations after correlated failures in exchange for avoiding a required storage barrier at completion. Isolate replicas across failure domains; see Deployment.

Use durability=persisted when acknowledged messages must be recoverable after volatile copies are lost. Select consumer_offset_durability=persisted independently when explicit offset stores and deletes need the same guarantee. Ordinary flush thresholds can remain at their defaults.

Neither policy turns HTTP ack=none or a poll's auto-commit into an awaited durable result. See Which responses prove completion.

The storage mechanisms are implemented in the partition persistence worker and prepare journal. The crash-recovery tests exercise acknowledged persisted messages and offsets below ordinary flush thresholds, on both a singleton and a three-replica cluster.