Framing

Every message on a binary transport is a fixed 256-byte header followed by an optional body. The header is a #[repr(C)] struct decoded by pointer cast (zero-copy), so field offsets are fixed and enforced at compile time in the server. Three header shapes cross the client boundary:

• RequestHeader: client to server.
• ReplyHeader: server to client, answers one request.
• EvictionHeader: server to client, session-terminal rejection with no body.

The header shape is identified by the command byte at offset 60. Values a client sends or receives:

Other command values (Prepare, PrepareOk, view-change traffic, and so on) are replica-to-replica consensus messages and never appear on a client connection. They are documented on the Server-to-server page.

The size field at offset 48 is the total frame length in bytes, header included. A body, when present, immediately follows the 256 header bytes and has length size - 256. Readers can decode size before typing the header. That offset is a protocol constant.

RequestHeader

Client to server. 256 bytes.

Operation discriminants

The operation byte tells the server which state-machine operation the request carries. Replicated operations are identified by operation alone. Non-replicated operations (reads, ping) all use operation = 2 and carry their concrete u32 command code in header bytes 196..200.

Values a client may send:

Value 0 is reserved and rejected. The 64..127 range is reserved for server-internal operations (currently 64 through 68 are assigned) and every value in it is refused from clients.

The planes matter for delivery semantics:

• Metadata operations replicate through the metadata consensus group. The server deduplicates them by (client, request) within its retained client history. Recent replies are cached and replayed; an older request can remain recognized as already applied after its reply has left the cache. This is not an unlimited exactly-once guarantee.
• Partition operations normally replicate through their partition's consensus group. Each partition retains client request watermarks and a 128-id committed window to suppress duplicates. There is no reply cache: a recognized duplicate gets an empty success, including no offset confirmations for a retried send. Capacity eviction or a new client identity loses this coverage. Single-replica consumer-offset writes with ack = 0 bypass consensus and this deduplication.
• Non-replicated operations bypass consensus and deduplication. A poll can still submit a separate consumer-offset write when auto-commit is enabled.

Request numbering

request is a per-client counter the server's client table tracks for metadata-plane operations:

• New metadata operations must use a request above the retained watermark; a retry reuses its original number and arguments.
• The Rust SDK advances one shared counter for metadata operations, partition operations, logout, and DeleteSegments. Non-replicated operations use the current value without advancing it.
• Register uses request = 0; other operations except NonReplicated require a non-zero request number.
• The counter is a watermark, not a contiguous sequence: any value above the last accepted one is admissible.

session is the fence epoch handed back by the login reply, derived from the committed Register log position. Echo it after login. Registering the same client again mints a higher epoch; the metadata client table rejects an older epoch. Partition dispatch uses the identity bound to the authenticated connection.

The partition window is measured in the shared client request-id space. An unseen request arriving 128 or more ids below that partition's watermark is treated as already committed and receives success without executing. Avoid allowing an unresolved write to fall outside that window; the protocol does not provide an unlimited reordered-retry guarantee.

ReplyHeader

Server to client. 256 bytes, followed by size - 256 bytes of body.

Decode order for a client:

1. Read 256 bytes, check command. 13 (Eviction) means the session is dead: map the eviction reason to an error and stop. 8 (Reply) continues.
2. Read the remaining size - 256 body bytes.
3. If status is nonzero, the request failed pre-commit. The status value is the IggyError code and the body is empty.
4. Otherwise decode the body. For result-framed operations, strip the result section first.

Result section

Replies for all metadata operations, including their server-internal rewrites, and for the partition-plane consumer-offset writes (StoreConsumerOffset, DeleteConsumerOffset) are result-framed. Non-empty Register replies use the same framing. The body starts with a result section ahead of the typed payload.

[count: u32]
count x { index: u32, result: u32 }

• Success: count = 0, and the typed response payload (if any) follows the 4 count bytes.
• Business or transient rejection: one entry { index: 0, result: error_code } and no payload.

A nonzero header status has an empty body. With status = 0, decode the result section for the operations above: it can report either a committed outcome or a pre-commit transient rejection. Its presence alone does not prove commitment. A successful login-register reply starts with count = 0, followed by the login response payload. An empty Register body is a terminal failure, not a successful login.

Replies to non-replicated commands aren't result-framed: after status = 0 the body is the response payload directly.

EvictionHeader

Server to client. 256 bytes, never a body. An eviction is session-terminal: it says "this session is dead", carries no per-request correlation, and a client should deinitialize and re-login.

Eviction reasons:

Value 0 is reserved and never sent.