Benchmarking

Benchmarks should be the first-class citizens. We believe that performance is crucial for any system, and we strive to provide the best possible performance for our users. Please check, why we believe that the transparent benchmarking is so important.

We've also built the benchmarking platform where anyone can upload the benchmarks and compare the results with others. This is the another open-source project available here.

Iggy comes with a built-in benchmarking tool, iggy-bench. It's written in Rust and uses the tokio runtime for asynchronous I/O, mimicking the example client applications, so you can use it to estimate the performance of the server in your environment. It is part of the core repository and lives in the core/bench directory.

Running benchmarks

First build the project in release mode:

cargo build --release

iggy-bench doesn't start a server. It only connects, and fails with a connection error when nothing is listening. Start iggy-server yourself first:

cargo run --bin iggy-server -r

or use the repo's driver script, which builds, starts the server, runs a send and a poll benchmark, and shuts the server down:

./scripts/run-benches.sh

With a server running, invoke the benchmark kind and transport of your choice. The tool creates the streams, topics, and partitions it needs, then sends or polls messages.

Benchmark kinds

Eight kinds are available, each with a short alias:

1. Pinned producer (pp): N producers, each sending to its own stream-topic with a single partition:

   cargo r --bin iggy-bench -r -- pinned-producer tcp

2. Pinned consumer (pc): N consumers, each polling its own stream-topic with a single partition:

   cargo r --bin iggy-bench -r -- pinned-consumer tcp

3. Pinned producer and consumer (ppc), in parallel:

   cargo r --bin iggy-bench -r -- pinned-producer-and-consumer tcp

4. Balanced producer (bp): N producers sending to M partitions in K streams with balanced partitioning:

   cargo r --bin iggy-bench -r -- balanced-producer tcp

5. Balanced consumer group (bcg): N consumers assigned to M consumer groups polling from K partitions:

   cargo r --bin iggy-bench -r -- balanced-consumer-group tcp

6. Balanced producer and consumer group (bpcg), in parallel:

   cargo r --bin iggy-bench -r -- balanced-producer-and-consumer-group tcp

7. End-to-end producing consumer (e2e): each task produces and then consumes its own messages, measuring the full round trip:

   cargo r --bin iggy-bench -r -- end-to-end-producing-consumer tcp

8. End-to-end producing consumer group (e2ecg): N producing consumers assigned to M consumer groups:

   cargo r --bin iggy-bench -r -- end-to-end-producing-consumer-group tcp

Transports

Every kind runs over one of four transports: tcp, quic, http, or websocket (alias ws). Each transport subcommand takes --server-address for a non-default server, for example:

cargo r --bin iggy-bench -r -- pinned-producer tcp --server-address 0.0.0.0:8090

Message count, batch size, message size, and the number of producers/consumers/streams are all flags on the kind subcommand. iggy-bench --help and iggy-bench examples show the full surface.

Storing and comparing results

Each transport subcommand accepts a trailing output subcommand that persists the run (raw JSON data and charts):

cargo r --bin iggy-bench -r -- pinned-producer tcp output -o performance_results --identifier my-host

-o/--output-dir defaults to performance_results. Optional flags (--remark, --gitref, extra info) annotate the run. You can inspect persisted results with the in-repo report and runner crates (core/bench/report, core/bench/runner), and browse or compare them in the dashboard (core/bench/dashboard), which also powers the public benchmarking platform. A prebuilt dashboard image is available: docker pull apache/iggy-bench-dashboard.

Performance

The server is thread-per-core and shared-nothing, built on io_uring (via compio), with shard and CPU pinning configurable under [system.sharding]. Throughput and latency depend heavily on hardware, transport, and payload shape (messages-per-batch * message-size). Run the benchmarks on your own hardware, or browse current, dated results on the benchmarking platform.