Node.js SDK

The Iggy Node.js SDK is a client library that allows you to interact with the Iggy API from your Node.js and TypeScript applications. It communicates with the Iggy server over TCP or TLS using the binary protocol. The package is available on npm and the source code can be found on GitHub.

These docs target server 0.10.0 and Node SDK 0.11.0. The Node package 0.8.0 uses the older protocol. The current SDK speaks only the VSR wire protocol and doesn't fall back to older formats.

Installation

npm install apache-iggy@0.11.0

Quick start

The samples below expect an Iggy server on 127.0.0.1:8090. See Getting started for the server prerequisites:

docker run --rm \
  --cap-add=SYS_NICE --security-opt seccomp=unconfined --ulimit memlock=-1:-1 \
  -p 8090:8090 \
  -e IGGY_TCP_ADDRESS=0.0.0.0:8090 \
  -e IGGY_NODE_ADVERTISED_ADDRESS=localhost \
  -e IGGY_ROOT_USERNAME=iggy -e IGGY_ROOT_PASSWORD=iggy \
  apache/iggy:0.10.0

# Or from a 0.10.0 source checkout
cargo run --bin iggy-server -- --fresh --with-default-root-credentials

The environment variables make the server reachable through the published port (it binds to 127.0.0.1 inside the container by default) and set the root credentials the samples log in with. Without --with-default-root-credentials (or the IGGY_ROOT_USERNAME / IGGY_ROOT_PASSWORD variables), a fresh server refuses to start. For source runs, --fresh deletes existing local server data. Explicit credential environment variables take precedence over the default-root flag. Bootstrap settings do not replace credentials already stored in recovered data.

Save the following snippets as producer.mjs and consumer.mjs in the project where you installed the SDK. Run node producer.mjs before node consumer.mjs. A consumer group sample follows them.

Producer

import { Client, Partitioning } from 'apache-iggy';

const STREAM_NAME = 'sample-stream';
const TOPIC_NAME = 'sample-topic';

const client = new Client({
  transport: 'TCP',
  options: { port: 8090, host: '127.0.0.1' },
  credentials: { username: 'iggy', password: 'iggy' },
});

// Re-running this example is fine: only create what is missing.
const streams = await client.stream.list();
const stream =
  streams.find((s) => s.name === STREAM_NAME) ??
  (await client.stream.create({ name: STREAM_NAME }));

const topics = await client.topic.list({ streamId: stream.id });
const topic =
  topics.find((t) => t.name === TOPIC_NAME) ??
  (await client.topic.create({
    streamId: stream.id,
    name: TOPIC_NAME,
    partitionCount: 1,
    compressionAlgorithm: 1, // 1 = None, 2 = Gzip
  }));

const messages = Array.from({ length: 10 }, (_, i) => ({
  id: i + 1,
  headers: [],
  payload: `message-${i + 1}`,
}));

await client.message.send({
  streamId: stream.id,
  topicId: topic.id,
  messages,
  partition: Partitioning.Balanced,
});

console.log(`Sent ${messages.length} message(s)`);

await client.destroy();

The client resolves the target partition and routes requests to its primary. A primary can differ from the metadata leader.

Consumer

import { Client, CommitPolicy, PollingStrategy, Consumer } from 'apache-iggy';

const STREAM_NAME = 'sample-stream';
const TOPIC_NAME = 'sample-topic';
const PARTITION_ID = 0;

const client = new Client({
  transport: 'TCP',
  options: { port: 8090, host: '127.0.0.1' },
  credentials: { username: 'iggy', password: 'iggy' },
});

// Next starts after consumer 0's stored offset.
// Autocommit requests offset storage before application processing.
const polledMessages = await client.message.poll({
  streamId: STREAM_NAME,
  topicId: TOPIC_NAME,
  consumer: Consumer.Single,
  partitionId: PARTITION_ID,
  strategy: PollingStrategy.Next,
  options: { count: 10, commit: CommitPolicy.OnRead },
});

for (const message of polledMessages.messages) {
  const payload = message.payload.toString('utf8');
  console.log(`Offset: ${message.headers.offset}, Payload: ${payload}`);
}

await client.destroy();

The poll response does not wait for the auto-commit to finish. The sample processes messages after polling, so auto-commit does not confirm that the application handled each message.

Consumer group

A consumer group distributes partitions among its members. The high-level consumer creates and joins the group, then reads its assigned partitions. Save this as consumer-group.mjs:

import { Client } from 'apache-iggy';

const client = new Client('iggy://iggy:iggy@127.0.0.1:8090');
const consumer = await client.consumerGroup({
  name: 'sample-group',
  streamId: 'sample-stream',
  topicId: 'sample-topic',
});
try {
  for await (const received of consumer) {
    console.log(received.message.payload.toString('utf8'));
    await consumer.commit(received.checkpoint);
  }
} finally {
  await consumer.shutdown();
  await client.destroy();
}

This loop waits for messages until it is stopped. It stores checkpoints after printing. The group has its own offsets, separate from a standalone consumer.

Running the samples again

All three samples can be run again without changes:

• node producer.mjs finds the existing stream and topic and appends ten more messages.
• node consumer.mjs prints only the new messages, because PollingStrategy.Next continues after the stored offset. With nothing new to read, it prints nothing.
• node consumer-group.mjs does the same from the group's stored offset.

To start again with an empty server, stop the container and run the docker run command again. --rm removes the container and its data.

Configuration

The client constructor accepts a few options:

• reconnect: automatic reconnection with { enabled, interval, maxRetries }.
• heartbeatInterval: the client pings the server every 5 seconds by default. 0 disables heartbeats. When server heartbeat eviction is enabled, silent consumer-group members can be evicted, so keep it well below the server's heartbeat timeout.
• maxResponseFrameSize: response frames larger than this limit (default 64 MiB) are rejected and close the connection. Raise it when polling very large batches.
• poolSize: the client currently supports exactly one pooled connection. min/max above 1 throw.
• TLS: set transport: 'TLS' and pass Node tls.ConnectionOptions in options. See the tcp-tls example.

Beyond the basics

• Consumer groups: client.group manages groups (including ensure-and-join in one call), and client.message.reader or client.consumerGroup reads the assigned partitions. A one-shot poll names one partition.
• Consumer streams: A high-level consumer supports async iteration, handler-based consumption and toReadable().
• Administration: users, personal access tokens, offsets, partitions, segments, and cluster metadata are all available on the client.
• Errors: the package exports ResponseError, DeserializeError, ProtocolFrameError, and VsrEvictionError for precise error handling.

See the foreign/node README for the full API walkthrough.

Examples

Examples are available in the examples/node directory, written in TypeScript. See Examples for the list and how to run them.