Kafka vs WarpStream Configuration Reference

The purpose of this page is to compare configuration options in Kafka and WarpStream 1:1 as applicable.

The primary goal of this page is to reassure experienced Kafka operators migrating to WarpStream that most of the configuration and settings they're accustomed to tuning and managing in Apache Kafka have no equivalent in WarpStream and don't need to be managed at all due to WarpStream's architecture.

group.max.session.timeout.ms	Not supported.
auto.create.topics.enable	Same. Managed via Kafka API or WarpStream console.
auto.leader.rebalance.enable	N/A. WarpStream Agents do not have leaders.
controlled.shutdown.enable	WarpStream Agents accept a "gracefulShutdownDuration" flag that defaults to 80 seconds.
controlled.shutdown.max.retries	N/A. Nothing can go wrong during an Agent clean shutdown because they're stateless.
default.replication.factor	N/A. Doesn't apply to WarpStream because the Agents are stateless and data is stored durably in object storage at all times. Response is hard-coded to 3 to signify high durability.
delete.topic.enable	Same, set as a cluster level configuration via the Kafka API.
kafka.http.metrics.host	Not supported.
kafka.http.metrics.port	Agent flag "httpPort" to expose Prometheus metrics.
kafka.log4j.dir	Not supported.
kerberos.auth.enable	N/A
leader.imbalance.check.interval.seconds	N/A. WarpStream Agents do not have leaders.
leader.imbalance.per.broker.percentage	N/A. WarpStream Agents do not have leaders.
log.cleaner.dedupe.buffer.size	Not supported. WarpStream Agents have an equivalent to this setting internally, but its not configurable by default.
log.cleaner.delete.retention.ms	Topic-level config not supported. Broker-level config, "delete.retention.ms", supported.
log.cleaner.enable	N/A. Compacted topics are always compacted.
log.cleaner.min.cleanable.ratio	N/A. WarpStream uses different heuristics to determine when to compact the topic. This is handled automatically.
log.cleaner.threads	N/A. WarpStream is written in Go, not Java, and doesn't have user-controlled threadpools. Concurrency is managed automatically.
log.retention.bytes	Not supported. Response hard-coded to -1 to signify "no limit".
log.retention.check.interval.ms	N/A. Retention is enforced using a different mechanism called the "dead scanner" in WarpStream, and that process is managed automatically.
log.retention.hours	Same as Kafka.
Logcleaner ratio	N/A. WarpStream uses different heuristics to determine when to compact the topic. This is handled automatically.
log.roll.hours	N/A. WarpStream uses object storage and doesn't have physical log files that need to be rolled.
log.segment.bytes	N/A. Object storage file size is an implementation detail of WarpStream compaction, and WarpStream files contain data from many different topic-partitions.
message.max.bytes	Agent environment variable WARPSTREAM_MAX_PRODUCE_RECORD_SIZE_BYTES
min.insync.replicas	N/A. Response hard-coded to 1 as object storage handles replication / durability.
num.io.threads	N/A. WarpStream is written in Go, not Java, and doesn't have user-controlled threadpools. Concurrency is managed automatically.
num.partitions	Same as Kafka.
num.replica.fetchers	N/A. WarpStream is written in Go, not Java, and doesn't have user-controlled threadpools. Concurrency is managed automatically.
offsets.topic.num.partitions	N/A. WarpStream virtual clusters do not publish offset topics. Offsets are stored in WarpStream control plane / metadata store.
offsets.topic.replication.factor	N/A. WarpStream virtual clusters do not publish offset topics. Offsets are stored in WarpStream control plane / metadata store.
port	Agent environment variable "WARPSTREAM_KAFKA_PORT".
replica.fetch.max.bytes	N/A. WarpStream Agents are stateless and therefore have no replicas.
replica.lag.max.messages	N/A. WarpStream Agents are stateless and therefore have no replicas.
replica.lag.time.max.ms	N/A. WarpStream Agents are stateless and therefore have no replicas.
unclean.leader.election.enable	N/A. WarpStream has no leaders, and therefore unclean leader elections are impossible.
zookeeper.session.timeout.ms	N/A. WarpStream does not depend on Zookeeper.
zookeeper.connection.timeout.ms	N/A. WarpStream does not depend on Zookeeper.
security.inter.broker.protocol	Not supported.
ssl.client.auth	Agent environment variable "WARPSTREAM_REQUIRE_MTLS_AUTHENTICATION"
ssl.truststore.location	Agent environment variable "WARPSTREAM_TLS_CLIENT_CA_CERT_FILE"
ssl.truststore.password	Not supported.
ssl.keystore.location	Agent environment variables "WARPSTREAM_TLS_SERVER_CERT_FILE", "WARPSTREAM_TLS_SERVER_PRIVATE_KEY_FILE"
ssl.keystore.password	N/A
ssl.key.password	N/A
ssl.protocol	Agent environment variable "WARPSTREAM_TLS_ENABLED".
ssl.enabled.protocols	N/A
ssl.keystore.type	N/A
ssl.truststore.type	N/A
broker.id.generation.enable	N/A. Broker IDs in WarpStream are virtual and generated automatically.
sasl.kerberos.service.name	N/A
num.network.threads	N/A. WarpStream is written in Go, not Java, and doesn't have user-controlled threadpools. Concurrency is managed automatically.
num.recovery.threads.per.data.dir	N/A. WarpStream is written in Go, not Java, and doesn't have user-controlled threadpools. Concurrency is managed automatically. Also WarpStream Agents are stateless and have no concept of a recovery.
num.recovery.threads.per.data.dir	N/A. WarpStream is written in Go, not Java, and doesn't have user-controlled threadpools. Concurrency is managed automatically. Also WarpStream Agents are stateless and have no concept of a recovery.
socket.send.buffer.bytes	N/A
socket.receive.buffer.bytes	N/A
socket.request.max.bytes	N/A
replica.fetch.wait.max.ms	N/A. WarpStream Agents are stateless and therefore have no replicas.
replica.socket.timeout.ms	N/A. WarpStream Agents are stateless and therefore have no replicas.
replica.socket.receive.buffer.bytes	N/A. WarpStream Agents are stateless and therefore have no replicas.
replica.high.watermark.checkpoint.interval.ms	N/A. WarpStream Agents are stateless and therefore have no replicas.
controller.socket.timeout.ms	N/A
controller.message.queue.size	N/A
zookeeper.sync.time.ms	N/A. WarpStream does not depend on Zookeeper.
queued.max.requests	N/A
fetch.purgatory.purge.interval.requests	N/A
producer.purgatory.purge.interval.requests	N/A
authorizer.class.name	N/A. There is only one ACL Authorized implementation in WarpStream.
allow.everyone.if.no.acl.found	Same as Kafka.
broker.id	N/A. WarpStream Broker IDs are generated automatically.
log.dirs	N/A. WarpStream Agents are stateless and store data exclusively in object storage.
zookeeper.connect	N/A. WarpStream does not depend on Zookeeper.
listeners	Agent environment variables: "WARPSTREAM_DISCOVERY_KAFKA_HOSTNAME_OVERRIDE" and "WARPSTREAM_KAFKA_PORT"