Change Log

Contains a history of changes made to the Agent.

Subscribe to the RSS Feed

https://console.warpstream.com/agent-changelog.rss

By default all Agent upgrades are designed to be seamless and backwards compatible. Occasionally we have to make make breaking changes which we document in the migrations sub-page where you can see if any breaking changes were made between the version you're running and the version you're upgrading to.

Change Log

Release v818

July 9, 2026

Added a new warpstream local command that runs a fully in-memory, self-contained WarpStream cluster (Kafka + Schema Registry + Tableflow) with no connection to the WarpStream control plane. Intended as a replacement for warpstream playground in CI pipelines.
Automatically run warpstream local when warpstream demo or warpstream playground is started in CI, with a warning that local mode is designed for CI environments.
Agent: dynamically tune the direct load prefetch concurrency based on observed CPU usage instead of using a fixed value of 16. Underloaded agents can still serve up to 64MiB of concurrent prefetch per request, but as CPU usage rises the concurrency automatically steps down to avoid the agent DDOSing itself with prefetch work it can't transmit in time, which previously caused wasted GET requests and slow recovery under high load.
Orbit: add support for JSON key/value secrets in Cloud Provider's Secrets Manager. A secret reference in the Orbit config can now specify an optional key to extract a single value from a secret stored as a JSON object (the standard AWS Secrets Manager key/value pattern). AWS secrets stored as binary (SecretBinary) are now supported as well.
Ensure Close() is called on the pooled buffer after all replicas complete on the quorum-success path of multi-bucket PutBytes.
Add cluster-level default partition auto scaler settings (warpstream.default.partitions_auto_scaler.* broker configs) that are applied to newly created topics, so a cluster can enable partition auto scaling by default for all new topics.

Release v817

July 8, 2026

Tableflow:
- Expose Schema Registry writer schema ID and version as warpstream.sr.writer_schema_id and warpstream.sr.writer_schema_version to ingested records before applying Bento transforms so that the transform logic can condition on those properties.
- Make BigLake integration automatically detect catalogs with credential vending mode and send X-Iceberg-Access-Delegation header on Iceberg REST calls if so.

Release v816

July 6, 2026

Allow Orbit to chunk requests when copying consumer group offsets.
Schema Registry: include Confluent-compatible guid values in schema API responses and support looking up schemas by GUID with GET /schemas/guids/{guid}.

Release v815

July 2, 2026

Add support of -disableConsumerGroupMetrics (env var WARPSTREAM_DISABLE_CONSUMER_GROUP_METRICS) and -disableConsumerGroupsMetricsTags (env var WARPSTREAM_DISABLE_CONSUMER_GROUP_METRICS_TAGS) in agent metrics mode to offer the same knobs as in regular agent mode over cluster level metrics cardinality. By default -disableConsumerGroupsMetricsTags is set to empty string (the default value for regular agents is partition).
Tableflow:
- Add support for OAuth as an authentication mechanism for source Kafka clusters.
Fix Go runtime CPU utilization reporting so stale high samples decay when the Go runtime has not emitted a fresh CPU metrics snapshot.
Auto enable bucket pre-warming when the bucket is of type Rapid Storage.
Reduce default value of GOGC for Agents running only pipelines role from 1000 to 200.

Release v814

July 1, 2026

Fixed a bug in the Prometheus metrics scraping path where the warpstream_max_offset and warpstream_min_offset metrics were summed across partitions instead of taking the max/min when the partition tag is disabled.
Fix the metric double publishing problem introduced in v808 when the dedicated metrics mode was enabled (the regular agents would keep publishing the same metrics).
Improve produce latency for GCP Rapid Buckets by pre-warming upload streams.
Add a new -additionalBackgroundTasksBucketURLs flag and WARPSTREAM_ADDITIONAL_BACKGROUND_TASKS_BUCKET_URLS environment variable that accepts a comma-separated list of bucket URLs that should have background tasks run on them like scanning for dead files to delete or ripcord files to ingest. This is important for customers who are migrating from one bucket to another. The new flag supersedes the existing -additionalDeadscannerBucketURLs flag and WARPSTREAM_ADDITIONAL_DEADSCANNER_BUCKET_URLS environment variable.
Add support for OIDC workload identity federation: when WARPSTREAM_AGENT_WORKLOAD_IDENTITY_TOKEN_SOURCE is set, the agent obtains a platform OIDC token and exchanges it with the control plane for a short-lived agent token, removing the need for a static agent key. Currently aws (STS) and gcp (metadata server) sources are supported, but other platforms may be added in the future. Supported for single-region clusters only.
Add an opt-in file ID allocator (behind the use_range_file_id_allocator flag, default off) that hands out IDs from several discontiguous reserved ranges to spread object-storage key prefixes more evenly and reduce metadata-layer hot-spotting. The key layout is unchanged. The WARPSTREAM_USE_RANGE_FILE_ID_ALLOCATOR environment variable (true/false) is the final per-Agent override for the flag.

Release v813

June 29, 2026

Tableflow:
- Update internal compaction API usage.
- Fix bug where the schema registry integration feature was not ungated properly.
- Add support of Cloud Provider's Secrets Manager in TableFlow config.
Add a new flag -skipBucketsPermissionCheck (env var WARPSTREAM_SKIP_BUCKETS_PERMISSION_CHECK) that allows to skip the permission check for buckets when starting an agent. The default value is false.
Improve DNS resolution of S3 addresses by automatically injecting a . at the end of any S3 endpoint we try to connect to. This is a workaround because the AWS SDK does not support it to this day https://github.com/aws/aws-sdk-go/issues/1380.
Re-enable GCS direct connectivity client by default.
Disable some connection-level backpressure mechanisms when TLS is enabled because once we've paid the price of accepting a TLS connection and doing the formal handshake, paradoxically, backpressuring that connection by closing it due to high load makes the situation worse as the client will just try to re-establish the connection and the TLS handshake is extremely expensive.
Stop playground agents from logging query engine state refresh errors when query engine credentials are not required.

Release v812

June 22, 2026

Move orbit consumer offset copying job entirely into the agents.

Release v811

June 22, 2026

Tableflow:
- Add Schema Registry integration for Protobuf schemas.
- Fix Tableflow ingestion getting stuck (no ranges were processed, possibly due to timeout) on source topics written by transactional producers (e.g. CDC), where the offset at the high watermark is a transaction control marker that a read-committed consumer never receives.
Add support of Cloud Provider's Secrets Manager in Orbit config.
Schema Registry: prevent an incompatible type change of a oneof field when that field's oneof block is renamed or dissolved. This matches a bug fix in Confluent's protobuf compatibility that will be released in Confluent Platform 8.3.1.

Release v810

June 19, 2026

Tableflow:
- Support sorting and column statistics on float and double columns.
Schema Registry: Match Confluent's protobuf compatibility behavior by allowing a type change on a oneof field is now allowed when that field's oneof block is renamed or dissolved.

Release v809

June 18, 2026

Orbit now creates empty source topics with a non-zero log end offset at that offset in WarpStream. This fixes a bug where an empty source topic could be migrated to WarpStream at offset 0 instead of preserving its source log end offset.
Add a cli-beta validate-pipeline-config command for validating managed data pipeline config files locally. Usage: warpstream cli-beta validate-pipeline-config --config-file pipeline.yaml.
Restrict which clients see support for KIP-714 client metrics APIs. The GetTelemetrySubscriptions and PushTelemetry endpoints will no longer be advertised to clients older than this version.

Release v808

June 18, 2026

Tableflow:
- Add support for sorting and column statistics tracking for Iceberg tables.
- Fix time-based partition transform on an Avro date field.
Update the publish metrics job implementation to scrape metrics from the control plane Prometheus endpoint. This should not change any of the existing metric, but will add a few that were only available in the control plane Prometheus endpoint.

Release v807

June 12, 2026

Orbit Auto Migration is now available.

Release v806

June 12, 2026

Forward produce backpressure metrics for observability.
Add support for OAuth as an authentication mechanism for source clusters in Orbit.
Tableflow: Validate properly the required fields nested under untyped maps via additionalProperties for JSON ingestion.
Bump agent base image (cgr.dev/chainguard/wolfi-base) to pick up busybox fix for CVE-2023-39810.

Release v805

June 10, 2026

KIP-714 client metrics are now generally available to all customers.

Release v804

June 9, 2026

Add global sort duration in batcher to flush_end log.
Fix a bug where the agent could have problems listing ripcord files if there were too many of them and your bucket URL was long, because the generated output was too big.
Fix Orbit and Tableflow source connections silently dropping SASL when TLS is enabled without mTLS.

Release v803

June 8, 2026

Warning: This version drops SASL when connecting to an Orbit or Tableflow source cluster over TLS without mTLS. It is fixed in v804. Please upgrade to v804 directly and skip v803.

Fix panic when emitting the client availability zone mismatch diagnostic during interzone load balancing.
Upgrade Bento to v1.18.1.
Fix Orbit and Tableflow source cluster TLS configuration to allow setting a private CA (mtls_server_ca_cert_env) without requiring mTLS.
Schema Registry: the error_code for incompatible schema responses now returns 40901 (was 409) to align with Confluent's documented API. The HTTP status code remains 409.
Schema Registry: Avro schemas with invalid field defaults are now rejected at registration time (aligns with Confluent SR 8.2.1 behavior). Existing schemas registered before this change are unaffected.
Add batching for JoinGroup and SyncGroup requests via RSM batch commands, reducing the number of RSM proposals when many consumers join/sync concurrently.

Release v802

June 4, 2026

Bump build to go 1.26.4 to fix CVE-2026-42504
Add an agent flag to override max inflight fetch compressed bytes per CPU.
Upgrade Bento to v1.18.0
Tableflow:
- Fix bug for dlq_keep_settings.retention: we now allow for d and w units for days and weeks respectively.
- Fix bug that considered null values set within transforms as non-null when fields were marked as required.
- Tableflow agents can now create Iceberg table metadata even when they cannot access external blob store buckets directly. This can be enabled by setting the tableflowMetadataSyncMode flag to proxy.
- Support converting between booleans and strings, and between integers and strings, in both directions when mapping records to the destination schema.
- Breaking Change (AWS Glue): Fix a bug in the AWS Glue integration that caused us to hit the TableVersions limit when syncing the table. This fixes the error Number of TABLE_VERSION resources exceeds the limit 100000 per TABLE. Note that agents will now require additional roles: glue:GetTableVersions and glue:BatchDeleteTableVersion.
- Fix some tags on the event emitted when sending records to the DLQ.

Release v801

May 29, 2026

Deprecate a legacy code path for computing Data Lake sorting bounds and column statistics.
Fix events scheduler writing a corrupt active state when an event type was disabled while events were globally enabled. Agents now drop records with stream ID 0 instead of poisoning the file batch.
Tableflow: Implement basic validation for required / nullable fields in JSON.

Release v800

May 27, 2026

Tableflow:
- Fix bug on transforms: Bento transforms replacing the full root object instead of modifying it field by field are now handled correctly (e.g. root = { ...} now works).
Fix warpstream_consumer_group_estimated_lag_very_coarse_do_not_use_to_measure_e2e_seconds reporting full retention as the lag for partitions with no committed offset. The metric now uses the age of the oldest surviving record (bounded by retention).

Release v799

May 27, 2026

Fix events queries failing with UNSUPPORTED_SASL_MECHANISM on clusters that restrict enabledSASLMechanisms to exclude PLAIN.

Release v798

May 26, 2026

Fix a query engine bug that could cause Agent panics for some sorted and limited queries.
Fix false positive diagnostic for cross az traffic when the query engine was used.
Fix false positive diagnostic for small fetch timeout when the query engine was used.
Return clear error when schema registry request body size exceeds limit.

Release v797

May 19, 2026

Tableflow:
- Fix decoding of Protobuf oneof fields (introduced in v796, upgrade directly to v797 if you use a configuration with Protobuf schemas).
- Recreate BigLake tables when the Iceberg table UUID changes.
Fix DNS-backed Agent file cache retries so failed replicas can fall back to other resolved Agents.
Add per-pipeline log level controls for managed data pipelines.
Fix event queries on agents with SASL enabled.

Release v796

May 18, 2026

Warning: There is a problem for Tableflow protobuf schemas with that version fixed in v797. Please upgrade to v797 directly and skip v796 if you use Tableflow with a protobuf schema.

Upgrade the embedded version of Bento to v1.17.0.
Start supporting regional migrations.
Add a rate limiter in the metrics mode to throttle metrics emission. This avoids flooding the Datadog agent when Datadog metrics are enabled.
Adds an experimental agent flag that can be used to reduce E2E latency for workloads that do not produce new records consistently on all partitions all the time.
Tableflow:
- Breaking change for very old agent versions using protobuf, they would need to upgrade to version 796+.
- Add support for field type remapping

Release v795

May 14, 2026

Forward schema registry metrics for observability.
Tableflow: Fix the Tableflow configuration used in the demo mode.
Reduced telemetry sent from Agents to control plane.

Release v794

May 12, 2026

Add a diagnostic for invalid regex subscriptions in modern consumer group heartbeat requests.
Bump golang.org/x/net to v0.54.0 to fix CVE-2026-33814
Detect when Kafka clients request one availability zone via ws_az but produce or fetch against an Agent in a different availability zone and fire a diagnostic.

Release v793

May 11, 2026

Unless customers specify a -metadataURL/WARPSTREAM_METADATA_URL the agents will start using 3 different "zonal" endpoints to communicate with WarpStream - meaning one per WarpStream control plane availability zone. The previous behavior can be put back by setting the environment variable WARPSTREAM_AGENT_ENABLE_ZONAL_URLS to false. Removes a dependency on inconsistent behavior across cloud providers when load balancing across multiple availability zones and increases the product's resiliency to zonal outages significantly.
Bump build to go 1.26.3
Fixed CVEs: CVE-2026-39820 / CVE-2026-33811 / CVE-2026-42499 / CVE-2026-39820 / CVE-2026-33811 / CVE-2026-39836
Agent will now enforce read-only Schema Registry credentials and only allow read schema operations for read-only credentials.

Release v792

May 8, 2026

Add optional PROXY protocol v2 support on the Kafka listener via kafkaProxyProtocol (or WARPSTREAM_KAFKA_PROXY_PROTOCOL). When enabled, every connection must include a v2 PROXY header before any other bytes; v1 headers are rejected. Operators can also set kafkaProxyProtocolPrincipalTLVType (a hex byte in the PP2 user-defined range 0xE0–0xEF) to source the connection's ACL principal from a custom TLV in the PROXY header — the TLV value must already include the User: prefix and overrides any principal that would otherwise be derived from mTLS.
Tableflow:
- Fix a bug in ingestion that was not respecting some types from the input schema.
- Fix ingestion of date types for Avro input schemas.
- Fix ingestion of Avro map types with optional values when the deprecated way of specifying schemas is used.
- Fix several correctness bugs in the Protobuf decoder so records now align with the proto3 spec (oneof last-wins, merging of repeated singular message fields, strict wire-type validation).

Release v791

May 6, 2026

Fix a bug in the agent metrics mode that would cause it to stall when any of the metrics scraped contained a comma in its labels.
Mark file cache closed-pipe logs as debug.

Release v790

May 5, 2026

Tableflow: Make ingestion handle partition evolution.
Bump our Chainguard Wolfi Docker image to 79af0917ba7ac066ebf1f99e5967ee9e77e6a350facec5be43a7a236c705cf10 to fix CVE-2026-5450 and CVE-2026-5928.
Fix a bug in the query engine when querying across events that mix integers and floats in the same field name.
Fix a bug in the query when combining multiple filters like: filter a=="b" | filter c=="d" where only the latter filter would be applied.

Release v789

April 30, 2026

Fix a bug where object store errors due to networking issues (DNS or TCP timeouts, context cancellation, etc) were misclassified as NotFound errors, causing red-herring invariant violations.
Tableflow: Fix ingestion of list types with optional elements.
Public agent docker images and release tarballs now ship notices.txt (auto-generated from the binary's transitive Go dependencies) and non_ibm_license.txt at the container/archive root, for IBM third-party legal compliance.
Fix the reason for a decision made during compaction in a log from being wrong.
Treat "unexpected EOF" as a retriable error for long-running compactions.

Release v788

April 29, 2026

Bump github.com/aws/smithy-go to v1.25.1. This fixes a very slow memory leak associated with using S3 Express One Zone in the AWS SDK.

Release v787

April 28, 2026

Query engine: disable the admin RPC metadata cache because it causes a bug with inconsistent results when querying events.
Query engine: fix ::int, ::float, and ::str casts in event queries, and make integer/float comparisons work in filters.
Agents will now automatically backpressure Kafka protocol requests when their CPU is >= 98% for a sustained period of time. This threshold can be adjusted by setting the WARPSTREAM_HIGH_CPU_BACKPRESSURE_THRESHOLD_PERCENT environment variable and disabled by setting it to 0.
All Datadog metrics now include virtual_cluster_id, agent_id, and agent_version tags.

Release v786

April 28, 2026

Emit cluster-level Prometheus-style metrics (consumer group lag, topic details, diagnostics, tableflow state, ...) as cluster_metrics CloudEvents whenever events are enabled on the cluster. Emission is independent of the Datadog push: events are still emitted even when DisableAllMetrics or DisableConsumerGroupMetrics is set. Events are always at topic-level granularity (partition tag is always stripped) to keep the per-job event volume bounded. The event's metric_name field carries the canonical warpstream_-prefixed name (e.g. warpstream_consumer_group_lag), identical to the Prometheus scrape name and the describe_cluster_metrics API response, so the same string identifies a metric regardless of how it is queried.
Tableflow: improve DLQ health reporting so records routed to the DLQ are no longer surfaced as skipped, reducing misleading health warnings and making it clearer when data was retained for later replay.
Improve query engine performance by caching metadata lookups and aligning query time boundaries.
Move modern consumer group heartbeat regex matching and topic DESCRIBE ACL filtering from saasy into the agent, and forward the resolved regex stream IDs to saasy for statemachine application.
Allow reserved keywords like topic to parse as part of .-deleted field identifiers in the query engine.

Release v785

April 23, 2026

Emit cluster-level Prometheus-style metrics (consumer group lag, topic details, diagnostics, tableflow state, ...) as cluster_metrics CloudEvents whenever events are enabled on the cluster. Emission is independent of the Datadog push: events are still emitted even when DisableAllMetrics or DisableConsumerGroupMetrics is set. Events are always at topic-level granularity (partition tag is always stripped) to keep the per-job event volume bounded.
Make metrics pod use external HTTP client to have longer connect timeouts.
Make metrics pod re-use HTTP client across requests to reduce time spent on establishing new connections.
Change consoleURL parameter name to apiURL .
Fix a bug that would cause the original error to be dropped from error messages and logs when hedging requests with the fast retrier.
Fix bug in query parser for handling sub expressions.
Make tableflow events all have the table name and table UUID fields set properly.
Fix panic in query engine related to nested expression and or statement.
Emit a tableflow event when the offset scraper job handler fails.
Enable the agent query engine by default and require disableQueryEngine to opt out of query handling.
Removed the OBJECT_STORAGE_PUBLIC_NETWORK_PATH diagnostic (added in v765).
Added a diagnostic that fires when a produce request contains records for both lightning and classic topics, which removes the latency benefit of the lightning topic.

Release v784

April 20, 2026

Fix a memory leak when using the metrics agent mode.
Bump github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream to v1.7.8, github.com/aws/aws-sdk-go-v2/service/kinesis to v1.43.5, github.com/aws/aws-sdk-go-v2/service/lambda to v1.89.0 and github.com/aws/aws-sdk-go-v2/service/s3 to v1.99.0 to fix GHSA-xmrv-pmrh-hhx2.
WarpStream Agents will no longer report the availability zone that they're running in as the value of the Rack field in Broker metadata in the response of Metadata RPCs. Instead, they will always report their Rack as: "warpstream-fake-rack". The reason for this is that WarpStream's service discovery system tracks availability zones of clients using client ID features, not this Rack field. In addition, when clients enable rack-awareness for consumers in their Kafka clients without properly configuring WarpStream's zone-aware service discovery system, the presence of the availability zone in the Rack field in the Broker's metadata, combined with the fact that it will change from time to time due to how WarpStream's partition assignment strategies work, may result in an excessive number of consumer group rebalances. As a result, since this field has no value in WarpStream clusters, we're hard-coding the value of Rack so that misconfigured clients will not experience excessive consumer group rebalances.
Tableflow:
- Fix ingestion of Avro fixed and time types.
- Performance improvement when sorting is enabled on tables.

Release v783

April 16, 2026

Tableflow:
- Reject invalid combinations of WARPSTREAM_BUCKET_URL, WARPSTREAM_INGESTION_BUCKET_URL and WARPSTREAM_COMPACTION_BUCKET_URL with an explicit error message for datalake agents with events enabled.
Bump go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp to v1.43.0 for CVE-2026-39882 vulnerability.
Fix connection reuse on the query engine.
Fix a bug that didn't allow users to manually disable events for tableflow clusters.

Release v782

April 14, 2026

Tableflow: add warpstream.tableflow_dlq_records_counter agent metric tagged by topic and strategy (skip or keep) to track records handled by the DLQ during ingestion. Replaces the previous warpstream.ingestion_job_dlq_skip_strategy_counter which only covered the skip strategy.
Improve query engine performance by fixing connection reuse.
Improve produce latency by parallelizing record sorting in the agent batcher hot path.
Fix Avro schema validation to handle references using short name within a namespace properly.
Bump go.opentelemetry.io/otel/sdk to v1.43.0 for CVE-2026-39883 vulnerability.
Improve how runtime CPU measurements are taken to be more precise and not persist inaccurate values for long period of time by occasion.
Improved agent feature versioning scheme to auto-compute latest released version from feature declarations, eliminating manual maintenance and potential for version bump errors
Query engine: fix distributed aggregation for aliased count() and count(expr) so merge/final stages consume the correct intermediate count columns.
CLI alter-broker-config now surfaces per-broker error responses instead of silently reporting success.
Support setting warpstream.default.topic.type via AlterConfigs and IncrementalAlterConfigs.
Unknown warpstream-prefixed broker and topic configs now return an error instead of being silently ignored.

Release v781

April 9, 2026

Breaking Change (GCP): WarpStream Agents now depend on storage.buckets.get access for all configured GCP (gcs://) bucket URLs. This can be overriden by setting the environment variable WARPSTREAM_GCS_PERFORM_BUCKET_STORAGE_CLASS_LOOKUP=false .
Allow configuring the tracer block in managed data pipeline configs.
Automatically disable idempotent writes for managed data pipelines writing to WarpStream clusters via kafka_franz_warpstream output blocks.
Fix non-determinism in generating schema hash for unrecognized key in Avro schemas.
Bump build to go 1.26.2
Add support for GCP Rapid Buckets.
Optimize metrics code, particularly when emitting Datadog metrics.
Fix the docker image base to properly use the multi-arch wolfi-base image

Release v780

April 8, 2026

Reduce max diagnostic groups per type from 10 to 5 to lower payload size sent to the control plane.
Remove spammy error log when diagnostic groups exceed the per-type limit.
Forward sampled flush_event logs for observability.
Emit forwarded flush synchronous duration metric.
Bump our Chainguard Wolfi Docker image to 79af0917ba7ac066ebf1f99e5967ee9e77e6a350facec5be43a7a236c705cf10 to fix CVE-2026-4437.
Bump github.com/go-jose/go-jose/v4 to v4.1.4 to fix GHSA-78h2-9frx-2jm8.
Prevent a single fetch request from triggering thousands of concurrent HTTP requests if it's querying data for many topic-partitions whose individual data is spread across many files.
Install CURL in Agent image so it can be used in health checks for services like ECS that rely on container having command line utilities available.
Tableflow: tableflow events are generally available and agent-side events handling is enabled by default. This is a breaking change since tableflow agents now require a bucket URL to be configured (either WARPSTREAM_BUCKET_URL or both WARPSTREAM_INGESTION_BUCKET_URL and WARPSTREAM_COMPACTION_BUCKET_URL). Agents without bucket URLs will fail to start.
Enable events when bootstrapping demo and playground clusters.
Added SLOW_CONSUMER push diagnostic: detects consumers that are too slow between fetch requests using a dynamic threshold (baseline + 1s per MB of previous response size) and emits the diagnostic after 3 consecutive slow fetches on a connection.
Fix GOMEMLIMIT parsing to support byte suffixes like "24GiB" and "1000MB" instead of requiring plain integers.
Fix GOGC, GOMAXPROCS, and GOMEMLIMIT heartbeat reporting to not report a value on parse error. Previously a parse error would silently report 0.

Release v779

April 1, 2026

Fix bug that prevented query engine from running queries successfully on Agents configured to only run the Jobs role.

Release v778

April 1, 2026

Add principal impersonation via record headers. Configured impersonator principals can produce messages on behalf of other users by setting a designated record header, enabling ACL evaluation as the impersonated principal.
Fix a bug that caused events queries to fail on agents with mTLS enabled.
Enable dual-mode Prometheus native histograms alongside classic histograms for improved metric accuracy.
Query engine: route query jobs only to agents with the query engine enabled and fail fast if an agent is configured to accept only query jobs without enabling the query engine.
Replace mutex-protected rand.Rand instances with lock-free top-level rand functions in certain hot paths.
Fix a bug that was preventing events from being emitted/enabled in public Agent builds.
Enable agent-side events handling and query engine automatically for Kafka agents.
Expose a /debug/pprof/trace endpoint for Go runtime traces
Tableflow: fix a bug where compaction failed on certain parquet file layouts with gaps between column chunks, leaving small files unmerged.
Tableflow: add type: "record_ingestion_failed" to logs corresponding to decoding errors during ingestion.
Tableflow: add warpstream.tableflow_ingestion_lag_seconds and warpstream.tableflow_query_lag_seconds metrics. Ingestion lag measures how far behind the ingestion process is from the source Kafka topic. Query lag measures the time from when data is produced to Kafka until it is queryable (ingestion lag + catalog sync delay).

Release v777

March 26, 2026

Fix schema validation in playground by passing missing schema registry URL to config.

Release v776

March 26, 2026

Tableflow:
- Add warpstream.tableflow_partition_offset_lag gauge for Tableflow ingestion offset lag in records. When topic or partition tags are omitted, offset lag is summed across partitions (like consumer_group_lag); time lag remains the max per scope.

Release v775

March 25, 2026

Add topic, partition, and streamID to orbit offset error messages for easier debugging.

Release v774

March 25, 2026

Cache metrics in fetch and produce Kafka handlers to reduce tagset allocations when updating metrics.
Tableflow:
- Fix panic in demo mode by adding missing input_schema to demo config.
Add topic tag to agent_kafka_produce_with_offset_uncompressed_bytes_counter metric when high cardinality metrics are enabled.

Release v773

March 24, 2026

Don't swallow circuit breaker errors and make it clear in error logs that are showing cached circuit breaker errors what is happening.
- For instance consumers could start failing when some Kafka topics are deleted but are still trying to be accessed, because the metadata response would be incorrect
Don't count structured errors from the control plane (400s) as failures in the control plane client circuit breakers.

Release v772

March 23, 2026

Fixed IPv6 address parsing.
Tableflow:
- Make the TABLEFLOW_ prefix optional for environment variables used to specify Kafka cluster credentials.
- Fix BigQuery integration for Tableflow tables failing to update when the Iceberg schema evolves (e.g., new columns added).
Add support for ConsumerGroupDescribe API (key 69, KIP-848) for the modern consumer group protocol. Disabled by default.
Make the ORBIT prefix optional for environment variables used to specify Kafka cluster credentials.
Fix the empty_response tag value for the warpstream.agent_kafka_fetch_single_attempt_outcome metric.
Bump google.golang.org/grpc to v1.79.3 for CVE-2026-33186 vulnerability.
Bump our Chainguard Wolfi Docker image to ce84795834de56c47f27b6be64388628677d18e799e9b5a05fe6538aaa17bc79 to fix CVE-2026-2673.
Upgrade Bento to v1.16.1 to get Datadog and Bigtable outputs support.
Query engine: stop clamping query tail offsets from cached stream metadata so Event Explorer queries do not intermittently miss recent time buckets.
Add Datadog-compatible HTTP log intake endpoints on Agents so Datadog log batches can be routed into WarpStream topics.
Add a zstd produce fallback decoder path so Agents can accept some batches that fail github.com/DataDog/zstd with unexpected EOF but succeed with klauspost/compress/zstd.

Release v771

March 19, 2026

Fix the metrics prefix pushed by the agent in metrics mode: in previous versions we would add back a "warpstream_" prefix to all metrics, which was redundant.

Release v770

March 18, 2026

Fix: re-include nc in the warpstream-agent image so that healthchecks can be performed properly on ECS.

Release v769

March 18, 2026

Tableflow:
- Add BigLake Metastore integration via Iceberg REST Catalog API for GCP data lake tables.
- Add Hive Metadata Store integration

Release v768

March 12, 2026

Disable GCS direct connectivity client by default as it is causing a regression in some GCP regions.

Release v767

March 11, 2026

Fix tls_insecure_skip_verify not being applied for TLS+SASL connections without mTLS in Tableflow and Orbit source clusters.
Prevent Agents from trying to issue file cache requests against Agents that no longer exist when fetching old file extents by enforcing a maximum cache staleness in the activation cache.

Release v766

March 11, 2026

Bump build to go 1.26.1
Tableflow:
- Enhance logging when metadata upload fails.
- Add a module attribute to job logs to group them logically.
- Fix the command to query the Tableflow table logged by the demo agent.
Fail agent startup if the ripcord mode is used for an agent connecting to a Tableflow or a Schema Registry cluster.
Add observability by forwarding agent produce error outcomes.
Switch agent Docker base images from Alpine to Chainguard Wolfi https://images.chainguard.dev/directory/image/wolfi-base/overview
Misc: New way of releasing static agent features, this should be a transparent change.
Add create-acls, describe-acls, and delete-acls clibeta commands for managing ACLs from the CLI, providing a kafka-acls.sh equivalent.

Release v765

March 5, 2026

Tableflow: Add retries to the bucket access checker and remove extra slash used in the request URL.
Add agent_version label/tag to all metrics emitted by the agents.
Bump github.com/docker/cli to v29.2.1 for CVE-2025-15558 vulnerability.
Add L7 proxy detection diagnostic for inter-agent traffic. When a load balancer or reverse proxy is detected between agents, a new high-severity diagnostic is surfaced to help users identify and remove proxy interference.
Added new OBJECT_STORAGE_PUBLIC_NETWORK_PATH diagnostic that detects when S3 or Azure Blob Storage endpoints resolve to public IPs, indicating a missing VPC endpoint (or equivalent). This helps identify unnecessary networking costs from object storage traffic going over the public internet.
Fix a bug where some log attributes were grouped into a !BADKEY key.

Release v764

March 3, 2026

Tableflow: make Protobuf type google.protobuf.Timestamp usable as a custom partitioning field.

Release v763

February 27, 2026

Increase the maximum allowed value for batchMaxSizeBytes (max uncompressed batch flush size) from 64MiB to 128MiB.
Improve Schema Registry to properly handle trailing slashes in URL paths (e.g., /config/ now works the same as /config).

Release v762 [do not use]

This version has a known bug that causes some records to be incorrectly skipped by consumers. Do not use it.

February 26, 2026

Make sure BYOC schema registry response's Content-Type is application/json if the request's Accept header is application/json.
Fixed a bug where the demo agent would get stuck waiting for the Tableflow Iceberg table to be created.
Optimizes fetch to use less CPU and IO when fetching a single record, or when max.partition.fetch.bytes is low.

Release v761

February 25, 2026

Add PSI (Pressure Stall Information) metrics collection for CPU, memory, and I/O pressure. New metrics: pressure_cpu_some_pct, pressure_cpu_full_pct, pressure_memory_some_pct, pressure_memory_full_pct, pressure_io_some_pct, pressure_io_full_pct. These metrics report the percentage of time tasks were stalled due to resource contention over the last sampling interval (0-100%).
Update orbit to cover for protocol incompatibility in some systems which implement kafka api

Release v760

February 24, 2026

Tableflow: Performance improvement on ingestion saving 5 to 10% of CPU usage.
Add metrics category to --jobSelector flag, allowing agents to run only metrics publishing jobs (e.g., --jobSelector metrics).

Release v759

February 24, 2026

Add --jobSelector flag (env: WARPSTREAM_JOB_SELECTOR) to specify which job types an agent handles.

Release v758

February 20, 2026

Tableflow:
- Fix ingestion of Avro and JSON when lists or map values contain decimal fields.
- Make Proto unsigned integers (uint32, uint64, fixed32, fixed64) convert into decimals. Before that, making a Protobuf backward-compatible change that changed a uint32 into a uint64 (or vice-versa) was forbidden because it would end in a non-backward-compatible Iceberg change (from long to string or vice-versa). Now it's possible. This is a breaking change for Protobuf tables as the Iceberg column type changes for those uints.
- Add support for google.protobuf.Timestamp type in Protobuf. It gets converted to an Iceberg timestamptz.
Fix time_since_last_top_level_fetch_ms to avoid negative values when there are concurrent fetch requests.
Improve the "Cross AZ Kafka Clients" diagnostic to show more than 1 clientId.
Fix a bug in the instance type detection logic for Kubernetes environments that could cause it to fail to detect the instance type in some cases and return "unknown" instead.
Add log message for mTLS authentication that includes certificate serial number and validity period in order to help monitor client certificate expirations.
This functionality is disabled by default and can be controlled with the -logMTLSCertificateInfo flag or WARPSTREAM_LOG_MTLS_CERTIFICATE_INFO environment variable.

Release v757

February 19, 2026

Added a new agent flag -injectPrincipalHeader (env var WARPSTREAM_INJECT_PRINCIPAL_HEADER) that, when set to a header key name, injects the kafka principal username as a header in each produced record. This allows downstream consumers to identify which user produced each record.
Fixes a bug in the ripcord deletion queue that could cause the "last opened sequence" to grow until the agent is restarted, because the agent silently discards files that should be deleted in some circumstances.
Query engine: Fix a bug that caused fields escaped with backticks to not be queryable properly.
Tableflow:
- Emit the correct logical type annotation for time fields in Parquet files.
- Fix ingestion of Avro records with decimal fields.

Release v756

February 17, 2026

Reject all traffic to HTTP fetch endpoints if mTLS is enabled on the Agents.
Validate SASL credentials in HTTP fetch endpoints even if SASL authentication is not required by the Agents.
Tableflow: fix regression requiring the IngestionBucketURL to be set for datalake clusters (new unreleased feature).

Release v755

February 12, 2026

Add new /v1/kafka/fetch HTTP/JSON endpoint for manually issuing Fetch requests against the Agents.
Add convenience endpoint GET /v1/kafka/fetch_single_record?topic=TOPIC&partition=PARTITION&offset=OFFSET for fetching a single record by offset.
Add convenience endpoint GET /v1/kafka/topics/{topic}/partitions/{partition}/records/{offset} as a REST-style alternative for fetching a single record by offset.
Due to false positives, skip the Unregistered WarpStream Environment Variables diagnostic if the agents is deployed as a Kubernetes service with a name that begins with "warpstream_" or "warpstream-".

Release v754

February 12, 2026

Fixed a bug introduced in v746 where the Agents could get deadlocked after experiencing tends of thousands of errors.

Release v753

February 11, 2026

[experimental] Add -disableOutOfRangeOrbit flag (env: WARPSTREAM_DISABLE_OUT_OF_RANGE_ORBIT): when enabled, orbit-managed topics return KAFKA_STORAGE_ERROR instead of OFFSET_OUT_OF_RANGE in the fetch path, preventing clients from resetting offsets when data has not yet been replicated from the source cluster.

Release v752

February 11, 2026

Redact agent key from startup logs.
Tableflow: Fix rare bug in parquet footer length tracking that could cause compaction jobs to fail with "unsupported thrift type" errors when reading input files.

Release v751

February 9, 2026

Fix default value for -batchMaxCompressedSizeBytes to be 16MB instead of a value higher than the max.

Release v750

February 9, 2026

Warning: There is a problem with that version fixed in v751. Please upgrade to v751 directly and skip v750.

Flushing behaviour update
- Before v750, the agent would flush a new file if it was over your configured -batchMaxSizeBytes value, and its default value was 4MiB. This means that the agent would not create files that contain more than 4i of uncompressed data by default.
- After this release, by default, the Agent will create a new file if it estimates that it's going to be more than 1MiB after compression by default.
- In more details:
  - This version introduces a -batchMaxCompressedSizeBytes flag (and the corresponding WARPSTREAM_BATCH_MAX_COMPRESSED_SIZE_BYTES environment variable).
  - If you define just -batchMaxSizeBytes, then -batchMaxCompressedSizeBytes will not be used, the agent will use exactly your uncompressed bytes limit to choose when to create a new file.
  - If you define just -batchMaxCompressedSizeBytes, then -batchMaxSizeBytes will be automatically set to a very high value (64MiB)
  - If you define no flags, then -batchMaxCompressedSizeBytes will be set to 1MiB and -batchMaxSizeBytes will be set to 64MiB.
  - If you define both, a new file will be created when it becomes higher than either limit.
SyncGroup handler now returns validation errors (invalid group ID, missing member ID, etc.) embedded in the response instead of as a top-level error. This improves compatibility with Kafka clients and ensures these expected protocol errors are not logged/metriced as server errors.
Tableflow:
- Fix BigQuery integration bug, where orphaned tables were preventing table recreation.
- Update ingestion to handle empty topics gracefully
- Fix timestamp field indexing for JSON format: timezone is not required anymore for fields with type timestamp.
Fix bug in batcher which would increase the number of files flushed when disable_only_flush_after_timeout is set to true.
Include virtual cluster ID in auto-generated client IDs for Managed Data Pipelines. This ensures it works with VirtualClusterIDClientStrictValidation enabled.
Improve performance of metrics code by swapping a mutex for an RWMutex.
Bump franz-go package from v1.18.1 to v1.20.6.
Add low severity diagnostic to detect misspelled environment variables beginning with WARPSTREAM_.
Fix noisy ACL denial logs for "all topics" metadata requests. When clients refresh metadata for all topics, unauthorized topics are now silently filtered without logging denials (per Kafka protocol behavior).

Release v749

February 2, 2026

Tableflow: Support skip_raw_record_values that doesn't store the raw record values in the produced data files.
Bump build to go 1.25.6 to fix vulnerability CVE-2025-61726.

Release v748

January 30, 2026

Return an error when MTLS is enabled (without -spiffeMTLSAuthentication) but the agent is unable to determine the username from the client certificate subject.
Add a new push diagnostic that fires when the timeout on produce requests is less than 5 seconds. This helps identify clients with misconfigured timeout settings that may cause issues with WarpStream's object storage-based architecture.
Allow clients to reauthenticate via SASL even when the control plane is down for Ripcord mode.
Tableflow:
- Add support for custom partitioning.
- Add support for compressing data files, default being snappy.
- New metrics:
  - warpstream.tableflow_partitions_count: Total number of table partitions in the cluster.
  - warpstream.tableflow_partitions_limit: Maximum number of table partitions allowed in the cluster.

Release v747

January 29, 2026

Buffer audit logs agent side in case of failure to publish them.
Patch CVE-2025-15467 & CVE-2025-69419 vulnerabilities in libssl3/libcrypto3.
Tableflow:
- Fix decoding of JSON records with date fields serialized in the "YYYY-MM-DD" format.
- Update how we track source data timestamp metadata (kafka timestamps) for more efficient retention enforcement.

Release v746

January 27, 2026

Flushes will wait for the entire batch timeout even if the batch max size bytes limit is exceeded. Files will be split and flushed in parallel in accordance to the batch max size bytes limit.

Release v745

January 23, 2026

Add support for SASL OAUTHBEARER authentication mechanism in the WarpStream Agent for Kafka clients that support it.
New flags and environment variables:
-saslOAuthIssuerURL / WARPSTREAM_SASL_OAUTH_ISSUER_URL: The OAuth issuer URL used to validate tokens presented by clients.
-saslOAuthAudience / WARPSTREAM_SASL_OAUTH_AUDIENCE: The expected audience claim in the OAuth tokens presented by clients.
When these settings are configured, the Agent will accept OAuth tokens from clients and validate them against the specified issuer and audience.
ACL rules can be created with OAuth principals in the format User:<subject> to allow or deny access based on the subject claim in the token.
Tableflow: Add support for Protobuf schemas.

Release v744

January 21, 2026

Fix issue with lightning topics configuration validation in previous versions that would prevent the agents from starting on some machines (introduced in v737)
Tableflow: Add memory safeguard to prevent potential OOM when ingesting data spanning many partitions.