Audit Logs

This page describes how to setup WarpStream Audit Logs.

Audit Logs are currently available as an Early Access (EA) feature. Please contact us if you'd like to register to be included in the Early Access program.

Introduction

WarpStream Audit Logs provide a way to capture, protect, and preserve Kafka authentication actions, authorization actions, and organization operations into a Kafka topic for Pro and Enterprise clusters. Audit logs also track account-level operations such as creating, deleting, and modifying WarpStream resources such as API keys, Kafka clusters, user accounts, etc.

WarpStream Audit Logs are produced into a fully-managed WarpStream cluster deployed on WarpStream's cloud infrastructure. Audit Logs can either be accessed through the WarpStream Console, or consumed using the Kafka protocol and exported anywhere. Audit Logs are retained for 90 days.

Getting Started

Pre-requisites: you need to use the agent version v736 or above.

In order to enable Audit Logs, navigate to the "Audit" section from the Console left panel.

Once enabled Audit Logs will start flowing:

• Cluster audit logs: all Pro and Enterprise clusters will emit audit logs (following the convention described here)

• Platform audit logs: all requests to the WarpStream console except GET requests will emit an audit log

Audit Logs are visible under the "Events" tab:

To consume Audit Logs using a Kafka client, follow the instructions for connecting to the cluster in the Audit section.

Audit Logs structure

WarpStream audit logs follow the CloudEvents spec and conform to the schema described here.

Here are a few examples:

• A Kafka failure

{
  "datacontenttype": "application/json",
  "data": {
    "requestMetadata": {
      "clientId": "warpstream_session_id=caxxxx,warpstream_client_lib=franzgo,warpstream_ioskh=api-xxxx.kafka.discoveryv2.prod-w.eu-central-1.warpstream.com,ws_vci=vci_xxx",
      "correlationId": "4",
      "clientAddress": [
        {
          "ip": "10.0.189.33:43006"
        }
      ]
    },
    "result": {
      "data": {
        "errorMessage": "The group id does not exist.",
        "errorCode": 69
      },
      "status": "FAILURE"
    },
    "serviceLocation": {
      "provider": "aws",
      "region": "eu-central-1"
    },
    "request": {
      "accessType": "MODIFICATION",
      "data": {
        "groupName": "deletion/franzgo/0/per_node/benchmark-default-7584f676c7-7xnqd"
      }
    },
    "authenticationInfo": {
      "credentials": {
        "mechanism": "SASL_SCRAM",
        "idSecretCredentials": {
          "credentialId": "ccun_xxxx"
        }
      }
    },
    "cloudResources": [
      {
        "scope": {
          "resources": [
            {
              "resourceId": "vcn_prod_w_benchmark_1",
              "type": "KAFKA_CLUSTER"
            }
          ]
        }
      },
      {
        "scope": {
          "resources": [
            {
              "resourceId": "deletion/franzgo/0/per_node/benchmark-default-7584f676c7-7xnqd",
              "type": "GROUP"
            }
          ]
        }
      }
    ],
    "methodName": "kafka.DeleteGroups",
    "serviceName": "crn://warpstream/wi_xxx/kafka_cluster=vci_xxxx"
  },
  "subject": "crn://warpstream/wi_xxxx/kafka_cluster=vci_xxxx",
  "specversion": "1.0",
  "id": "f9e1c2d0-a556-4255-bc24-40ae6eecea66",
  "source": "crn://warpstream/wi_xxxx/kafka_cluster=vci_xxxx",
  "time": "2025-12-17T08:49:53.726976069Z",
  "type": "warpstream.kafka/request"
}

• A Kafka success:

{
    "datacontenttype": "application/json",
    "data": {
        "requestMetadata": {
            "clientId": "warpstream_session_id=xxxx,ws_az=asia-southeast1-b,warpstream_client_lib=franzgo,warpstream_ioskh=,ws_vci=vci_xxxx",
            "correlationId": "3",
            "clientAddress": [
                {
                    "ip": "10.228.7.2:41932"
                }
            ]
        },
        "result": {
            "data": {
                "errorCode": 0
            },
            "status": "SUCCESS"
        },
        "serviceLocation": {
            "provider": "gcp",
            "region": "asia-southeast1"
        },
        "request": {
            "accessType": "MODIFICATION",
            "data": {
                "configs": [
                    {
                        "name": "warpstream.topic.type",
                        "value": "classic"
                    },
                    {
                        "name": "retention.ms",
                        "value": "86400000"
                    },
                    {
                        "name": "cleanup.policy",
                        "value": "compact,delete"
                    },
                    {
                        "name": "delete.retention.ms",
                        "value": "21600000"
                    },
                    {
                        "name": "min.compaction.lag.ms",
                        "value": "10800000"
                    },
                    {
                        "name": "max.compaction.lag.ms",
                        "value": "3600000"
                    },
                    {
                        "name": "warpstream.schema.registry.type",
                        "value": "STANDARD"
                    }
                ],
                "replicationFactor": 1,
                "numPartitions": 4,
                "name": "compacted_per_node_benchmark-default-cd47b8768-qzhfb"
            }
        },
        "authenticationInfo": {
            "credentials": {
                "mechanism": "SASL_SCRAM",
                "idSecretCredentials": {
                    "credentialId": "ccun_xxxx"
                }
            }
        },
        "cloudResources": [
            {
                "scope": {
                    "resources": [
                        {
                            "resourceId": "vcn_benchmark_staging_b_14",
                            "type": "KAFKA_CLUSTER"
                        }
                    ]
                }
            }
        ],
        "methodName": "kafka.CreateTopics",
        "serviceName": "crn://warpstream/wi_xxxx/kafka_cluster=vci_xxxx"
    },
    "subject": "crn://warpstream/wi_xxxx/kafka_cluster=vci_xxxx",
    "specversion": "1.0",
    "id": "4a4ebc83-2ee6-4531-bea6-9339053ef9fd",
    "source": "crn://warpstream/wi_xxxx/kafka_cluster=vci_xxxx",
    "time": "2025-12-17T09:29:36.120450058Z",
    "type": "warpstream.kafka/request"
}

• A platform success:

{
    "datacontenttype": "application/json",
    "data": {
        "requestMetadata": {
            "requestPath": "/api/v1/list_agent_pools"
        },
        "result": {
            "status": "SUCCESS"
        },
        "request": {
            "accessType": "MODIFICATION"
        },
        "authenticationInfo": {
            "principal": {
                "email": "[email protected]",
                "confluentUser": {
                    "resourceId": "ui_xxxx"
                }
            },
            "result": "SUCCESS"
        },
        "methodName": "list_agent_pools",
        "serviceName": "crn://warpstream/unknown="
    },
    "subject": "crn://warpstream/unknown=",
    "specversion": "1.0",
    "id": "5f6af1dd-be65-4011-a401-4a658bef151b",
    "source": "crn://warpstream/unknown=",
    "time": "2025-12-17T09:41:09.965911245Z",
    "type": "warpstream.api/request"
}

Consume Audit Logs

Under the "Credentials" tab of the Audit page, users can create SASL credentials that can then be used to consume from the Audit Logs topic. The "Connect" tab contains everything you need to know to start consuming with code samples.

Billing for Audit Logs

Audit Logs are billed slightly differently than WarpStream BYOC clusters because the cluster is hosted on WarpStream's cloud infrastructure. The following dimensions and unit prices apply to Audit Logs:

Dimension	Definition
Uncompressed GiB Written	Logical data written to the WarpStream cluster. Metered as the sum of logical data written to the cluster.
GiB-Minutes	Logical data stored on the WarpStream cluster.
Network ingress	Network ingress to the WarpStream cluster
Network egress	Network egress from the WarpStream cluster

Pricing

Each dimension has a unit price. Unit prices for Uncompressed GiB Written are tiered, based on monthly consumption.

Uncompressed GiB Written

The Uncompressed GiB Written metric is billed as follows:

Tier Size	Price per GiB
0 - 5 TiB	$0.02
5 - 25 TiB	$0.015
25 - 75 TiB	$0.0113
75 - 250 TiB	$0.0084
250 - 500 TiB	$0.0063
>500 TiB	$0.0047

Note that these tiers likely exceed any reasonable expectations for the volume of Audit Logs that would be produced for nearly any WarpStream use case. Audit Logs volume varies based on how often each logged event occurs, but the vast majority of use cases will produce no more than 200 KiB/sec (<500 GiB/month), resulting in charges of less than $10/month for writes. For more information on how tiered pricing works in WarpStream, please refer to Tiered Pricing.

GiB-Minutes

GiB-Minutes measures the storage volume of Audit Logs retained in the WarpStream Audit Logs cluster. The price per GiB-Minute of logs is $0.00000046926/GiB-minute. Most users should expect less than $10/month in storage charges for Audit Logs.

Network ingress

Network ingress is the volume of compressed data written to the cluster. The price per GiB of network ingress is $0.04/GiB. Audit Logs that are produced to the Audit Logs cluster are compressed, and in most cases, charges for network ingress should not exceed $10/month for Audit Logs.

Network egress

Network egress is the volume of compressed data consumed from the cluster. Pricing per GiB of network egress is as follows:

Destination	Price per GiB
Same cloud provider	$0.04
Internet	$0.12

Network egress is billed based on the location of the consumer. For example, if a Kafka client running in us-west-2 consumes from an Audit Logs cluster in us-east-1, consumption will be charged at $0.04/GiB consumed. If the Kafka client is running on a laptop, the data will egress from AWS and traverse the internet, which will be charged at $0.012/GiB.

The estimated monthly charges for network egress depend on the user's consumption pattern, in addition to the volume of logs that are being produced.