# WarpStream ## WarpStream - [Introduction](https://docs.warpstream.com/warpstream/overview/readme.md): The official documentation for WarpStream, a diskless, Apache Kafka-compatible data streaming platform built directly on top of cloud object stores such as S3. - [Architecture](https://docs.warpstream.com/warpstream/overview/architecture.md): This page provides an overview of WarpStream's architecture. - [Service Discovery](https://docs.warpstream.com/warpstream/overview/architecture/service-discovery.md): This page explains how WarpStream's custom service discovery mechanism works, as well as how it interacts with Kafka's service discovery system. - [Write Path](https://docs.warpstream.com/warpstream/overview/architecture/write-path.md): The following page is a summary of WarpStream's approach to persisting data produced from clients. - [Read Path](https://docs.warpstream.com/warpstream/overview/architecture/read-path.md): The following page describes the mechanism for serving Fetch requests from WarpStream. - [Life of a Request (Simplified)](https://docs.warpstream.com/warpstream/overview/architecture/life-of-a-request.md): This page explains the flow for a Kafka Produce or Fetch request from client to WarpStream Agent to WarpStream Cloud and back again. - [Change Log](https://docs.warpstream.com/warpstream/overview/change-log.md): Contains a history of changes made to the Agent. - [Migrations](https://docs.warpstream.com/warpstream/overview/change-log/migrations.md) - [Install the WarpStream Agent / CLI](https://docs.warpstream.com/warpstream/getting-started/install-the-warpstream-agent.md): This page contains instructions for installing the WarpStream Agent via CURL, Docker, or raw binaries. - [Run the Agents Locally](https://docs.warpstream.com/warpstream/getting-started/run-the-agent-locally.md): Instructions on how to run the WarpStream Agent locally for testing / development purposes. - ["Hello World" for Apache Kafka](https://docs.warpstream.com/warpstream/getting-started/hello-world-using-kafka.md): This page goes through the basic "Hello World" Kafka functionality (produce and consume) with WarpStream. - [Deploy the Agents](https://docs.warpstream.com/warpstream/agent-setup/deploy.md): How to deploy the Agents. - [Kubernetes Known Issues](https://docs.warpstream.com/warpstream/agent-setup/deploy/kubernetes-known-issues.md) - [Rolling Restarts and Upgrades](https://docs.warpstream.com/warpstream/agent-setup/deploy/rolling-restarts-and-upgrades.md): This page explains how rolling restarts and upgrades work in WarpStream. - [Object Storage Configuration](https://docs.warpstream.com/warpstream/agent-setup/different-object-stores.md): This page describes how to properly configure object storage for BYOC Agent deployments. - [Set up Monitoring](https://docs.warpstream.com/warpstream/agent-setup/monitor-the-warpstream-agents.md) - [Diagnostics](https://docs.warpstream.com/warpstream/agent-setup/monitor-the-warpstream-agents/diagnostics.md) - [Important Metrics and Logs](https://docs.warpstream.com/warpstream/agent-setup/monitor-the-warpstream-agents/important-metrics-and-logs.md): On this page, we include a sample list of the most important logs and metrics emitted by the Agents. - [Hosted Prometheus Endpoint](https://docs.warpstream.com/warpstream/agent-setup/monitor-the-warpstream-agents/hosted-prometheus-endpoint.md): This page describes how to use WarpStream's hosted prometheus endpoint for collecting control plane metrics. - [Recommended List of Alerts](https://docs.warpstream.com/warpstream/agent-setup/monitor-the-warpstream-agents/recommended-list-of-alerts.md): On this page we list the key metrics you should create alerts on. - [Monitoring Consumer Groups](https://docs.warpstream.com/warpstream/agent-setup/monitor-the-warpstream-agents/monitoring-consumer-groups.md): How to monitor your consumer groups. - [Datadog Integration](https://docs.warpstream.com/warpstream/agent-setup/monitor-the-warpstream-agents/datadog-integration.md) - [Pre-made Datadog Dashboard](https://docs.warpstream.com/warpstream/agent-setup/monitor-the-warpstream-agents/premade-datadog-dashboard.md) - [Pre-made Grafana Dashboard](https://docs.warpstream.com/warpstream/agent-setup/monitor-the-warpstream-agents/pre-made-grafana-dashboard.md) - [Infrastructure as Code](https://docs.warpstream.com/warpstream/agent-setup/infrastructure-as-code.md): This section contains information on how to configure WarpStream using "Infrastructure as Code" tools like Terraform and Helm. - [Terraform Provider](https://docs.warpstream.com/warpstream/agent-setup/infrastructure-as-code/terraform-provider.md): Links to WarpStream Terraform provider resources. - [Helm charts](https://docs.warpstream.com/warpstream/agent-setup/infrastructure-as-code/helm-charts.md): WarpStream Helm charts. - [Terraform Modules](https://docs.warpstream.com/warpstream/agent-setup/infrastructure-as-code/terraform-modules.md) - [Protecting important resources from accidental deletion](https://docs.warpstream.com/warpstream/agent-setup/infrastructure-as-code/protecting-important-resources-from-accidental-deletion.md) - [Configure Clients](https://docs.warpstream.com/warpstream/kafka/configure-kafka-client.md): This pages explains how to configure your Apache Kafka client with WarpStream. - [Tuning for Performance](https://docs.warpstream.com/warpstream/kafka/configure-kafka-client/tuning-for-performance.md): Instructions on how to tune various Kafka clients for performance with WarpStream. - [Automatic Fetch Size Auto-tuning](https://docs.warpstream.com/warpstream/kafka/configure-kafka-client/client-configuration-auto-tuning.md): How to allow the Agents to auto-tune settings - [Configure Clients to Eliminate AZ Networking Costs](https://docs.warpstream.com/warpstream/kafka/configure-kafka-client/configure-clients-to-eliminate-az-networking-costs.md): How to configure your Kafka clients to keep all traffic zone-local. - [Force Interzone Load Balancing](https://docs.warpstream.com/warpstream/kafka/configure-kafka-client/configure-clients-to-eliminate-az-networking-costs/tune-interzone-load-balancing.md): This page described how to force inter-zone load balancing for clients that don't regularly refresh their Metadata. - [Configuring Kafka Client ID Features](https://docs.warpstream.com/warpstream/kafka/configure-kafka-client/configuring-kafka-client-id-features.md): WarpStream uses the client id setting that you set on Apache Kafka clients to control how certain features are activated - [Client Metrics (KIP-714)](https://docs.warpstream.com/warpstream/kafka/configure-kafka-client/client-metrics-kip-714.md): Configure your Kafka clients to push runtime metrics to WarpStream using KIP-714 client metrics subscriptions, then query them from the Events Explorer. - [Hosted Metadata Endpoint](https://docs.warpstream.com/warpstream/kafka/configure-kafka-client/administrate-your-byoc-clusters-with-serverless.md): This page explains how you can leverage WarpStream's hosted metadata endpoint to administer your WarpStream BYOC clusters from anywhere. - [Port Forwarding (K8s)](https://docs.warpstream.com/warpstream/kafka/configure-kafka-client/port-forwarding-k8s.md): This page provides documentation on how to solve a common problem with WarpStream: connecting to Agents deployed in K8s by port-forwarding. - [Known Issues](https://docs.warpstream.com/warpstream/kafka/configure-kafka-client/known-issues.md) - [Manage Security](https://docs.warpstream.com/warpstream/kafka/manage-security.md): This page describes how to manage the various security aspects of WarpStream's Kafka product. - [ACLs](https://docs.warpstream.com/warpstream/kafka/manage-security/configure-acls.md): The following page provides a management guide for ACLs (Access Control Lists). This document provides instructions on how to enable, disable, create, and delete ACLs for your WarpStream clusters. - [TLS](https://docs.warpstream.com/warpstream/kafka/manage-security/protect-data-in-motion-with-tls-encryption.md): This page described how to configure the Agents to terminate TLS. - [SASL Authentication](https://docs.warpstream.com/warpstream/kafka/manage-security/sasl-authentication.md): This page describe how to configure the WarpStream Agents with SASL authentication. - [Mutual TLS (mTLS)](https://docs.warpstream.com/warpstream/kafka/manage-security/mutual-tls-mtls.md): This page describes how to configure the Agents to enforce mTLS. - [SASL/OAUTHBEARER Authentication](https://docs.warpstream.com/warpstream/kafka/manage-security/sasl-oauthbearer-authentication.md) - [Manage Connectors](https://docs.warpstream.com/warpstream/kafka/manage-connectors.md): This page describes the various options for connector products in WarpStream. - [Managed Data Pipelines](https://docs.warpstream.com/warpstream/kafka/manage-connectors/bento.md): This page explains how to use the WarpStream Managed Data Pipelines functionality. - [Cookbooks](https://docs.warpstream.com/warpstream/kafka/manage-connectors/bento/cookbooks.md): This page contains a collection of WarpStream-specific Bento recipes to help you accomplish common tasks. - [Deploy (advanced)](https://docs.warpstream.com/warpstream/kafka/advanced-agent-deployment-options.md): This section explains how to configure more advanced deployment options for the WarpStream BYOC Agents. - [Agent Roles](https://docs.warpstream.com/warpstream/kafka/advanced-agent-deployment-options/splitting-agent-roles.md): How to run different services on different sets of Agents - [Agent Groups](https://docs.warpstream.com/warpstream/kafka/advanced-agent-deployment-options/agent-groups.md): How to split Agents for a cluster into different "groups". - [Workload Identity Federation](https://docs.warpstream.com/warpstream/kafka/advanced-agent-deployment-options/workload-identity-federation.md): Authenticate WarpStream Agents to the control plane with short-lived, federated OIDC tokens instead of a long-lived static Agent Key. - [Low Latency Clusters](https://docs.warpstream.com/warpstream/kafka/advanced-agent-deployment-options/low-latency-clusters.md): Configure the WarpStream Agent with lightning topics and a S3 Express, DynamoDB or Spanner storage layer to reduce Produce latency. - [S3 Express](https://docs.warpstream.com/warpstream/kafka/advanced-agent-deployment-options/low-latency-clusters/s3-express.md) - [GCP Rapid Buckets (beta)](https://docs.warpstream.com/warpstream/kafka/advanced-agent-deployment-options/low-latency-clusters/gcp-rapid-buckets-beta.md) - [AWS DynamoDB](https://docs.warpstream.com/warpstream/kafka/advanced-agent-deployment-options/low-latency-clusters/aws-dynamo-db.md) - [Google Spanner (beta)](https://docs.warpstream.com/warpstream/kafka/advanced-agent-deployment-options/low-latency-clusters/google-spanner-beta.md) - [Lightning topics](https://docs.warpstream.com/warpstream/kafka/advanced-agent-deployment-options/low-latency-clusters/lightning-topics.md): This page explains how to configure topics as Lightning Topics and what impact that will have. - [Multi-Region Clusters](https://docs.warpstream.com/warpstream/kafka/advanced-agent-deployment-options/multi-region.md) - [Migrating Control Plane Regions](https://docs.warpstream.com/warpstream/kafka/advanced-agent-deployment-options/migrating-regions.md) - [Network Architecture Considerations](https://docs.warpstream.com/warpstream/kafka/advanced-agent-deployment-options/configure-warpstream-agent-within-a-container-or-behind-a-proxy.md): This page describes a variety of different approaches that can be used to deploy WarpStream with more advanced network setups. - [Agent Configuration Reference](https://docs.warpstream.com/warpstream/kafka/advanced-agent-deployment-options/agent-configuration.md): Reference documentation for WarpStream Agent flags. - [Reducing Infrastructure Costs](https://docs.warpstream.com/warpstream/kafka/advanced-agent-deployment-options/reducing-infrastructure-costs.md): How to reduce infrastructure costs for WarpStream BYOC clusters. - [Ripcord](https://docs.warpstream.com/warpstream/kafka/advanced-agent-deployment-options/ripcord.md): Ripcord enables WarpStream Agents to continue ingesting data and accepting Produce requests even when the control plane is unavailable. - [Reference](https://docs.warpstream.com/warpstream/kafka/reference.md): This section contains various reference information about the WarpStream Kafka product. - [Compression](https://docs.warpstream.com/warpstream/kafka/reference/compression.md): Overview of how compression works in WarpStream and how to configure it. - [Protocol and Feature Support](https://docs.warpstream.com/warpstream/kafka/reference/protocol-and-feature-support.md) - [Kafka vs WarpStream Configuration Reference](https://docs.warpstream.com/warpstream/kafka/reference/protocol-and-feature-support/kafka-vs-warpstream-configuration-reference.md): The purpose of this page is to compare configuration options in Kafka and WarpStream 1:1 as applicable. - [Compacted topics](https://docs.warpstream.com/warpstream/kafka/reference/protocol-and-feature-support/compacted-topics.md): Information about compacted topics in WarpStream. - [Broker Configuration Reference](https://docs.warpstream.com/warpstream/kafka/reference/protocol-and-feature-support/broker-configuration-reference.md) - [Topic Configuration Reference](https://docs.warpstream.com/warpstream/kafka/reference/protocol-and-feature-support/topic-configuration-reference.md) - [HTTP Endpoints](https://docs.warpstream.com/warpstream/kafka/reference/protocol-and-feature-support/http-endpoints.md): This pages describes the HTTP endpoints available in the WarpStream Kafka product. - [Partitions Auto-Scaler](https://docs.warpstream.com/warpstream/kafka/reference/partitions-auto-scaler-beta.md): This page describes the functionality of WarpStream's Partitions Auto-Scaler. - [Serverless Clusters](https://docs.warpstream.com/warpstream/kafka/reference/serverless-clusters.md) - [Partition Assignment Strategies](https://docs.warpstream.com/warpstream/kafka/reference/partition-assignment-strategies.md): This page explains the various different partition assignment strategies in WarpStream. - [Control Plane Utilization](https://docs.warpstream.com/warpstream/kafka/reference/control-plane-utilization.md): The purpose of this page is to provide an intuitive understanding of the factors that influence WarpStream Control Plane utilization, and how to reduce it to make your workloads more efficient. - [Benchmark](https://docs.warpstream.com/warpstream/kafka/benchmarking.md): How to Benchmark WarpStream. - [Orbit (Cluster Linking)](https://docs.warpstream.com/warpstream/kafka/orbit.md): Replicate and migrate Kafka clusters. - [Auto Migration](https://docs.warpstream.com/warpstream/kafka/orbit/auto-migration.md): Migrate to WarpStream with zero client restarts - [Tableflow Setup](https://docs.warpstream.com/warpstream/tableflow/tableflow.md): This page describes how to setup WarpStream Tableflow. - [Monitoring Tableflow](https://docs.warpstream.com/warpstream/tableflow/monitoring.md): Key metrics, diagnostics, events, and recommended alerts for Tableflow clusters. - [Iceberg REST Catalog](https://docs.warpstream.com/warpstream/tableflow/iceberg-catalog.md): This page describes how to connect with Tableflow's read-only REST catalog. - [Integrate With Query Engines and External Catalogs](https://docs.warpstream.com/warpstream/tableflow/catalogs-and-query-engines.md) - [Snowflake](https://docs.warpstream.com/warpstream/tableflow/catalogs-and-query-engines/snowflake.md): This pages describes how to integrate Tableflow with Snowflake so that you can query Iceberg tables created by WarpStream in Snowflake. - [Databricks Unity](https://docs.warpstream.com/warpstream/tableflow/catalogs-and-query-engines/databricks.md): This pages describes how to integrate Tableflow with Databricks so that you can query Iceberg tables created by WarpStream in Databricks. - [BigQuery](https://docs.warpstream.com/warpstream/tableflow/catalogs-and-query-engines/bigquery.md): This page describes how to integrate Tableflow with Google BigQuery so that you can query Iceberg tables created by WarpStream directly in BigQuery. - [BigLake](https://docs.warpstream.com/warpstream/tableflow/catalogs-and-query-engines/biglake.md): This page describes how to integrate with Google BigLake Metastore so that you can query Iceberg tables created by Tableflow directly in BigQuery via BigLake. - [DuckDB](https://docs.warpstream.com/warpstream/tableflow/catalogs-and-query-engines/duckdb.md) - [ClickHouse](https://docs.warpstream.com/warpstream/tableflow/catalogs-and-query-engines/clickhouse.md) - [AWS Glue](https://docs.warpstream.com/warpstream/tableflow/catalogs-and-query-engines/aws-glue.md) - [Amazon Athena](https://docs.warpstream.com/warpstream/tableflow/catalogs-and-query-engines/amazon-athena.md) - [Hive Metadata Store](https://docs.warpstream.com/warpstream/tableflow/catalogs-and-query-engines/hive-metadata-store.md) - [API Reference](https://docs.warpstream.com/warpstream/tableflow/api-reference.md) - [Setup](https://docs.warpstream.com/warpstream/schema-registry/warpstream-byoc-schema-registry.md): This page explains how to setup WarpStream's BYOC Schema Registry. - [Enforce Schemas](https://docs.warpstream.com/warpstream/schema-registry/schema-validation.md): This page describes how to configure the topic and agent to perform schema validation. - [Schema Linking](https://docs.warpstream.com/warpstream/schema-registry/warpstream-schema-linking.md): Replicate and migrate schemas to WarpStream BYOC Schema Registry. - [Manage Security](https://docs.warpstream.com/warpstream/schema-registry/manage-security.md): This section explains how to manage security for WarpStream's Schema Registry product. - [Basic Authentication](https://docs.warpstream.com/warpstream/schema-registry/manage-security/basic-authentication.md) - [Billing](https://docs.warpstream.com/warpstream/reference/billing.md): An overview of WarpStream's billing model. - [Direct billing](https://docs.warpstream.com/warpstream/reference/billing/direct-billing.md) - [AWS Marketplace](https://docs.warpstream.com/warpstream/reference/billing/aws-marketplace.md) - [Chargebacks](https://docs.warpstream.com/warpstream/reference/billing/chargebacks.md) - [Audit Logs](https://docs.warpstream.com/warpstream/reference/audit-logs.md): This page describes how to setup WarpStream Audit Logs. - [Events](https://docs.warpstream.com/warpstream/reference/events.md) - [MCP Server](https://docs.warpstream.com/warpstream/reference/mcp-server.md): This page describes the WarpStream Console MCP (Model Context Protocol) server for IDEs and AI assistants. - [Accessing the Confluent Support Portal](https://docs.warpstream.com/warpstream/reference/accessing-the-confluent-support-portal.md): This page describes how to access the Confluent customer support portal. - [Security and Privacy Considerations](https://docs.warpstream.com/warpstream/reference/security-and-privacy-considerations.md): This page describes the various security and privacy considerations for WarpStream's BYOC deployment model. - [Manage Console Access](https://docs.warpstream.com/warpstream/reference/manage-console-access.md): This section describes the various option for managing access to WarpStream's console. - [Enable SAML Single Sign-on (SSO)](https://docs.warpstream.com/warpstream/reference/manage-console-access/enable-saml-single-sign-on-sso.md): This page explains how to configure WarpStream's SAML / SSO functionality. - [Trusted Domains](https://docs.warpstream.com/warpstream/reference/manage-console-access/trusted-domains.md): This page explains how to configure WarpStream's trusted domains functionality. - [Workspaces and Access Controls](https://docs.warpstream.com/warpstream/reference/manage-console-access/workspaces-and-access-controls.md): Overview of access control in the WarpStream web console. - [Secrets Overview](https://docs.warpstream.com/warpstream/reference/secrets-overview.md): This page provides an overview of the various secrets defined by WarpStream. - [API Reference](https://docs.warpstream.com/warpstream/reference/api-reference.md): This section of the documentation contains the API reference for the WarpStream public API endpoints. - [API Keys](https://docs.warpstream.com/warpstream/reference/api-reference/api-keys.md): APIs for interacting with API Keys. - [Create](https://docs.warpstream.com/warpstream/reference/api-reference/api-keys/create.md): Create a new API Key. - [Delete](https://docs.warpstream.com/warpstream/reference/api-reference/api-keys/delete.md): Delete an API key by ID. The deleted API key must belong to the same workspace as the API key used to authenticate the request. - [List](https://docs.warpstream.com/warpstream/reference/api-reference/api-keys/list.md): List all API Keys in a workspace. - [Virtual Clusters](https://docs.warpstream.com/warpstream/reference/api-reference/virtual-clusters.md): APIs for interacting with Virtual Clusters. - [Create](https://docs.warpstream.com/warpstream/reference/api-reference/virtual-clusters/create.md): Create a new Virtual Cluster. - [Delete](https://docs.warpstream.com/warpstream/reference/api-reference/virtual-clusters/delete.md): Delete a Virtual Cluster by ID and name. - [Describe](https://docs.warpstream.com/warpstream/reference/api-reference/virtual-clusters/describe.md): Describe a specific Virtual Cluster by ID. - [List](https://docs.warpstream.com/warpstream/reference/api-reference/virtual-clusters/list.md): List all Virtual Clusters in your account. - [Create a Multi-Region Cluster](https://docs.warpstream.com/warpstream/reference/api-reference/virtual-clusters/create-a-multi-region-cluster.md) - [DescribeConfiguration](https://docs.warpstream.com/warpstream/reference/api-reference/virtual-clusters/describeconfiguration.md): Describe the configuration of a specific Virtual Cluster by ID. - [UpdateConfiguration](https://docs.warpstream.com/warpstream/reference/api-reference/virtual-clusters/updateconfiguration.md): Update the configuration of a specific Virtual Cluster by ID. - [Rename](https://docs.warpstream.com/warpstream/reference/api-reference/virtual-clusters/rename.md): Rename a virtual cluster. - [Update Tier](https://docs.warpstream.com/warpstream/reference/api-reference/virtual-clusters/rename-1.md): Update a virtual cluster's tier. - [Virtual Clusters Credentials](https://docs.warpstream.com/warpstream/reference/api-reference/virtual-clusters-credentials.md): APIs for interacting with Virtual Clusters Credentials. - [Create](https://docs.warpstream.com/warpstream/reference/api-reference/virtual-clusters-credentials/create.md): Create new Virtual Cluster Credentials. - [Delete](https://docs.warpstream.com/warpstream/reference/api-reference/virtual-clusters-credentials/delete.md): Delete a Virtual Cluster by ID and name. - [List](https://docs.warpstream.com/warpstream/reference/api-reference/virtual-clusters-credentials/list.md): List all Virtual Clusters Credentials of a given Virtual Cluster. - [ACLs](https://docs.warpstream.com/warpstream/reference/api-reference/acls.md) - [Create ACL](https://docs.warpstream.com/warpstream/reference/api-reference/acls/create-acl.md) - [List ACLs](https://docs.warpstream.com/warpstream/reference/api-reference/acls/list-acls.md) - [Delete ACLs](https://docs.warpstream.com/warpstream/reference/api-reference/acls/delete-acls.md) - [Monitoring](https://docs.warpstream.com/warpstream/reference/api-reference/monitoring.md): APIs for monitoring WarpStream and Consumers. - [Describe All Consumer Groups](https://docs.warpstream.com/warpstream/reference/api-reference/monitoring/describe-all-consumer-groups.md): Describe all consumer groups. - [Describe Account Quotas](https://docs.warpstream.com/warpstream/reference/api-reference/monitoring/describe-quotas.md): Describe account quotas. - [Pipelines](https://docs.warpstream.com/warpstream/reference/api-reference/pipelines.md) - [List Pipelines](https://docs.warpstream.com/warpstream/reference/api-reference/pipelines/list-pipelines.md): Lists all pipelines within a virtual cluster. - [Create Pipeline](https://docs.warpstream.com/warpstream/reference/api-reference/pipelines/create-pipeline.md): Creates a new pipeline in a virtual cluster. - [Delete Pipeline](https://docs.warpstream.com/warpstream/reference/api-reference/pipelines/delete-pipeline.md): Deletes a specific pipeline configuration. - [Describe Pipeline](https://docs.warpstream.com/warpstream/reference/api-reference/pipelines/describe-pipeline.md): Retrieves details about a specific pipeline, including its configurations. - [Create Pipeline Configuration](https://docs.warpstream.com/warpstream/reference/api-reference/pipelines/create-pipeline-configuration.md): Creates a configuration for a pipeline. - [Change Pipeline State](https://docs.warpstream.com/warpstream/reference/api-reference/pipelines/change-pipeline-state.md): Changes the state of an existing pipeline. - [Invitations](https://docs.warpstream.com/warpstream/reference/api-reference/invitations.md): APIs to manage invitations to a WarpStream account - [Create](https://docs.warpstream.com/warpstream/reference/api-reference/invitations/create.md): Create invitations. - [Invoices](https://docs.warpstream.com/warpstream/reference/api-reference/invoices.md): API endpoint for retrieving current and past invoices - [Get Pending Invoice](https://docs.warpstream.com/warpstream/reference/api-reference/invoices/get-pending-invoice.md) - [Get Past Invoice](https://docs.warpstream.com/warpstream/reference/api-reference/invoices/get-past-invoice.md) - [Get Invoice Breakdowns](https://docs.warpstream.com/warpstream/reference/api-reference/invoices/get-invoice-breakdowns.md): Provides a daily breakdown during a time period. - [Get Pending Chargebacks](https://docs.warpstream.com/warpstream/reference/api-reference/invoices/get-pending-invoice-1.md) - [Get Past Chargebacks](https://docs.warpstream.com/warpstream/reference/api-reference/invoices/get-pending-invoice-2.md) - [Tableflow](https://docs.warpstream.com/warpstream/reference/api-reference/tableflow.md) - [List Tables](https://docs.warpstream.com/warpstream/reference/api-reference/tableflow/list-tables.md) - [Get Table](https://docs.warpstream.com/warpstream/reference/api-reference/tableflow/get-table.md) - [Delete/Recreate Table](https://docs.warpstream.com/warpstream/reference/api-reference/tableflow/delete-recreate-table.md) - [Topics](https://docs.warpstream.com/warpstream/reference/api-reference/topics.md) - [Create topic](https://docs.warpstream.com/warpstream/reference/api-reference/topics/create-topic.md) - [List topics](https://docs.warpstream.com/warpstream/reference/api-reference/topics/list-topics.md) - [Describe topic](https://docs.warpstream.com/warpstream/reference/api-reference/topics/describe-topic.md) - [Update topic](https://docs.warpstream.com/warpstream/reference/api-reference/topics/update-topic.md) - [Delete topic](https://docs.warpstream.com/warpstream/reference/api-reference/topics/delete-topic.md) - [Undelete topic](https://docs.warpstream.com/warpstream/reference/api-reference/topics/undelete-topic.md) - [Client Metrics](https://docs.warpstream.com/warpstream/reference/api-reference/client-metrics.md): APIs for managing KIP-714 client metrics subscriptions on a virtual cluster. - [List Subscriptions](https://docs.warpstream.com/warpstream/reference/api-reference/client-metrics/list.md): List all client metrics subscriptions on a virtual cluster. - [Describe Subscription](https://docs.warpstream.com/warpstream/reference/api-reference/client-metrics/describe.md): Describe a single client metrics subscription by name. - [Update Subscriptions](https://docs.warpstream.com/warpstream/reference/api-reference/client-metrics/update.md): Create or update a batch of client metrics subscriptions. - [Delete Subscriptions](https://docs.warpstream.com/warpstream/reference/api-reference/client-metrics/delete.md): Delete a batch of client metrics subscriptions by name. - [Users](https://docs.warpstream.com/warpstream/reference/api-reference/users.md): APIs for interacting with Users - [Delete](https://docs.warpstream.com/warpstream/reference/api-reference/users/delete.md): Delete a User. - [List](https://docs.warpstream.com/warpstream/reference/api-reference/users/list.md): List all Users in the account. - [Update Roles](https://docs.warpstream.com/warpstream/reference/api-reference/users/update-roles.md): Assign and unassign a User's roles. - [User Roles](https://docs.warpstream.com/warpstream/reference/api-reference/user-roles.md): APIs for interacting with User Roles - [Create](https://docs.warpstream.com/warpstream/reference/api-reference/user-roles/create.md): Create a new User Role. - [Delete](https://docs.warpstream.com/warpstream/reference/api-reference/user-roles/delete.md): Delete a User Role. - [List](https://docs.warpstream.com/warpstream/reference/api-reference/user-roles/list.md): List all User Roles in the account. - [Update](https://docs.warpstream.com/warpstream/reference/api-reference/user-roles/update.md): Update a User Role. - [Workspaces](https://docs.warpstream.com/warpstream/reference/api-reference/workspaces.md): APIs for interacting with Workspaces. - [Create](https://docs.warpstream.com/warpstream/reference/api-reference/workspaces/create.md): Create a new Workspace. - [Delete](https://docs.warpstream.com/warpstream/reference/api-reference/workspaces/delete.md): Delete a workspace. - [List](https://docs.warpstream.com/warpstream/reference/api-reference/workspaces/list.md): List all Workspaces in your account. - [Rename](https://docs.warpstream.com/warpstream/reference/api-reference/workspaces/rename.md): Rename a Workspace. - [Orbit Auto Migration](https://docs.warpstream.com/warpstream/reference/api-reference/orbit-auto-migration.md): Migrate to WarpStream with zero client restarts - [Initiate](https://docs.warpstream.com/warpstream/reference/api-reference/orbit-auto-migration/initiate.md): Start an Orbit Auto Migration on one or more topics - [Abort](https://docs.warpstream.com/warpstream/reference/api-reference/orbit-auto-migration/abort.md): Cancels an ongoing migration - [Status](https://docs.warpstream.com/warpstream/reference/api-reference/orbit-auto-migration/status.md): Query the migration status of auto-migration topics. - [CLI Reference](https://docs.warpstream.com/warpstream/reference/cli-reference.md) - [warpstream agent](https://docs.warpstream.com/warpstream/reference/cli-reference/warpstream-agent.md) - [warpstream demo](https://docs.warpstream.com/warpstream/reference/cli-reference/warpstream-demo.md) - [warpstream cli](https://docs.warpstream.com/warpstream/reference/cli-reference/warpstream-cli.md) - [benchmark-consumer](https://docs.warpstream.com/warpstream/reference/cli-reference/warpstream-cli/benchmark-consumer.md) - [benchmark-producer](https://docs.warpstream.com/warpstream/reference/cli-reference/warpstream-cli/benchmark-producer.md) - [console-consumer](https://docs.warpstream.com/warpstream/reference/cli-reference/warpstream-cli/console-consumer.md) - [console-producer](https://docs.warpstream.com/warpstream/reference/cli-reference/warpstream-cli/console-producer.md) - [consumer-group-lag](https://docs.warpstream.com/warpstream/reference/cli-reference/warpstream-cli/consumer-group-lag.md) - [diagnose-record](https://docs.warpstream.com/warpstream/reference/cli-reference/warpstream-cli/diagnose-record.md) - [file-reader](https://docs.warpstream.com/warpstream/reference/cli-reference/warpstream-cli/file-reader.md) - [file-scrubber](https://docs.warpstream.com/warpstream/reference/cli-reference/warpstream-cli/file-scrubber.md) - [warpstream kcmd (a.k.a. warpstream cli-old)](https://docs.warpstream.com/warpstream/reference/cli-reference/warpstream-kcmd.md) - [warpstream local](https://docs.warpstream.com/warpstream/reference/cli-reference/warpstream-local.md) - [warpstream playground](https://docs.warpstream.com/warpstream/reference/cli-reference/warpstream-playground.md) - [Integrations](https://docs.warpstream.com/warpstream/reference/integrations.md) - [Arroyo](https://docs.warpstream.com/warpstream/reference/integrations/arroyo.md): This page describes how to integrate WarpStream with Arroyo, a distributed stream processing engine written in Rust, that is designed to efficiently perform statement computations on streams of data. - [AWS Lambda Triggers](https://docs.warpstream.com/warpstream/reference/integrations/aws-lambda-triggers.md): This page explains how to use WarpStream with AWS's Self Managed Apache Kafka Trigger - [ClickHouse](https://docs.warpstream.com/warpstream/reference/integrations/clickhouse.md): This page describes how to integrate WarpStream with ClickHouse, ingest data from WarpStream into ClickHouse, and query the data in ClickHouse. - [Debezium](https://docs.warpstream.com/warpstream/reference/integrations/use-warpstream-with-debezium.md): Instructions on how to use WarpStream with Debezium. - [Decodable](https://docs.warpstream.com/warpstream/reference/integrations/decodable.md): This page describes how to integrate WarpStream with Decodable, ingest data into Decodable from WarpStream, and query the data in Decodable. - [DeltaStream](https://docs.warpstream.com/warpstream/reference/integrations/deltastream.md): Learn how to connect DeltaStream and WarpStream. - [docker-compose](https://docs.warpstream.com/warpstream/reference/integrations/use-the-agent-in-docker-compose.md): This page describes how to configure the WarpStream Agent's so they run in docker-compose. - [DuckDB](https://docs.warpstream.com/warpstream/reference/integrations/duckdb.md): DuckDB is an open-source, column-oriented, relational database management system (RDBMS) designed for analytical processing and interactive querying. - [ElastiFlow](https://docs.warpstream.com/warpstream/reference/integrations/elastiflow.md): ElastiFlow is a network flow and SNMP analytics/monitoring solution that enables insights into network performance and security using NetFlow, IPFIX, sFlow, and SNMP with data platforms like Kafka. - [Estuary](https://docs.warpstream.com/warpstream/reference/integrations/estuary.md): Estuary allows you to build real-time ETL/ELT data pipelines between various platforms supported by an array of connectors. - [Fly.io](https://docs.warpstream.com/warpstream/reference/integrations/deploy-warpstream-to-fly.io.md): Instructions on how to deploy the WarpStream Agents to Fly.io - [Imply](https://docs.warpstream.com/warpstream/reference/integrations/imply.md): This page describes how to integrate, ingest and query data in Imply from WarpStream. Imply is powered by Apache Druid, a real-time analytics database. - [InfluxDB](https://docs.warpstream.com/warpstream/reference/integrations/influxdb.md): InfluxDB is an open-source time series database that is a perfect companion to WarpStream's Apache Kafka-compatible clusters. - [Kafbat UI/kafka-ui](https://docs.warpstream.com/warpstream/reference/integrations/kafbat-ui-kafka-ui.md): Kafbat UI and kafka-ui are web UIs for managing and monitoring Kafka clusters. - [Kestra](https://docs.warpstream.com/warpstream/reference/integrations/kestra.md): This page describes how to integrate WarpStream with Kestra. Kestra is an event-driven data orchestration platform with a UI and command-line interface. - [Materialize](https://docs.warpstream.com/warpstream/reference/integrations/materialize.md): This page describes how to set up a connection between WarpStream and Materialize, ingest data into Materialize, and create a materialized view of this data. - [MinIO](https://docs.warpstream.com/warpstream/reference/integrations/minio.md): Instructions on integrating WarpStream with MinIO. - [MirrorMaker](https://docs.warpstream.com/warpstream/reference/integrations/mirrormaker.md): This page describes the settings tweaks you may have to do to leverage MirrorMaker to migrate to Warpstream. - [MotherDuck](https://docs.warpstream.com/warpstream/reference/integrations/motherduck.md): MotherDuck provides cloud-based, serverless access to DuckDB - [ngrok](https://docs.warpstream.com/warpstream/reference/integrations/ngrok.md): How to leverage ngrok for testing WarpStream Agents running locally. - [Ockam](https://docs.warpstream.com/warpstream/reference/integrations/ockam.md): Instructions on integrating WarpStream with Ockam. - [OpenTelemetry Collector](https://docs.warpstream.com/warpstream/reference/integrations/opentelemetry-collector.md): This page describes how to connect the OpenTelemetry Collector to WarpStream using the Kafka exporter. - [Parquet](https://docs.warpstream.com/warpstream/reference/integrations/sqlite.md): Apache Parquet is an open-source, column-oriented data file format designed for efficient data storage and retrieval. It forms the backbone of many datalake and table format systems. - [Quix Streams](https://docs.warpstream.com/warpstream/reference/integrations/quix-streams.md): This page describes how to use the Quix Streams Python library to read and aggregate data from WarpStream and ingest the aggregations into a local DuckDB database for offline querying. - [Railway](https://docs.warpstream.com/warpstream/reference/integrations/railway.md): Instructions on how to deploy the WarpStream Agents to Railway. - [Redpanda Console](https://docs.warpstream.com/warpstream/reference/integrations/redpanda-console.md): Redpanda Console is a web application that helps you manage and debug your Kafka workloads, as well as Kafka protocol-compatible systems such as Redpanda and WarpStream. - [RisingWave](https://docs.warpstream.com/warpstream/reference/integrations/use-warpstream-with-risingwave.md): Instructions on how to use WarpStream with RisingWave. - [Rockset](https://docs.warpstream.com/warpstream/reference/integrations/rockset.md): This page describes how to integrate WarpStream with Rockset, ingest data into Rockset from WarpStream, and query the data in Rockset. - [ShadowTraffic](https://docs.warpstream.com/warpstream/reference/integrations/shadowtraffic.md): ShadowTraffic is a containerized service for declaratively generating data, packed with knobs to perfectly mimic your production traffic to Kafka-compatible, and other destinations. - [SQLite](https://docs.warpstream.com/warpstream/reference/integrations/sqlite-1.md): SQLite is an open-source, embedded, serverless RDBMS that is popular for its small size and ease of use. - [Streambased](https://docs.warpstream.com/warpstream/reference/integrations/streambased.md): Instructions on integrating WarpStream with Streambased. - [Streamlit](https://docs.warpstream.com/warpstream/reference/integrations/streamlit.md): Streamlit is a Python library that enables the simple creation of web apps. - [Timeplus](https://docs.warpstream.com/warpstream/reference/integrations/timeplus.md): This page describes how to integrate WarpStream with Timeplus, to perform SQL commands on a data stream to perform queries, transformations and ETL. - [Tinybird](https://docs.warpstream.com/warpstream/reference/integrations/tinybird.md): This page describes how to integrate WarpStream with Tinybird, ingest data into Tinybird from WarpStream, and then create an API endpoint for your applications to access the result set. - [Upsolver](https://docs.warpstream.com/warpstream/reference/integrations/upsolver.md): This page describes how to integrate WarpStream with Upsolver, ingest data into Upsolver from WarpStream, then process and write the data to one of the available targets in Upsolver. - [Dedicated Control Plane Cells and Release Windows](https://docs.warpstream.com/warpstream/reference/dedicated-control-plane-cells-and-release-windows.md): This page explains how WarpStream's dedicated control plane cells work, along with support for custom release windows. - [Control Plane Private Connectivity](https://docs.warpstream.com/warpstream/reference/control-plane-private-connectivity.md) - [Deprecation Policy](https://docs.warpstream.com/warpstream/reference/deprecation-policy.md): This page describes our policy in the event of removal of customer-usable functionality from WarpStream services. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on a page URL with the `ask` query parameter: ``` GET https://docs.warpstream.com/warpstream/overview/readme.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.