Secrets Overview

This page provides an overview of the various secrets defined by WarpStream.

API Keys

API Keys authenticate requests to the WarpStream control plane. WarpStream defines three kinds of API Keys, Application Keys, Agent Keys, and Account Keys. Application keys and Agent keys are workspace-specific and can be found in the WarpStream console in the API Keys sidebar view. Account keys are managed in the Workspaces management view and are only visible to users who have admin access across all workspaces. The Workspaces management view is accessed via the dropdown menu on the top-left of the web console.

Application Keys

Application Keys authenticate requests to WarpStream's public HTTP API. For example they are useful for managing resources inside your account via infrastructure-as-code. Every Application Key belongs to a single workspace and is only authorized to manage resources belonging to that workspace. When a workspace-specific resource is created via the public API, e.g. a new virtual cluster, the new resource is always assigned to the same workspace that the Application Key used to authenticate the request belongs to.

See WarpStream's API reference to learn how to include your Account Key or Application Key in your API requests.

Agent Keys

Agent Keys authenticate an Agent with the WarpStream control plane in order to manage a BYOC virtual cluster. Each Agent Key is associated with a single cluster. An Agent pointing to a virtual cluster that doesn't match its Agent Key will fail with an authorization error.

Agent Keys apply to BYOC virtual clusters only. When a BYOC virtual cluster is created, a matching Agent Key is generated automatically. Serverless clusters bypass the need to deploy an Agent and therefore do not support Agent Keys.

You can view all Agent Keys in your account in the console's API Keys sidebar view. You can view all the Agent Keys for a given BYOC cluster in the cluster's Agent Keys tab.

See WarpStream's documentation on Agent Configuration to learn how to pass a key to your Agent.

Account Keys

Account keys are available only to customers for whom workspaces are enabled. Contact us for details.

Account Keys authenticate API requests to manage resources that are not workspace-specific, i.e. user roles, workspaces, and Account Keys themselves. In addition to these, Account Keys can also manage Application Keys, even though Application Keys are workspace-specific. This exception allows a single provisioning script to manage the full lifecycle of WarpStream resources with an Account Key as its only secret input.

Such a provisioning script would first create a workspace and an Application Key inside that workspace using the Account Key for authentication. It would then use the newly created Application Key to manage resources inside the new workspace. Requests to manage any other workspace-specific resources cannot be authenticated with an Account Key. They must be authenticated with the Application Key instead.

Below is a Terraform snippet that provisions a virtual cluster inside a new workspace. It first uses an Account Key to create a new workspace and an Application Key inside that workspace, then creates a cluster inside the workspace using the Application Key.

terraform {
  required_providers {
    warpstream = {
      source = "warpstreamlabs/warpstream"
    }
  }
}

# 1. Authenticate a WarpStream Terraform provider with an account key.
provider "warpstream" {
  token = "aks_[ACCOUNT_KEY]"
}

# 2. Provision a new workspace.
resource "warpstream_workspace" "workspace_prod" {
  name = "production workspace"
}

# 3. Provision an application key inside that workspace.
resource "warpstream_application_key" "prod_key" {
  name         = "akn_application_key_prod"
  workspace_id = warpstream_workspace.workspace_prod.id
}

# 4. Authenticate a new WarpStream provider using that application key.
provider "warpstream" {
  alias = "prod"
  token = warpstream_application_key.prod_key.key
}

# 5. Provision a cluster in the new workspace by passing the provider from step 4.
resource "warpstream_virtual_cluster" "prod_cluster" {
  provider = warpstream.prod
  name     = "vcn_production"
  type     = "byoc"
  tier     = "dev"
}

Cluster Credentials

Whereas Agent Keys authenticate your Agent with the WarpStream control plane, cluster credentials authenticate your Kafka client with your Agent using SASL.

By default, SASL is disabled for BYOC virtual clusters because the Agents run in your cloud account. However, you can configure your cluster to require SASL authentication by following the WarpStream documentation on SASL Authentication.