LogoLogo
WarpStream.comSlackDiscordContact UsCreate Account
  • Overview
    • Introduction
    • Architecture
      • Service Discovery
      • Write Path
      • Read Path
      • Life of a Request (Simplified)
    • Change Log
  • Getting Started
    • Install the WarpStream Agent / CLI
    • Run the Demo
    • "Hello World" for Apache Kafka
  • BYOC
    • Run the Agents Locally
    • Deploy the Agents
      • Object Storage Configuration
      • Kubernetes Known Issues
      • Rolling Restarts and Upgrades
    • Client Configuration
      • Tuning for Performance
      • Configure Clients to Eliminate AZ Networking Costs
        • Force Interzone Load Balancing
      • Configuring Kafka Client ID Features
      • Known Issues
    • Infrastructure as Code
      • Terraform Provider
      • Helm charts
      • Terraform Modules
    • Monitoring
      • Pre-made Datadog Dashboard
      • Pre-made Grafana Dashboard
      • Important Metrics and Logs
      • Recommended List of Alerts
      • Monitoring Consumer Groups
      • Hosted Prometheus Endpoint
    • Authentication
      • SASL Authentication
      • Mutual TLS (mTLS)
      • Basic Authentication
    • Advanced Agent Deployment Options
      • Agent Roles
      • Agent Groups
      • Protect Data in Motion with TLS Encryption
      • Low Latency Clusters
      • Network Architecture Considerations
      • Agent Configuration Reference
      • Reducing Infrastructure Costs
      • Client Configuration Auto-tuning
    • Hosted Metadata Endpoint
    • Managed Data Pipelines
      • Cookbooks
    • Schema Registry
      • WarpStream BYOC Schema Registry
      • Schema Validation
      • WarpStream Schema Linking
    • Orbit
    • Port Forwarding (K8s)
  • Reference
    • ACLs
    • Billing
      • Direct billing
      • AWS Marketplace
    • Benchmarking
    • Compression
    • Protocol and Feature Support
      • Kafka vs WarpStream Configuration Reference
      • Compacted topics
    • Secrets Overview
    • Security and Privacy Considerations
    • API Reference
      • API Keys
        • Create
        • Delete
        • List
      • Virtual Clusters
        • Create
        • Delete
        • Describe
        • List
        • DescribeConfiguration
        • UpdateConfiguration
      • Virtual Clusters Credentials
        • Create
        • Delete
        • List
      • Monitoring
        • Describe All Consumer Groups
      • Pipelines
        • List Pipelines
        • Create Pipeline
        • Delete Pipeline
        • Describe Pipeline
        • Create Pipeline Configuration
        • Change Pipeline State
      • Invoices
        • Get Pending Invoice
        • Get Past Invoice
    • CLI Reference
      • warpstream agent
      • warpstream demo
      • warpstream cli
      • warpstream cli-beta
        • benchmark-consumer
        • benchmark-producer
        • console-consumer
        • console-producer
        • consumer-group-lag
        • diagnose-record
        • file-reader
        • file-scrubber
      • warpstream playground
    • Integrations
      • Arroyo
      • AWS Lambda Triggers
      • ClickHouse
      • Debezium
      • Decodable
      • DeltaStream
      • docker-compose
      • DuckDB
      • ElastiFlow
      • Estuary
      • Fly.io
      • Imply
      • InfluxDB
      • Kestra
      • Materialize
      • MinIO
      • MirrorMaker
      • MotherDuck
      • Ockam
      • OpenTelemetry Collector
      • ParadeDB
      • Parquet
      • Quix Streams
      • Railway
      • Redpanda Console
      • RisingWave
      • Rockset
      • ShadowTraffic
      • SQLite
      • Streambased
      • Streamlit
      • Timeplus
      • Tinybird
      • Upsolver
    • Partitions Auto-Scaler (beta)
    • Serverless Clusters
    • Enable SAML Single Sign-on (SSO)
    • Trusted Domains
    • Diagnostics
      • GoMaxProcs
      • Small Files
Powered by GitBook
On this page
  • Prerequisites
  • Important considerations
  • ACL Principal
  • Super users
  • Caching of ACLs
  • allow.everyone.if.no.acl.found
  • Enabling and Disabling ACLs
  • To Enable ACLs
  • To Disable ACLs
  • Enable/Disbable Using the HTTP API
  • Generating Credentials
  • Managing ACLs via Kafka API
  • Create an ACL
  • Delete an ACL
  • Denied Authorization Logs

Was this helpful?

  1. Reference

ACLs

The following page provides a management guide for ACLs (Access Control Lists). This document provides instructions on how to enable, disable, create, and delete ACLs for your WarpStream clusters.

PreviousPort Forwarding (K8s)NextBilling

Last updated 1 month ago

Was this helpful?

Prerequisites

Before you begin, ensure you have administrative access to the WarpStream console and are familiar with the basic concepts of ACLs.

The current version of ACLs supports SASL and mTLS authentication method. Review for more information.

Important: ACLs are only compatible with Agent version 526 and above. Make sure you use a compatible agent version, if your cluster has self-hosted agents.

Important considerations

There are a few things to keep in mind when using ACLs, which are covered below.

ACL Principal

ACLs operate using Principals, which are entities capable of being authenticated by the Authorizer. In the context of WarpStream agents, clients authenticate as a specific principal by using either SASL or mTLS authentication.

With SASL, the principal is identified by one of the three following:

  1. The assigned username (prefixed with ccun_)

  2. Chosen name during , (prefixed with ccn_)

  3. Chosen name during , without prefix ccn_

In the screenshot below, valid principal examples include: user_1234, ccn_user_1234, and ccun_3cc6c05a56b396fc084ef1e113150970cd148c87b5f64babfd3b8cf67d4d8840.

Super users

For SASL, you can authorize as super user by setting your SASL username as either the generated cluster credentials username(ccun_) or by using the "Name" which you set while generating credentials.

  • If you set a tlsPrincipalMappingRule in the Agents, then you need to make sure that the string extracted from the DN which will be used as the ACL principal matches either the "Name" you picked when generating credentials or matches the generated cluster credentials username(ccun_).

  • If you don't set a tlsPrincipalMappingRule in the Agents and if you set the "Name" when generating credentials as the DN of your certificate, then the certificate will authorize as superuser.

Caching of ACLs

For performance reasons, ACLs are cached, so changes to ACLs may take between 30 seconds to 1 minute to take effect. Plan accordingly when updating ACLs in a production environment to ensure a smooth transition.

allow.everyone.if.no.acl.found

Warpstream does not implement the allow.everyone.if.no.acl.found Kafka configuration, because:

  1. It's not safe for production environments.

  2. You can get the equivalent (and safer) behaviour by simply using Super users, who are authorized to access everything.

Enabling and Disabling ACLs

Warning

Before enabling ACLs, be aware that improper configurations may lead to denied access for producers and consumers. Make sure to configure ACLs correctly to avoid any disruption of service.

To Enable ACLs

  1. Select the desired Virtual Cluster, for example vcn_default.

  2. Click on the ACLs tab.

  3. You will see the ACLs: Disabled status label if ACLs are currently disabled.

  4. Click on the Enable ACLs button to change its status to Enabled.

To Disable ACLs

  1. Follow the same steps to navigate to the ACLs tab of your Virtual Cluster.

  2. If ACLs are enabled, you will see the ACLs: Enabled status label.

  3. Click on the Disable ACLs button to turn off ACLs for the cluster.

Enable/Disbable Using the HTTP API

Alternatively, you can switch ACLs on and off using the HTTP APIs, which is a convenient alternative to doing it through the console UI. Refer toDescribeConfiguration and UpdateConfiguration for details.

Generating Credentials

To interact with WarpsSream clusters, you need to generate credentials:

  1. Within the WarpStream console, navigate to the Credentials tab of your Virtual Cluster.

  2. Click on Generate Credentials.

  3. Provide a name for the credentials.

  4. If necessary, enable Super User for broader permissions.

    • Important: If a resource has no associated ACLs, then only superusers can access that resource.

  5. Click Generate Credentials. The credentials will be displayed for you to use with the CLI.

Important: Credentials are shown only once. Store them securely.

Managing ACLs via Kafka API

To manage ACLs via the Kafka API, use the Kafka Admin API with appropriate Kafka credentials.

Create an ACL

To create an ACL, use the createAcls method with the required ACL details:

_, err = admin.CreateACLs(
	context.Background(),
	kafka.ACLBindings{{
		Type:                kafka.ResourceTopic,
		Name:                "topic_name",
		ResourcePatternType: kafka.ResourcePatternTypeLiteral,
		Principal:           "User:<credential_username>",
		Host:                "*", // Request can come from any host.
		Operation:           kafka.ACLOperationWrite,
		PermissionType:      kafka.ACLPermissionTypeAllow,
	}},
)
if err != nil {
	fmt.Printf("Failed to create ACLs: %s\n", err.Error())
	os.Exit(1)
}

Delete an ACL

To delete an ACL, use the deleteAcls method with a filter matching the ACL you wish to remove:

_, err = admin.DeleteACLs(
	context.Background(),
	kafka.ACLBindingFilters{{
		Type:                kafka.ResourceTopic,
		Name:                "topic_name",
		ResourcePatternType: kafka.ResourcePatternTypeLiteral,
		Principal:           "User:<credential_username>",
		Host:                "*",
		Operation:           kafka.ACLOperationWrite,
		PermissionType:      kafka.ACLPermissionTypeAllow,
	}},
)
if err != nil {
	fmt.Printf("Failed to delete ACLs: %s\n", err)
	os.Exit(1)
}

The examples above use the librdkafka package from Go. You can also try the bin/kafka-acls.sh script from Apache Kafka®'s official binary release for more hands-on experience:

bin/kafka-acls.sh --bootstrap-server localhost:9092 --remove --allow-principal User:<credential_username> --allow-principal User:Alice --allow-host 198.51.100.0 --allow-host 198.51.100.1 --operation Read --operation Write --topic Test-topic
bin/kafka-acls.sh --bootstrap-server localhost:9092 --add --allow-principal User:Bob --allow-principal User:<credential_username> --allow-host 198.51.100.0 --allow-host 198.51.100.1 --operation Read --operation Write --topic Test-topic

Denied Authorization Logs

Logs for denied authorizations can be enabled by setting the -enableACLLogs flag or WARPSTREAM_ENABLE_ACL_LOGS environment variable to true on the Agents.

Any actions that are denied due to a missing ACL or an explicit DENY ACL will show up in these logs. An example log is below:

{"time":"2025-03-13T13:02:05.176353-05:00","level":"WARN","msg":"ACL Denied Operation on resource","principal":"User:ccun_9c5d...","operation":"CREATE","host":"127.0.0.1:56216","resource_type":"TOPIC","resource_name":"foo"}

In the example above we can see the user User:ccun_9c5d... tried to CREATE a TOPIC called foo and was denied.

With mTLS, the principal is identified from the Distinguished Name(DN) from the client TLS certificate. To use a custom principal, view the page. Importantly, note that mTLS ACL principals don't have to be identified using a valid cluster credential username(ccun_...) or cluster credential name like the SASL ACL principals.

In the context of ACLs, a 'superuser' is a specially designated user who possesses elevated privileges. Superusers are granted overarching access rights, allowing them to bypass standard ACL restrictions. This includes unrestricted abilities to create, modify, or delete resources, and to manage access controls for other users. In Warpstream you can create super users through the credentials, as explained at

For mTLS, the ACL principal is either the Distinguished Name(DN) from the client TLS certificate, or extracted from the DN according to the -tlsPrincipalMappingRule flag set in the Agents as described in the page.

Navigate to the .

mutual TLS authentication
mutual tls authentication
WarpStream console
Authentication
credential creation
credential creation
SASL Generating Credentials.