Protect Data in Motion with TLS Encryption

TLS Encryption Overview

By default Kafka/Schema Registry clients communicating with WarpStream Agents use PLAINTEXT, meaning that all data is sent in plain text (unencrypted). To encrypt data in motion (or data in transit) between your clients and your WarpStream Agents, you should configure them to use TLS Encryption.

WarpStream supports Transport Layer Security (TLS) encryption based on OpenSSL, an open source cryptography toolkit that provides an implementation of the Transport Layer Security (TLS).

Enabling TLS encryption might have a performance impact due to overhead of encrypting and decrypting data. This performance impact can vary depending on the operating system, linux kernel version, and CPU used. We recommend using the newest and best possible versions of your Operating system, Kernel, and CPU to minimize any possible impacts.

TLS uses private-key/certificate pairs, which are used during the TLS handshake process.

Each WarpStream Agent needs a private-key/certificate pair, and the Kafka client uses the certificate to authenticate to the WarpStream Agent.
Each logical client needs a private-key/certificate pair if client authentication is enabled, and the WarpStream Agent uses the certificate to authenticate the Kafka client.

Mutual (mTLS) Authentication

If you configure TLS encryption, you can optionally configure mutual (mTLS) authentication. You can configure just TLS encryption (by default, TLS encryption includes certificate authentication of the server) and use a separate mechanism for client authentication (for example, mTLS or SASL). By default, TLS encryption enables one-way authentication in which the client authenticates the server certificate. For bidirectional authentication, where the broker also authenticates the client certificate, you can use mTLS.

When you use mTLS Authentication, the WarpStream Agent authenticates the Kafka client and the Kafka client also authenticates the WarpStream Agent. This bidirectional, or mutual, authentication provides an additional layer of security for your WarpStream cluster.

Configure TLS Encryption for a WarpStream Cluster

Configuring TLS for a WarpStream Cluster can be done in one of two ways. Which option is the best depends on your requirements and deployment environment.

Option 1: Configuring the WarpStream Agents to terminate TLS

This option is the recommended configuration when using TLS. This ensures that there is full end-to-end Kafka client to WarpStream Agent encryption. This comes at the disadvantage of needing to configure each WarpStream Agent with a TLS certificate.

Create x509 encoded TLS certificates
- Every organization has different policies on how to create and manage certificate. We recommend talking with your IT team for how to best create certificates in your organization.
- If running WarpStream in Kubernetes using cert-manager can be the easiest way to create certificates. It supports a wide range of certificate providers including private certificate authorities.
- We recommend that either the certificate is made with SANS and using hostnames to connect to your WarpStream Agents. Alternatively you can use IP SANS however that may be impractical in some deployments like Kubernetes.
Configure the WarpStream Agents to load the certificates
- Once your certificates are generated to must set the WARPSTREAM_TLS_SERVER_CERT_FILE environment variable to the public key of the certificate and set WARPSTREAM_TLS_SERVER_PRIVATE_KEY_FILE to the private key of the certificate.
Configure the WarpStream Agents to enable TLS encryption
- To enable TLS encryption for your Kafka agent, set the -kafkaTLSagent flag. Alternatively, you can set the WARPSTREAM_TLS_ENABLEDenvironment variable to true.
- To enable TLS encryption for your Schema Registry agent, set the -schemaRegistryTLSagent flag. Alternatively, you can set the WARPSTREAM_SCHEMA_REGISTRY_TLS_ENABLEDenvironment variable to true.

By default WarpStream Agents will load the TLS certificate and key from the agent's local filesystem. The agents can be configured to instead load the TLS certificate and key from blob storage by using the -tlsBlobURL flag or WARPSTREAM_TLS_BLOB_URL environment variable. For example -tlsBlobURL=s3://my-tls-bucket.

WarpStream uses modern x509 Certificate parsing algorithms which requires certificates to have both the Common Name and Subject Alternative Names (SAN) list to be set. The Common Name must also be in the SAN list.

TLS Certificate Configuration with the Helm Chart

When configuring certificates using our Helm Chart it is recommended to use the following values

certificate:
  # Set to true to enable TLS termination on the WarpStream agent
  # see TLS documentation for details https://docs.warpstream.com/warpstream/byoc/advanced-agent-deployment-options/protect-data-in-motion-with-tls-encryption#tls-encryption-overview
  enableTLS: true

  # The Kubernetes TLS secret that contains a certificate and private key
  # see https://kubernetes.io/docs/concepts/configuration/secret/#tls-secrets
  secretName: "warpstream-agents-cert"

deploymentKind: StatefulSet

The certificate should have the following SANS.

'warpstream-agent.default.svc.cluster.local'
'*.warpstream-agent-headless.default.svc.cluster.local'

Replace default with the namespace that the helm chart is installed into. If you are setting nameOverride or fullNameOverride fields then warpstream-agent may also need to be modified.

For example if you are using cert-manager to generate the certificates, the certificate resource should look like the following.

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: warpstream-agents-cert
  namespace: default
spec:
  dnsNames:
    - 'warpstream-agent.default.svc.cluster.local'
    - '*.warpstream-agent-headless.default.svc.cluster.local'
  secretName: warpstream-agents-cert
  issuerRef:
    name: my-ca-issuer
    kind: Issuer
    group: cert-manager.io

If you are deploying WarpStream with a load balancer infront, i.e WarpStream behind a TCP Load Balancer Without Direct Connectivity then deploymentKind does not need to be set to StatefulSet and the certificate SANS should only contain the hostname of the load balancer.

Option 2: Configuring a Load Balancer to terminate TLS

This option can be simpler to implement for WarpStream clusters behind a load balancer. This option requires that you Advertise the WarpStream Agents behind a traditional load balancer for it to work as expected. The disadvantage of this option is that the communication between the Load Balancer and WarpStream Agents are not encrypted. If you have a requirement for full end-to-end encryption we do not recommend using this option. This option also cannot use mTLS as an authentication mechanism due to TLS termination on Load Balancers not being able to pass-through the client certificate.

Every Load Balancer is configured differently for TLS termination, bellow is information from the 3 major Cloud Providers for how to configure TLS termination on their Load Balancers.

AWS
- EKS NLB
- Network Load Balancer
GCP
- GKE - As of 2024 GKE does not support TLS termination on Kubernetes Services with Load Balancer types
- Network Load Balancer SSL Proxy
Azure
- As of 2024 Azure Layer 4 load balancers do not support TLS termination

Configure TLS Encryption for Kafka clients

For configuring TLS encryption in your Kafka clients it is recommended to review the documentation for your Kafka client. Every Kafka client configures TLS differently and those configurations may change version to version.

We recommend using the Confluent Platform documentation to learn how to configure Java-based clients for TLS encryption.

TLS Profiles

By default the WarpStream Agent uses the default Golang TLS settings when serving the Kafka Protocol over TLS. These defaults change over time but and may not be suitable for all environments.

The WarpStream agent supports setting different TLS Profiles based on recommendations from Mozilla SSL.

The ability to configure TLS Profiles was added in Agent Version v657.

These profiles can be set via the -tlsProfile flag or WARPSTREAM_TLS_PROFILE environment variable.

All profiles use the following TLS Curves

tls.X25519
tls.CurveP256
tls.CurveP384
tls.CurveP521
tls.X25519MLKEM768

All TLS 1.3 connections use the following Cipher Suites:

tls.TLS_AES_128_GCM_SHA256
tls.TLS_AES_256_GCM_SHA384
tls.TLS_CHACHA20_POLY1305_SHA256

`golang-default`

Use the Golang defaults for TLS Version, Curves and Cipher Suites. These will change over time. See Golang TLS documentation for details.

`old`

This profile supports older TLS clients with the minimum TLS version set to 1.0 with support for the following Cipher Suites for TLS 1.0, 1.1 and 1.2:

tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
tls.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
tls.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
tls.TLS_RSA_WITH_AES_128_GCM_SHA256
tls.TLS_RSA_WITH_AES_256_GCM_SHA384
tls.TLS_RSA_WITH_AES_128_CBC_SHA256
tls.TLS_RSA_WITH_AES_128_CBC_SHA
tls.TLS_RSA_WITH_AES_256_CBC_SHA
tls.TLS_RSA_WITH_3DES_EDE_CBC_SHA

intermediate

This profile supports intermediate TLS clients with the minimum TLS version set to 1.2 with support for the following Cipher Suites:

tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305

modern

This profile supports modern TLS clients and TLS 1.3 only.

Transparent HTTPS Proxies

Some organizations use transparent HTTPS Proxies to decrypt and inspect HTTPS traffic. This may cause the WarpStream Agent to be unable to communicate with the WarpStream Control Plane.

You may see the following error if that is the case

Post "https://metadata.default.${region}$.${cloud}$.warpstream.com/api/v1/agent/agentpool": tls: failed to verify certificate: x509: certificate signed by unknown authority

Starting in version v657 the WarpStream agent as the ability to provide an HTTPS Proxy Certificate Authority for the Agent to trust. This can be done with the -httpsProxyCACertFile flag or WARPSTREAM_HTTPS_PROXY_CA_CERT_FILE environment variable.

An example values.yaml of this configuration via our Kubernetes Helm Chart is as follows:

extraEnv:
  - name: WARPSTREAM_HTTPS_PROXY_CA_CERT_FILE
    # The path to where the configmap is mounted.
    # In this example the item inside the configmap is
    # called `ca.crt` so it get's created as that file.
    value: /etc/https-proxy/ca.crt

volumeMounts:
  - name: https-proxy-ca
    # The path to where the confimap should be mounted.
    mountPath: /etc/https-proxy

volumes:
  - name: https-proxy-ca
    configMap:
      # Name of the configmap in Kubenretes that contains 
      # the x509 PEM file for the HTTPS Proxy.
      name: https-proxy-ca

PreviousAgent Groups NextLow Latency Clusters

Last updated 1 month ago

Was this helpful?