Confluent Platform Components

Connect eligible self-managed Confluent Platform components to WarpStream.

Some self-managed Confluent Platform components use broker detection to verify that they are connected to Confluent Cloud. WarpStream Agents can publish the broker information these components expect, allowing eligible components to connect to a WarpStream cluster.

This setting does not provide a Confluent license or change its terms. You must have a valid license that permits the component to connect to your WarpStream deployment. Contact your Confluent account team if you are unsure whether your license applies.

Supported deployments

This feature is available for WarpStream BYOC deployments running Agent v724 or later.

This compatibility mode is intended for eligible self-managed Confluent Platform components. It does not establish compatibility with every Confluent Platform component, make a WarpStream cluster available in the Confluent Cloud Console, or provision fully-managed Confluent Cloud connectors.

Before you begin

You need:

• A WarpStream BYOC deployment running Agent v724 or later.
• A valid license for the Confluent Platform component.
• The bootstrap URL and credentials for your WarpStream virtual cluster. You can copy them from the Connect tab in the WarpStream Console.
• Network connectivity from the component to the WarpStream Agents.

Enable Confluent component compatibility

Set the following environment variable on every WarpStream Agent in the deployment:

WARPSTREAM_ENABLE_CONFLUENT_COMPONENTS=true

Alternatively, pass the corresponding Agent flag:

-enableConfluentComponents

Restart the Agents after changing their configuration. A rolling restart is sufficient.

When enabled, the Agents create and populate the internal broker-information topic used by eligible Confluent Platform components. Topic creation is idempotent, so keep the setting enabled on every Agent.

This does not change your bootstrap URL. Continue using the WarpStream bootstrap URL; a *.confluent.cloud hostname is not required.

Configure the component

Configure the component to use the same bootstrap URL, authentication mechanism, and credentials as any other Kafka client connecting to the virtual cluster. For a typical SASL/PLAIN connection over TLS, use:

bootstrap.servers=<warpstream-bootstrap-url>
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
  username="<warpstream-credential-username>" \
  password="<warpstream-credential-secret>";

Add the Confluent license using the property required by the component, commonly:

confluent.license=<confluent-license-key>

Follow the component's documentation for any additional properties. In particular, a commercial Kafka connector may require a separate connection for its license topic:

confluent.topic.bootstrap.servers=<warpstream-bootstrap-url>
confluent.topic.security.protocol=SASL_SSL
confluent.topic.sasl.mechanism=PLAIN
confluent.topic.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
  username="<warpstream-credential-username>" \
  password="<warpstream-credential-secret>";

Connectors that use Reporter or a separate history topic also require those producers and consumers to use the WarpStream bootstrap URL and credentials. See Confluent's self-managed Kafka Connect configuration for the component-specific property prefixes.

If you use a different authentication method, replace the SASL/PLAIN properties with the settings described in Client Configuration.

Configure ACLs

When ACL enforcement is enabled, grant the component's principal the permissions required by the component. These commonly include:

• Read and write access to the application's source and destination topics.
• Read, write, create, and describe access to Kafka Connect's configuration, offset, and status topics.
• Read, write, create, and describe access to _confluent-command or another configured Confluent license topic.
• Read and describe access to __internal_confluent_only_broker_info.
• Access to Reporter, dead-letter queue, and schema-history topics when the connector uses them.

For instructions, see ACLs.

Verify the connection

Start the component and check its license or status endpoint. The component should report a valid license and should not reject the cluster as an unsupported broker type.

If the component does not start:

1. Confirm that every Kafka Agent is running v724 or later with WARPSTREAM_ENABLE_CONFLUENT_COMPONENTS=true.
2. Confirm that the component can reach the WarpStream bootstrap URL.
3. Test the credentials with one of the client examples from the virtual cluster's Connect tab.
4. If ACL enforcement is enabled, check the component logs for authorization failures on its internal topics.
5. Confirm that the license is valid for both the component and the deployment.

For eligible licenses and minimum component versions, see Manage Confluent Platform licenses.