Streambased

Instructions on integrating WarpStream with Streambased.

The Streambased folks have a great tutorial on their website on integrating WarpStream with command line Streambased. What follows is a description of using their Analytics Service for Kafka {A.S.K.) product, which is free and requires no downloads or setup. A video walkthrough can be found below.

Step 1: Create a topic in your WarpStream cluster

Obtain the Bootstrap Broker from the WarpStream console by navigating to your cluster and clicking the Connect tab. If you don't have SASL credentials yet, you must also create a set of credentials from the console by clicking on the Credentials tab.

Save these three values, as you will need them in the sasl.jaas.config parameter in the next step.

Step 2: Create your SQL connection string

Once you have constructed this connection string, it can be executed in a query or BI tool of your choice to establish the connection to your WarpStream cluster. From that point, you can interact with your stream like any other SQL-compatible data source.

The saasl.jaas.config has a bit of a gotcha in that the single quote has to be escaped. Streambased has provided a tool to format your information to ensure correctness properly.

SET SESSION streambased_connection='{
"bootstrap.servers":"YOUR_BOOTSTRAP_BROKER>",
"security.protocol":"SASL_SSL",
"sasl.jaas.config":"org.apache.kafka.common.security.plain.PlainLoginModule required username=''<YOUR_SASL_USERNAME>'' password=''<YOUR_SASL_PASSWORD>'';",
"sasl.mechanism":"PLAIN",
"schema.registry.url":"<SCHEMA_REGISTRY_ENDPOINT_URL>",
"basic.auth.credentials.source":"USER_INFO",
"basic.auth.user.info":"<SCHEMA_REGISTRY_KEY>:<SCHEMA_REGISTRY_SECRET>"
}';

Next Steps

Using the above configured SET SESSION command, you now have everything you need to connect with the SQL or BI tool of your choice to query your WarpStream producer stream.