# ClickHouse

A video walkthrough can be found below:

{% embed url="<https://youtu.be/fLpwJjwZ8oA>" %}

## Prerequisites

1. WarpStream account - get access to WarpStream by registering [here](https://console.warpstream.com/signup).
2. ClickHouse account - get access to ClickHouse by registering [here](https://www.decodable.co/).
3. A WarpStream cluster is up and running.

## Step 1: Create a topic in your WarpStream cluster

Obtain the Bootstrap Broker from the WarpStream console by navigating to your cluster and clicking the Connect tab. If you don't have SASL credentials, you can also [create a set of credentials](https://docs.warpstream.com/warpstream/kafka/manage-security/sasl-authentication#creating-credentials) from the console.

Store these values as environment variables for easy reference:

```bash
export BOOTSTRAP_HOST=<YOUR_BOOTSTRAP_BROKER> \
SASL_USERNAME=<YOUR_SASL_USERNAME> \
SASL_PASSWORD=<YOUR_SASL_PASSWORD>;
```

Then, if you don't already have an available topic, create one using the WarpStream CLI or in the UI, then follow **Step 2**:

{% code overflow="wrap" %}

```bash
warpstream kcmd -bootstrap-host $BOOTSTRAP_HOST -tls -username $SASL_USERNAME -password $SASL_PASSWORD -type create-topic -topic clickhouse_demo
```

{% endcode %}

You should see the following output in your Terminal:

`Created topic clickhouse_demo.`

## Step 2: Produce some records

Using the WarpStream CLI, produce several messages to your topic:

{% code overflow="wrap" %}

```bash
warpstream kcmd -bootstrap-host $BOOTSTRAP_HOST -tls -username $SASL_USERNAME -password $SASL_PASSWORD -type produce -topic clickhouse_demo --records '{"action": "click", "user_id": "user_0", "page_id": "home"},,{"action": "hover", "user_id": "user_0", "page_id": "home"},,{"action": "scroll", "user_id": "user_0", "page_id": "home"},,{"action": "click", "user_id": "user_1", "page_id": "home"},,{"action": "click", "user_id": "user_1", "page_id": "home"},,{"action": "click", "user_id": "user_2", "page_id": "home"}'
```

{% endcode %}

Note that the WarpStream CLI uses double commas (`,,)` as a delimiter between JSON records.

## Step 3: Connect ClickHouse to WarpStream

In the ClickHouse dashboard, navigate to "**Data sources**" and then click "**Get started**":

<figure><img src="https://77315434-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FjB7FxO8ty4EXO4HsQP4E%2Fuploads%2Fgit-blob-6b73933b23709f1348f2649b75dc1a82ed7ece21%2Fimage.png?alt=media" alt=""><figcaption><p>ClickHouse Data sources</p></figcaption></figure>

Select **WarpStream** from the list of options:

<figure><img src="https://77315434-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FjB7FxO8ty4EXO4HsQP4E%2Fuploads%2Fgit-blob-678c351d2e54e7a1d388b0b0de3cd90bcf1cb702%2Fimage.png?alt=media" alt=""><figcaption></figcaption></figure>

Fill in the fields using the data you generated in **Step 1**:

<figure><img src="https://77315434-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FjB7FxO8ty4EXO4HsQP4E%2Fuploads%2Fgit-blob-da303363c8d7582dca502a5c867044d1ab41c22b%2Fimage.png?alt=media" alt=""><figcaption></figcaption></figure>

Next, select the incoming data format (JSON or AVRO) and then the topic. For this example, we use JSON format, and we have topics '**products**' and '**reviews**'. We'll use '**products**', and ClickHouse will display a record from the Producer to confirm.

<figure><img src="https://77315434-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FjB7FxO8ty4EXO4HsQP4E%2Fuploads%2Fgit-blob-91d2af0f744e4babda1f1aba66571faa6acbb2ec%2Fimage.png?alt=media" alt=""><figcaption></figcaption></figure>

The data is then parsed, and you will then have the opportunity to make additional changes to the table that will be produced:

<figure><img src="https://77315434-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FjB7FxO8ty4EXO4HsQP4E%2Fuploads%2Fgit-blob-b7f926d3b986c2813cdd28693097342bbb29ae14%2Fimage.png?alt=media" alt=""><figcaption></figcaption></figure>

## Step 4: Query your data in ClickHouse

We now see our **WarpStream** pipe ingesting data into ClickHouse. Next, select '**SQL Console**' in the left-hand navigation to interact with your new ClickHouse table.

<figure><img src="https://77315434-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FjB7FxO8ty4EXO4HsQP4E%2Fuploads%2Fgit-blob-75be2246736e99929686da7e0943973ec00d80d1%2Fimage.png?alt=media" alt=""><figcaption></figcaption></figure>

We can now take full advantage of ClickHouse's power with WarpStream data. A simple SQL SELECT command is illustrated below:

<figure><img src="https://77315434-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FjB7FxO8ty4EXO4HsQP4E%2Fuploads%2Fgit-blob-cbb20544a0b647fdf5f9335feed79eedc99faf95%2Fimage.png?alt=media" alt=""><figcaption></figcaption></figure>

## Next Steps

Congratulations! You've set up a stream processing pipeline between WarpStream and ClickHouse and performed a basic SQL query. This is just the beginning of what you can do.

Next, check out the WarpStream docs for configuring the [WarpStream Agent](https://docs.warpstream.com/warpstream/agent-setup/deploy), or review the [ClickHouse docs](https://clickhouse.com/docs/en/integrations/clickpipes) to learn more about what is possible with WarpStream and ClickHouse!