Protecting important resources from accidental deletion

Warpstream offers two levels of protection against accidental deletion for two types of important resources:

• Virtual Clusters

• Topics

Soft deletion

The first one in soft deletion. By default, when you delete a Virtual Cluster or a Topic, Warpstream does not delete the associated data. It keeps the data around and allows you to restore the Virtual Cluster or Topic in the UI

Restoring a Virtual Cluster

When you delete a virtual cluster, either through the API or through the UI, it will appear in the "recently deleted clusters" section in the Virtual clusters page.

It will stay there for 30 days before Warpstream deletes the data associated with it.

You can restore it at any time by clicking the Restore Cluster button.

recently deleted clusters view

Restoring a Topic

By default, when you delete a Topic from either a Kafka client, or the API, or the UI, it is then impossible to produce and consume to it, but WarpStream also keeps the data around, for 24h by default.

You can restore it at any time by clicking Edit > Restore Topic in the "recently deleted topics" view.

recently deleted topics view

It is also possible to restore it by using the api/v1/undelete_topic API.

To configure how WarpStream interacts with topic deletion with these parameters, please use your usual Kafka client to set these cluster-level parameters (broker level configuration in Kafka terminology):

configuration parameter
warpstream.soft.delete.topic.enable	if true, topic deletion will be a soft deletion, and it will be possible to restore the topics. If false, deleting a topic will cause the immediate deletion of all of the associated data, with no way to recover it. Defaults to true.
warpstream.soft.delete.topic.ttl.hours	If warpstream.soft.delete.topic.enable is true, a deleted topic's data will be kept for this many hours before being irrecoverably deleted. Defaults to 24.

Deletion protection

The second mechanism to prevent accidental deletion of Virtual Clusters and Topics is what we call "deletion protection". You can set a special flag, either through terraform, or through the API, that prevents the deletion altogether.

Through terraform

To set the deletion protection flag using terraform, please refer to our terraform provider documentation for Virtual Clusters and for Topics.

Enabling deletion protection will look like this for virtual clusters:

resource "warpstream_virtual_cluster" "test_configuration" {
  name = "vcn_test_configuration"
  tier = "pro"
  configuration = {
    enable_deletion_protection = true
  }
}

And like this for topics:

resource "warpstream_topic" "topic" {
  topic_name         = "logs"
  partition_count    = 1
  virtual_cluster_id = warpstream_virtual_cluster.test.id
  enable_deletion_protection = true
}

Through the API

• To set the deletion protection flag on a Virtual Cluster, use the UpdateConfiguration API.

• To set the deletion protection flag on a Topic, set the topic-level configuration warpstream.deletion.protection.enabled to true in the topic using any Kafka client.