Protecting important resources from accidental deletion

Soft deletion

By default, when you delete a Virtual Cluster or a topic, Warpstream does not delete the associated data. The data is retained and you can restore the Virtual Cluster or topic.

Warpstream offers two levels of protection against accidental deletion for both Virtual Clusters and topics. First, both Virtual Clusters and topics are soft-deleted by default. Second, WarpStream offers deletion protection, which protects resources from being deleted.

Restoring a Virtual Cluster

When you delete a virtual cluster, either through the API or through the console, it will appear in the "Recently Deleted Clusters" section on the Virtual clusters page.

It will stay there for 30 days before WarpStream deletes the data associated with it.

You can restore the cluster at any time by clicking the Restore Cluster button.

Restoring a Topic

When you delete a topic from a Kafka client, the WarpStream API, or the console, you will no longer be able to produce to or consume from the topic. However, WarpStream will retain the data and allow you to restore the topic. Once a topic is restored, you will be able to resume producing and consuming. The default retention period for soft-deleted topics is 24 hours.

You can restore the topic by clicking Edit > Restore Topic in the "Recently Deleted Topics" view in the WarpStream Console.

It is also possible to restore it by using the api/v1/undelete_topic API.

To configure how WarpStream interacts with topic deletion with these parameters, please use your usual Kafka client to set these cluster-level parameters (in Kafka terminology, these are broker-level configurations):

configuration parameter
warpstream.soft.delete.topic.enable	if true, topic deletion will be a soft deletion, and it will be possible to restore the topics. If false, deleting a topic will cause the immediate deletion of all of the associated data, with no way to recover it. Defaults to true.
warpstream.soft.delete.topic.ttl.hours	If warpstream.soft.delete.topic.enable is true, a deleted topic's data will be kept for this many hours before being irrecoverably deleted. Defaults to 24.

Deletion protection

Virtual Clusters and topics in WarpStream can be protected from deletion. When deletion protection is enabled, you will be unable to delete resources until deletion protection is turned off. This prevents accidental destructive operations, which can occur when using infrastructure-as-code tools.

Through terraform

To set the deletion protection flag using Terraform, please refer to our Terraform provider documentation for Virtual Clusters and for Topics.

Enabling deletion protection will look like this for virtual clusters:

resource "warpstream_virtual_cluster" "test_configuration" {
  name = "vcn_test_configuration"
  tier = "pro"
  configuration = {
    enable_deletion_protection = true
  }
}

And like this for topics:

resource "warpstream_topic" "topic" {
  topic_name         = "logs"
  partition_count    = 1
  virtual_cluster_id = warpstream_virtual_cluster.test.id
  enable_deletion_protection = true
}

Through the API

• To set the deletion protection flag on a Virtual Cluster, use the UpdateConfiguration API.

• To set the deletion protection flag on a topic, set the topic-level configuration warpstream.deletion.protection.enabled to true.