Skip to content
WarpStream Documentation

Query-Based Deletions

APIs for interacting with Tableflow query based deletions.

POST /api/v1/tableflow/preview_deletion

Returns a sample of records that a filter would match the provided filter without deleting anything.

Field Type Required Description
virtual_cluster_id string Yes The ID of the Tableflow virtual cluster.
table_uuid string Yes The UUID of the table.
filter string Yes The deletion filter. See syntax here.
columns string No Comma-separated list of column paths to return for each matched record. Defaults to warpstream.value.
min_kafka_timestamp_unix_nanos integer No Minimum Kafka timestamp (inclusive, in nanoseconds) to apply the deletion from.
max_kafka_timestamp_unix_nanos integer Yes if min_kafka_timestamp_unix_nanos is set Maximum Kafka timestamp (inclusive, in nanoseconds) to apply the deletion to.
pagination_token object No The pagination_token from a previous response, to continue scanning where the previous preview left off.
curl https://api.warpstream.com/api/v1/tableflow/preview_deletion \
-H 'warpstream-api-key: XXXXXXXXXX' \
-H 'Content-Type: application/json' \
-d '{
"virtual_cluster_id": "vci_XXXXXXXX_XXXX_XXXX_XXXX_XXXXXXXXXXXX",
"table_uuid": "550e8400-e29b-41d4-a716-446655440000",
"filter": "user_id == \"user_123\"",
"columns": "user_id,email,warpstream.offset"
}'
{
"matched_records": [
{
"file_id": 1234,
"row_index": 17,
"content": {
"user_id": "user_123",
"email": "user@example.com",
"warpstream.offset": 98765
}
}
],
"pagination_token": {
"file_id": "1290",
"created_at_nanos": "1700000000000000000"
}
}
Field Type Description
matched_records array Up to 10 sample records matching the filter.
matched_records[].file_id integer The ID of the data file containing the record.
matched_records[].row_index integer The row position of the record within the data file.
matched_records[].content object The requested column values, keyed by column path.
pagination_token object Present when the scan stopped and there are more files to scan.

Each preview call scans a limited number of data files and stops once it finds 10 matches. If the response includes a pagination_token, pass it back unchanged in the next request to keep scanning. If no pagination_token is returned, either 10 matches were found or the whole table was scanned.

POST /api/v1/tableflow/create_deletion

Queues a request to delete every row in the table that matches the filter.

Field Type Required Description
virtual_cluster_id string Yes The ID of the Tableflow virtual cluster.
table_uuid string Yes The UUID of the table.
filter string Yes The deletion filter. See syntax here.
min_kafka_timestamp_unix_nanos integer No Minimum Kafka timestamp (inclusive, in nanoseconds) to apply the deletion from.
max_kafka_timestamp_unix_nanos integer Yes if min_kafka_timestamp_unix_nanos is set Maximum Kafka timestamp (inclusive, in nanoseconds) to apply the deletion to.
curl https://api.warpstream.com/api/v1/tableflow/create_deletion \
-H 'warpstream-api-key: XXXXXXXXXX' \
-H 'Content-Type: application/json' \
-d '{
"virtual_cluster_id": "vci_XXXXXXXX_XXXX_XXXX_XXXX_XXXXXXXXXXXX",
"table_uuid": "550e8400-e29b-41d4-a716-446655440000",
"filter": "user_id == \"user_123\""
}'
{
"deletion_id": 1
}
Field Type Description
deletion_id integer The ID of the created deletion request.

POST /api/v1/tableflow/list_deletions

Returns the deletion requests for a table, including pending requests and recently completed or canceled ones.

Field Type Required Description
virtual_cluster_id string Yes The ID of the Tableflow virtual cluster.
table_uuid string Yes The UUID of the table.
curl https://api.warpstream.com/api/v1/tableflow/list_deletions \
-H 'warpstream-api-key: XXXXXXXXXX' \
-H 'Content-Type: application/json' \
-d '{
"virtual_cluster_id": "vci_XXXXXXXX_XXXX_XXXX_XXXX_XXXXXXXXXXXX",
"table_uuid": "550e8400-e29b-41d4-a716-446655440000"
}'
{
"deletions": [
{
"id": 1,
"filter": "user_id == \"user_123\"",
"resolved_filter": "field_id#15 == \"user_123\"",
"min_kafka_timestamp_unix_nanos": 0,
"max_kafka_timestamp_unix_nanos": 0,
"created_at_unix_nanos": 1700000000000000000,
"status": "done",
"deletion_batch_id": 3,
"processed_files": 120,
"processed_records": 4500000,
"matching_records": 42,
"last_updated_at_unix_nanos": 1700016000000000000,
"scheduled_at_unix_nanos": 1700014400000000000,
"completed_at_unix_nanos": 1700016000000000000,
"completion_duration_nanos": 16000000000000,
"scheduled_duration_nanos": 1600000000000
}
]
}
Field Type Description
deletions[].id integer The ID of the deletion request.
deletions[].filter string The filter as it was submitted.
deletions[].resolved_filter string The stored filter, with columns referenced by Iceberg field ID.
deletions[].min_kafka_timestamp_unix_nanos integer The lower Kafka timestamp bound, or 0 if unset.
deletions[].max_kafka_timestamp_unix_nanos integer The upper Kafka timestamp bound, or 0 if unset.
deletions[].created_at_unix_nanos integer When the request was created, in nanoseconds since Unix epoch.
deletions[].status string One of queued, scheduled, done, or canceled.
deletions[].deletion_batch_id integer The ID of the batch the request was scheduled into, or 0 if it hasn’t been scheduled.
deletions[].processed_files integer The number of data files processed so far by the request’s batch.
deletions[].processed_records integer The number of records scanned so far by the request’s batch.
deletions[].matching_records integer The number of rows matched by this request’s filter so far.
deletions[].last_updated_at_unix_nanos integer When the request was last updated, in nanoseconds since Unix epoch.
deletions[].scheduled_at_unix_nanos integer When the request was scheduled into a batch. Omitted if it was never scheduled.
deletions[].completed_at_unix_nanos integer When the request completed. Omitted unless the status is done.
deletions[].completion_duration_nanos integer Time from creation to completion. Omitted unless the status is done.
deletions[].scheduled_duration_nanos integer Time from scheduling to completion. Omitted unless the status is done.

POST /api/v1/tableflow/cancel_deletion

Cancels a deletion request that is queued or scheduled. Requests that are already done or canceled can’t be canceled. Canceling a scheduled deletion stops it from being applied to any more files, but rows it has already removed will not be restored.

Field Type Required Description
virtual_cluster_id string Yes The ID of the Tableflow virtual cluster.
table_uuid string Yes The UUID of the table.
deletion_id integer Yes The ID of the deletion request to cancel.
curl https://api.warpstream.com/api/v1/tableflow/cancel_deletion \
-H 'warpstream-api-key: XXXXXXXXXX' \
-H 'Content-Type: application/json' \
-d '{
"virtual_cluster_id": "vci_XXXXXXXX_XXXX_XXXX_XXXX_XXXXXXXXXXXX",
"table_uuid": "550e8400-e29b-41d4-a716-446655440000",
"deletion_id": 1
}'
{}