Query-Based Deletions
APIs for interacting with Tableflow query based deletions.
POST /api/v1/tableflow/preview_deletion
Returns a sample of records that a filter would match the provided filter without deleting anything.
Request
Section titled “Request”| Field | Type | Required | Description |
|---|---|---|---|
virtual_ |
string | Yes | The ID of the Tableflow virtual cluster. |
table_ |
string | Yes | The UUID of the table. |
filter |
string | Yes | The deletion filter. See syntax here. |
columns |
string | No | Comma-separated list of column paths to return for each matched record. Defaults to warpstream.value. |
min_ |
integer | No | Minimum Kafka timestamp (inclusive, in nanoseconds) to apply the deletion from. |
max_ |
integer | Yes if min_kafka_timestamp_unix_nanos is set |
Maximum Kafka timestamp (inclusive, in nanoseconds) to apply the deletion to. |
pagination_ |
object | No | The pagination_token from a previous response, to continue scanning where the previous preview left off. |
CURL Example
Section titled “CURL Example”curl https://api.warpstream.com/api/v1/tableflow/preview_deletion \ -H 'warpstream-api-key: XXXXXXXXXX' \ -H 'Content-Type: application/json' \ -d '{ "virtual_cluster_id": "vci_XXXXXXXX_XXXX_XXXX_XXXX_XXXXXXXXXXXX", "table_uuid": "550e8400-e29b-41d4-a716-446655440000", "filter": "user_id == \"user_123\"", "columns": "user_id,email,warpstream.offset" }'Response
Section titled “Response”{ "matched_records": [ { "file_id": 1234, "row_index": 17, "content": { "user_id": "user_123", "email": "user@example.com", "warpstream.offset": 98765 } } ], "pagination_token": { "file_id": "1290", "created_at_nanos": "1700000000000000000" }}| Field | Type | Description |
|---|---|---|
matched_records |
array | Up to 10 sample records matching the filter. |
matched_records[].file_id |
integer | The ID of the data file containing the record. |
matched_records[].row_index |
integer | The row position of the record within the data file. |
matched_records[].content |
object | The requested column values, keyed by column path. |
pagination_token |
object | Present when the scan stopped and there are more files to scan. |
Each preview call scans a limited number of data files and stops once it finds 10 matches. If the response includes a pagination_token, pass it back unchanged in the next request to keep scanning. If no pagination_token is returned, either 10 matches were found or the whole table was scanned.
Create Deletion
Section titled “Create Deletion”POST /api/v1/tableflow/create_deletion
Queues a request to delete every row in the table that matches the filter.
Request
Section titled “Request”| Field | Type | Required | Description |
|---|---|---|---|
virtual_ |
string | Yes | The ID of the Tableflow virtual cluster. |
table_ |
string | Yes | The UUID of the table. |
filter |
string | Yes | The deletion filter. See syntax here. |
min_ |
integer | No | Minimum Kafka timestamp (inclusive, in nanoseconds) to apply the deletion from. |
max_ |
integer | Yes if min_kafka_timestamp_unix_nanos is set |
Maximum Kafka timestamp (inclusive, in nanoseconds) to apply the deletion to. |
CURL Example
Section titled “CURL Example”curl https://api.warpstream.com/api/v1/tableflow/create_deletion \ -H 'warpstream-api-key: XXXXXXXXXX' \ -H 'Content-Type: application/json' \ -d '{ "virtual_cluster_id": "vci_XXXXXXXX_XXXX_XXXX_XXXX_XXXXXXXXXXXX", "table_uuid": "550e8400-e29b-41d4-a716-446655440000", "filter": "user_id == \"user_123\"" }'Response
Section titled “Response”{ "deletion_id": 1}| Field | Type | Description |
|---|---|---|
deletion_id |
integer | The ID of the created deletion request. |
List Deletions
Section titled “List Deletions”POST /api/v1/tableflow/list_deletions
Returns the deletion requests for a table, including pending requests and recently completed or canceled ones.
Request
Section titled “Request”| Field | Type | Required | Description |
|---|---|---|---|
virtual_cluster_id |
string | Yes | The ID of the Tableflow virtual cluster. |
table_uuid |
string | Yes | The UUID of the table. |
CURL Example
Section titled “CURL Example”curl https://api.warpstream.com/api/v1/tableflow/list_deletions \ -H 'warpstream-api-key: XXXXXXXXXX' \ -H 'Content-Type: application/json' \ -d '{ "virtual_cluster_id": "vci_XXXXXXXX_XXXX_XXXX_XXXX_XXXXXXXXXXXX", "table_uuid": "550e8400-e29b-41d4-a716-446655440000" }'Response
Section titled “Response”{ "deletions": [ { "id": 1, "filter": "user_id == \"user_123\"", "resolved_filter": "field_id#15 == \"user_123\"", "min_kafka_timestamp_unix_nanos": 0, "max_kafka_timestamp_unix_nanos": 0, "created_at_unix_nanos": 1700000000000000000, "status": "done", "deletion_batch_id": 3, "processed_files": 120, "processed_records": 4500000, "matching_records": 42, "last_updated_at_unix_nanos": 1700016000000000000, "scheduled_at_unix_nanos": 1700014400000000000, "completed_at_unix_nanos": 1700016000000000000, "completion_duration_nanos": 16000000000000, "scheduled_duration_nanos": 1600000000000 } ]}| Field | Type | Description |
|---|---|---|
deletions[].id |
integer | The ID of the deletion request. |
deletions[].filter |
string | The filter as it was submitted. |
deletions[].resolved_filter |
string | The stored filter, with columns referenced by Iceberg field ID. |
deletions[].min_kafka_timestamp_unix_nanos |
integer | The lower Kafka timestamp bound, or 0 if unset. |
deletions[].max_kafka_timestamp_unix_nanos |
integer | The upper Kafka timestamp bound, or 0 if unset. |
deletions[].created_at_unix_nanos |
integer | When the request was created, in nanoseconds since Unix epoch. |
deletions[].status |
string | One of queued, scheduled, done, or canceled. |
deletions[].deletion_batch_id |
integer | The ID of the batch the request was scheduled into, or 0 if it hasn’t been scheduled. |
deletions[].processed_files |
integer | The number of data files processed so far by the request’s batch. |
deletions[].processed_records |
integer | The number of records scanned so far by the request’s batch. |
deletions[].matching_records |
integer | The number of rows matched by this request’s filter so far. |
deletions[].last_updated_at_unix_nanos |
integer | When the request was last updated, in nanoseconds since Unix epoch. |
deletions[].scheduled_at_unix_nanos |
integer | When the request was scheduled into a batch. Omitted if it was never scheduled. |
deletions[].completed_at_unix_nanos |
integer | When the request completed. Omitted unless the status is done. |
deletions[].completion_duration_nanos |
integer | Time from creation to completion. Omitted unless the status is done. |
deletions[].scheduled_duration_nanos |
integer | Time from scheduling to completion. Omitted unless the status is done. |
Cancel Deletion
Section titled “Cancel Deletion”POST /api/v1/tableflow/cancel_deletion
Cancels a deletion request that is queued or scheduled. Requests that are already done or canceled can’t be canceled. Canceling a scheduled deletion stops it from being applied to any more files, but rows it has already removed will not be restored.
Request
Section titled “Request”| Field | Type | Required | Description |
|---|---|---|---|
virtual_cluster_id |
string | Yes | The ID of the Tableflow virtual cluster. |
table_uuid |
string | Yes | The UUID of the table. |
deletion_id |
integer | Yes | The ID of the deletion request to cancel. |
CURL Example
Section titled “CURL Example”curl https://api.warpstream.com/api/v1/tableflow/cancel_deletion \ -H 'warpstream-api-key: XXXXXXXXXX' \ -H 'Content-Type: application/json' \ -d '{ "virtual_cluster_id": "vci_XXXXXXXX_XXXX_XXXX_XXXX_XXXXXXXXXXXX", "table_uuid": "550e8400-e29b-41d4-a716-446655440000", "deletion_id": 1 }'Response
Section titled “Response”{}