Create Pipeline

Creates a new pipeline in a virtual cluster.

POST /api/v1/create_pipeline

Request Body (JSON):

{
  "virtual_cluster_id": "vci_xxx",
  "pipeline_name": "my-pipeline"
}

CURL Example:

curl https://api.warpstream.com/api/v1/create_pipeline \
-H 'warpstream-api-key: XXXXXXXXXX' \
-H 'Content-Type: application/json' \
-d '{"virtual_cluster_id": "vci_xxx", "pipeline_name": "my-pipeline"}'

Response Body (JSON):

{
  "pipeline_id": "pipeline-xyz",
  "pipeline_name": "my-pipeline",
  "pipeline_state": "paused",
  "pipeline_type": "bento",
  "pipeline_deployed_configuration_id": "" // Initially empty
}

Create Orbit Pipeline

You can also use the create_pipeline api to create an Orbit pipeline. Simply add a pipeline_type of orbit to the request above.

POST /api/v1/create_pipeline

Request Body (JSON):

{
  "virtual_cluster_id": "vci_xxx",
  "pipeline_name": "my-pipeline",
  "pipeline_type": "orbit"
}

Response Body (JSON):

{
  "pipeline_id": "pipeline-xyz",
  "pipeline_name": "my-pipeline",
  "pipeline_state": "paused",
  "pipeline_type": "orbit",
  "pipeline_deployed_configuration_id": "" // Initially empty
}

CURL Example:

curl https://api.warpstream.com/api/v1/create_pipeline \
-H 'warpstream-api-key: XXXXXXXXXX' \
-H 'Content-Type: application/json' \
-d '{"virtual_cluster_id": "vci_xxx", "pipeline_name": "my-pipeline", "pipeline_type": "orbit"}'

Note that you can only have a single Orbit pipeline at a time. To create a new Orbit pipeline first delete the old one.

Create Schema Linking Pipeline

You can also use the create_pipeline api to create a Schema Linking pipeline. Simply add a pipeline_type of schema_linking to the request above.

POST /api/v1/create_pipeline

Request Body (JSON):

{
  "virtual_cluster_id": "vci_sr_xxx",
  "pipeline_name": "my-pipeline",
  "pipeline_type": "schema_linking"
}

Response Body (JSON):

{
  "pipeline_id": "pipeline-xyz",
  "pipeline_name": "my-pipeline",
  "pipeline_state": "paused",
  "pipeline_type": "schema_linking",
  "pipeline_deployed_configuration_id": "" // Initially empty
}

CURL Example:

curl https://api.prod.us-east-1.warpstream.com/api/v1/create_pipeline \
-H 'warpstream-api-key: XXXXXXXXXX' \
-H 'Content-Type: application/json' \
-d '{"virtual_cluster_id": "vci_sr_xxx", "pipeline_name": "my-pipeline", "pipeline_type": "schema_linking"}'

Note that you can only have a single Schema Linking pipeline at a time for each schema registry cluster. To create a new Schema Linking pipeline first delete the old one.