Use the Agent with an S3 Compatible Object Store (MinIO, Oracle Cloud, R2, etc)

This page explains how to use the WarpStream Agent with an S3-compatible object store like MinIO, Oracle Cloud, and R2.

The WarpStream Agent has native support (embeds the provider-specific SDK) for AWS S3, GCP GCS, and Azure Blob Storage. In addition, they can use the AWS S3 SDK to integrate with any "S3 compatible" object store solution like MinIO and Oracle Cloud Object Storage.

To use the Agent with an S3 compatible object store, provide the credentials and force the client to construct the URL using the "path style".

If you have a MinIO docker container running locally on your machine on port 9000, you can run the Agent like this after creating an Access Key in the MinIO UI:

AWS_ACCESS_KEY_ID="wKghTMkQrFqszshHJcop" \
warpstream demo \
-bucketURL "s3://warpstream-minio-bucket?s3ForcePathStyle=true&endpoint="

The MinIO team has a more detailed integration guide on their website as well.

Last updated


Apache, Apache Kafka, Kafka, and associated open source project names are trademarks of the Apache Software Foundation. Kinesis is a trademark of Amazon Web Services.