The bucketURL flag is the URL of the object storage bucket that the WarpStream Agent should write to. See the table below for how to configure it for different object store implementations.
Note that the WarpStream Agents will automatically write all of their data to a top-level warpstream prefix in the bucket. In addition, each cluster will write its data to a cluster-specific prefix (derived from the cluster ID) within the warpstream prefix so multiple WarpStream clusters can co-exist within the same object storage bucket without issue.
An S3 bucket with 16 different cluster prefixes under the top-level warpstream prefix.
The WarpStream Agent embeds the official GCP Golang SDK so authentication/authorization with the storage bucket can be handled in any of the expected ways.
The WarpStream Agent embeds the official Azure Golang SDK which expects the AZURE_STORAGE_ACCOUNT environment variable to be set, along with one of the two following environment variables: AZURE_STORAGE_KEY or AZURE_STORAGE_SAS_TOKEN.
For testing and local development only. All data will be lost once the Agent shuts down.
For testing and local development only. The file store implementation is not robust.