Comment on page
Security and Privacy Considerations
WarpStream's "Bring Your Own Cloud" product is designed to maintain strict security and privacy considerations by ensuring that raw data written to WarpStream clusters never leaves your VPC or object storage buckets.
The only data that ever leaves your VPC is metadata about your Kafka workloads that is required for the correct functioning of your clusters, which includes the following:
- 1.Topic names
- 2.Topic metadata (partition counts, configuration, etc)
- 3.File metadata (object store bucket name, compressed size, uncompressed size, etc)
- 4.Record timestamps and offsets (but never record keys or record contents)
- 5.Consumer group names, configuration and offsets
- 6.Kafka client IDs
- 7.Producer IDs, epochs, and sequence numbers
- 8.Agent Metadata (stored ephemerally in memory, never persisted to disk)
- 1.Number of connections (for load balancing)
- 2.Number of vCPUs (for determining how many concurrent jobs it can run) and utilization.
- 3.Internal / Private IP addresses. These addresses are not routable from the internet, and are required so that the Agents can cluster with each other within a single availability zone.
- 4.Availability zone.
- 9.A small sample of the Agent's error logs so that we can help diagnose and debug issues remotely. This can be disabled by setting the
-disableLogsCollectionflag orWARPSTREAM_DISABLE_LOGS_COLLECTION=trueenvironment variable.
Last modified 1mo ago