# Configure Clients to Eliminate AZ Networking Costs With WarpStream, there are no Availability Zone (AZ) networking costs between Agents. This means you can produce and consume data from different AZs without incurring additional networking expenses. The same applies to agent-client communication: WarpStream eliminates AZ networking costs, allowing you to connect clients only to agents within the same AZ. ## **Requirements for Zonal Alignment of Kafka clients** {% hint style="warning" %} WarpStream's zone-aware service discovery system is incompatible with standard Kafka client's rack-aware consumer strategy. When using WarpStream, do **not** configure the rack field on your Kafka clients or enable the rack-aware consumer strategy. If you enable those setting, your Kafka consumers will experience a large amount of unnecessary rebalances. Instead, unset those fields and follow the instructions on this page. {% endhint %} To ensure your Kafka clients connect to Agents within the same availability zone, you need to ensure there is at least one Agent in the same availability zone as your clients and provide the Kafka client’s availability zone. There are two ways to provide the availability zone information: 1. Specifying the availability zone in your client ID 2. Mapping subnets to availability zones in the Agent configuration ### Specifying the availability zone in your client ID Append the following value to your Kafka client's `ClientID`: `ws_az=`. This flag indicates the AZ in which the client is operating.
#### **Example** Here are examples using various kafka libraries of how to set up the `clientID` with the AZ flag
librdkafka/confluent-kafka Generic Configuration ```ini client.id=application-foo,ws_az=us-east-1a ``` Python ```python availability_zone = lookup_az() client_id = "application-foo" p = Producer({ 'client.id': f"{client_id},ws_az={availability_zone}", }) ``` Golang ```go availabilityZone := lookupAZ() clientID = "application-foo" p, err := kafka.NewProducer(&kafka.ConfigMap{ "client.id": fmt.Sprintf("%s,ws_az=%s", clientID, availabilityZone) }) if err != nil { panic(err) } ``` Javascript ```javascript availabilityZone = lookupAZ() clientID = "application-foo" const producer = new Kafka().producer({ 'client.id': `${clientID},ws_az=${availabilityZone}` }); ```
Java ```java String availabilityZone = lookupAZ() String clientID = "application-foo" Properties properties = new Properties(); properties.setProperty(ConsumerConfig.CLIENT_ID_CONFIG, String.format("%s,ws_az=%s", clientID, availabilityZone)); KafkaConsumer consumer = new KafkaConsumer<>(properties); ```
Franz-go ```go availabilityZone := lookupAZ() clientID = "application-foo" cl, err := kgo.NewClient( kgo.SeedBrokers("localhost:9092"), kgo.ClientID(fmt.Sprintf("%s,ws_az=%s", clientID, availabilityZone)), ) ```
Sarama ```go availabilityZone := lookupAZ() clientID = "application-foo" config := sarama.NewConfig() config.ClientID = fmt.Sprintf("%s,ws_az=%s", clientID, availabilityZone) client, err := sarama.NewConsumerGroup(brokers, groupID, config) ```
KafkaJS ```javascript availabilityZone = lookupAZ() clientID = "application-foo" const kafka = new Kafka({ clientId: `${clientID},ws_az=${availabilityZone}` }); ```
To find the AZ that your application is in it is recommended to use your Cloud Provider's Metadata API, for example in AWS querying the following URL: `http://169.254.169.254/latest/meta-data/placement/availability-zone` will give you the availability zone. Alternatively, if you're [using availability zone IDs](#mapping-availability-zones-across-aws-accounts) then you would query: `http://169.254.169.254/latest/meta-data/placement/availability-zone-id` Our [warpstream-go library](https://github.com/warpstreamlabs/warpstream-go/blob/main/pkg/cloudmetadata/cloud_metadata.go) has sample code that demonstrates how to query for your application's availability zone in every major cloud. To learn more about the WarpStream-specific client ID features (like ws\_si and ws\_az) check out [our documentation on configuring Kafka Client ID features](https://docs.warpstream.com/warpstream/kafka/configure-kafka-client/configuring-kafka-client-id-features). ### Mapping subnets to availability zones in the Agent configuration Pass a subnet mapping to the agents via the `-zonedCIDRBlocks`command-line flag or the `WARPSTREAM_ZONED_CIDR_BLOCKS` environment variable. This mapping allows the agents to determine which zone the Kafka client is sending traffic from. The value needs to be a `<>` delimited list of availability zone to CIDR range pairs, where each pair starts with an AZ, a `@`, and a comma separated list of CIDR blocks representing the range of IPs used by the Kafka clients in that given AZ. For example, `us-east-1a@10.0.0.0/19,10.0.32.0/19<>us-east-1b@10.0.64.0/19<>us-east-1c@10.0.96.0/19` indicates that the Kafka clients with IPs that match `10.0.0.0/19` and `10.0.32.0/19` belong to `us-east-1a`, those with IPs that match `10.0.64.0/19` belong to `us-east-1b`, and those with IPs that match `10.0.96.0/19` belong to `us-east-1c`. Note that if an AZ is appended to the Kafka client's `ClientID` and a subnet mapping is also provided to the agents but the values conflict with each other, the AZ from the `ClientID` will be used as the source of truth. ### Mapping availability zones across AWS accounts using IDs Availability zone names are not consistent between different AWS accounts. If you're deploying a WarpStream cluster where Agents and Clients will be deployed in different AWS accounts, you'll want the Agents to advertise their **availability zone ID** instead of their **availability zone name**. This can be accomplished by setting the `WARPSTREAM_LOOKUP_AVAILABILITY_ZONE_ID=true` environment variable on the Agents. Similarly, if you're using the client ID strategy, your Kafka clients will need to advertise their availability zone ID instead of availability zone name. Alternatively, if you're using the subnet-mapping strategy, your mapping should refer to avialability zone IDs instead of names. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.warpstream.com/warpstream/kafka/configure-kafka-client/configure-clients-to-eliminate-az-networking-costs.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.