> For the complete documentation index, see [llms.txt](https://docs.warpstream.com/warpstream/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.warpstream.com/warpstream/kafka/configure-kafka-client/consumer-group-protocol-kip-848.md). # Consumer Group Protocol (KIP-848) [KIP-848](https://cwiki.apache.org/confluence/display/KAFKA/KIP-848%3A+The+Next+Generation+of+the+Consumer+Rebalance+Protocol) is the next generation of Kafka's consumer group protocol. WarpStream has full support for this protocol including migration of groups on the older protocol to the newer protocol. ## Configuration ### Broker configs | Setting | Default | Description | | -------------------------------------- | --------------- | ---------------------------------------------------------------------------------------------------- | | `group.consumer.session.timeout.ms` | `45000` | How long a member can go without heartbeating before it's removed from the group. | | `group.consumer.heartbeat.interval.ms` | `5000` | How often members send heartbeats. | | `group.consumer.max.size` | `32000` | Maximum number of members in a group. | | `group.consumer.migration.policy` | `bidirectional` | Whether a group with active members can switch protocols. See [Migration policy](#migration-policy). | These can be altered using the alter config or incremental alter config kafka apis. ### Group configs You can override the session timeout and heartbeat interval for an individual group with `consumer.session.timeout.ms` and `consumer.heartbeat.interval.ms` consumer group configs using the incremental alter configs kafka api. ## Migrating from the older classic protocol Migrated consumers will resume from the last committed offset for a partition. ### 1. Upgrade your clients Upgrade to a client version that supports KIP-848. Opting in is a client-side setting: for the Java client and librdkafka, set `group.protocol=consumer`. See [Client migration guides](#client-migration-guides) for each client's minimum version and instructions. ### 2. Update your consumer configuration Remove settings that only apply to the classic protocol: `session.timeout.ms`, `heartbeat.interval.ms`, and `partition.assignment.strategy`. Some clients, including the Java client, refuse to start if they're set together with `group.protocol=consumer`. Use the cluster-wide or per-group settings in [Configuration](#configuration) instead. ### 3. Rolling restart your consumers **Rolling migration.** The first client which connects with WarpStream on the new protocol will cause the group to automatically convert to the modern protocol. Existing members still heartbeating on the old protocol are supported. As all clients finish rotating, all members of the group will be on the new protocol. This requires the [migration policy](#migration-policy) to be `upgrade` or `bidirectional`. WarpStream won't convert a group that still has classic members if any of the following are true: * The migration policy doesn't allow upgrades. * The group uses a custom assignor which isn't supported in the old protocol. ### Migration policy `group.consumer.migration.policy` controls whether a group with active members can switch protocols. Empty groups can always switch. | Value | Classic to KIP-848 | KIP-848 to classic | | --------------- | ------------------ | ------------------ | | `disabled` | Offline only | Offline only | | `upgrade` | Offline or rolling | Offline only | | `downgrade` | Offline only | Offline or rolling | | `bidirectional` | Offline or rolling | Offline or rolling | The default is `bidirectional`. ### Rolling back To roll back, switch clients back to the classic protocol, for example with `group.protocol=classic`. The group will only roll back once every member on the newer protocol has left the group. ## Client migration guides Each client library documents its own KIP-848 migration steps. This table covers the clients in [Tuning for Performance](/warpstream/kafka/configure-kafka-client/tuning-for-performance.md#client-settings). | Client | KIP-848 support | Migration guide | | -------------------------- | ------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Java client | 4.0 and later | [Consumer Rebalance Protocol: Upgrade & Downgrade](https://kafka.apache.org/43/operations/consumer-rebalance-protocol/#upgrade--downgrade) | | librdkafka | 2.12.0 and later | [Next Generation Consumer Group Protocol (KIP-848)](https://docs.confluent.io/platform/current/clients/librdkafka/html/md_INTRODUCTION.html#next-generation-consumer-group-protocol-kip-848) | | confluent-kafka-go | 2.12.0 and later | [KIP-848 Migration Guide](https://github.com/confluentinc/confluent-kafka-go/blob/HEAD/kafka/kip-848-migration-guide.md) | | confluent-kafka-python | 2.12.0 and later | [KIP-848 Migration Guide](https://docs.confluent.io/platform/current/clients/confluent-kafka-python/html/index.html#kip-848-migration-guide) | | confluent-kafka-javascript | 1.6.0 and later | Follow the [librdkafka guide](https://docs.confluent.io/platform/current/clients/librdkafka/html/md_INTRODUCTION.html#next-generation-consumer-group-protocol-kip-848). Set `group.protocol` outside the `kafkaJS` config block. | | franz-go | 1.22.0 and later | Enable [`kgo.ServerSideBalancer`](https://pkg.go.dev/github.com/twmb/franz-go/pkg/kgo#ServerSideBalancer), then follow the [upgrade and downgrade flow](https://kafka.apache.org/43/operations/consumer-rebalance-protocol/#upgrade--downgrade). | | Segment kafka-go | Not supported | None | | Sarama | Not supported | [Tracking issue](https://github.com/IBM/sarama/issues/3818) | | KafkaJS | Not supported | [Migrate to confluent-kafka-javascript](https://docs.confluent.io/kafka-clients/javascript/current/migration.html#migrate-to-javascript-client-from-kafkajs), then follow the [KIP-848 guide](https://docs.confluent.io/platform/current/clients/librdkafka/html/md_INTRODUCTION.html#next-generation-consumer-group-protocol-kip-848). | | Kafka Connect | Not supported. Connect workers use their own group protocol. | None | | Spark | Not applicable. Spark tracks offsets in its own checkpoints. | [Kafka-specific configurations](https://spark.apache.org/docs/latest/streaming/structured-streaming-kafka-integration.html#kafka-specific-configurations) | ## See also * [Broker Configuration Reference](/warpstream/kafka/reference/protocol-and-feature-support/broker-configuration-reference.md) * [Tuning for Performance](/warpstream/kafka/configure-kafka-client/tuning-for-performance.md) * [Monitoring Consumer Groups](/warpstream/agent-setup/monitor-the-warpstream-agents/monitoring-consumer-groups.md) * [Apache Kafka KIP-848](https://cwiki.apache.org/confluence/display/KAFKA/KIP-848%3A+The+Next+Generation+of+the+Consumer+Rebalance+Protocol) --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://docs.warpstream.com/warpstream/kafka/configure-kafka-client/consumer-group-protocol-kip-848.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.