Hosted Prometheus Endpoint

This page describes how to use WarpStream's hosted prometheus endpoint for collecting control plane metrics.

Overview

Almost all WarpStream metrics are exposed directly in the Agents. However, exposing some control plane metrics in the WarpStream Agents can be problematic, mostly due to cardinality.

For example, consumer group lag metrics are most useful when they're tagged by partition, but emitting consumer group metrics tagged by partition in the Agents would make the time series cardinality very high: O(m * n) where m is the number of partitions and n is the number of the Agents.

As a result, WarpStream offers a hosted Prometheus endpoint that exposes a subset of WarpStream's control plane metrics. This endpoint is authenticated and can be scraped by your monitoring system to collect some control plane metrics without incurring the additional cardinality of the Agent host / pod names.

Available Metrics

The hosted Prometheus endpoint only exposes a small subset of the metrics available in WarpStream. The vast majority of WarpStream's metrics are best obtained by scraping the Agent's Prometheus endpoint directly. However, a small subset of WarpStream's metrics are so high cardinality that scraping them from our hosted Prometheus endpoint may be more effective.

The metrics currently exposed by the hosted Prometheus endpoint are:

1. warpstream_consumer_group_state

2. warpstream_consumer_group_generation_id

3. warpstream_consumer_group_num_members

4. warpstream_consumer_group_num_topics

5. warpstream_consumer_group_num_partitions

6. warpstream_consumer_group_max_offset

7. warpstream_consumer_group_lag

8. warpstream_consumer_group_estimated_lag_very_coarse_do_not_use_to_measure_e2e_seconds

9. warpstream_produced_records

10. warpstream_max_offset

11. warpstream_min_offset

12. warpstream_num_records

13. warpstream_control_plane_utilization

14. warpstream_consumer_group_commit_ts

15. warpstream_diagnostic_status

16. warpstream_partition_size_uncompressed_bytes

Sample Prometheus Scraping Configuration

scrape_configs:
  - job_name: "warpstream"
    static_configs:
      - targets: ["api.warpstream.com"]
    metrics_path: "api/v1/monitoring/prometheus/virtual_clusters/$VIRTUAL_CLUSTER_ID"
    scheme: "https"
    basic_auth:
      username: prometheus
      password: $API_KEY

An API Key can be obtained from the "API Keys" tab in the WarpStream console. For more details, see our API Keys reference documentation.

CURLing Manually

curl -u prometheus:$API_KEY "https://api.warpstream.com/api/v1/monitoring/prometheus/virtual_clusters/$VIRTUAL_CLUSTER_ID"

An API Key can be obtained from the "API Keys" tab in the WarpStream console. For more details, see our API Keys reference documentation.