Monitoring ClickPipes

Monitor ClickPipes using Prometheus metrics.

In addition to in-console monitoring, ClickPipes exposes metrics to a Prometheus-compatible endpoint for scraping. These metrics are published with other ClickHouse Cloud service metrics, and allow you to integrate ClickPipes monitoring with your existing observability stack (e.g., Grafana, Datadog).

Metric labels

ClickPipes metrics use the standard service-level labels (clickhouse_org, clickhouse_service, clickhouse_service_name) plus the following ClickPipes-specific labels:

Label	Description
clickpipe_id	The unique identifier of the ClickPipe
clickpipe_name	The name of the ClickPipe
clickpipe_source	The source type (e.g., kafka, postgres, s3)
clickpipe_state	The state of the ClickPipe (e.g., Stopped, Provisioning, Running)

Sample response

The following shows what ClickPipes metrics look like in a Prometheus scrape response for a Kafka ClickPipe. The set of exposed metrics varies by pipe type — see Available metrics for which metrics apply to which pipe types:

# HELP ClickPipes_Info Always equal to 1. Label "clickpipe_state" contains the current state of the pipe: Stopped/Provisioning/Running/Paused/Failed
# TYPE ClickPipes_Info gauge
ClickPipes_Info{clickhouse_org="11dfa1ec-767d-43cb-bfad-618ce2aaf959",clickhouse_service="82b83b6a-5568-4a82-aa78-fed9239db83f",clickhouse_service_name="my service",clickpipe_id="642bb967-940b-459e-9f63-a2833f62ec44",clickpipe_name="my kafka pipe",clickpipe_source="confluent",clickpipe_state="Running"} 1

# HELP ClickPipes_FetchedEvents_Total Total number of records fetched from the source.
# TYPE ClickPipes_FetchedEvents_Total counter
ClickPipes_FetchedEvents_Total{clickhouse_org="11dfa1ec-767d-43cb-bfad-618ce2aaf959",clickhouse_service="82b83b6a-5568-4a82-aa78-fed9239db83f",clickhouse_service_name="my service",clickpipe_id="642bb967-940b-459e-9f63-a2833f62ec44",clickpipe_name="my kafka pipe",clickpipe_source="confluent"} 5535376

# HELP ClickPipes_SentEvents_Total Total number of records sent to ClickHouse
# TYPE ClickPipes_SentEvents_Total counter
ClickPipes_SentEvents_Total{clickhouse_org="11dfa1ec-767d-43cb-bfad-618ce2aaf959",clickhouse_service="82b83b6a-5568-4a82-aa78-fed9239db83f",clickhouse_service_name="my service",clickpipe_id="642bb967-940b-459e-9f63-a2833f62ec44",clickpipe_name="my kafka pipe",clickpipe_source="confluent"} 5534250

# HELP ClickPipes_FetchedBytes_Total Total uncompressed bytes fetched from the source.
# TYPE ClickPipes_FetchedBytes_Total counter
ClickPipes_FetchedBytes_Total{clickhouse_org="11dfa1ec-767d-43cb-bfad-618ce2aaf959",clickhouse_service="82b83b6a-5568-4a82-aa78-fed9239db83f",clickhouse_service_name="my service",clickpipe_id="642bb967-940b-459e-9f63-a2833f62ec44",clickpipe_name="my kafka pipe",clickpipe_source="confluent"} 873286202

# HELP ClickPipes_SentBytes_Total Total uncompressed bytes sent to ClickHouse.
# TYPE ClickPipes_SentBytes_Total counter
ClickPipes_SentBytes_Total{clickhouse_org="11dfa1ec-767d-43cb-bfad-618ce2aaf959",clickhouse_service="82b83b6a-5568-4a82-aa78-fed9239db83f",clickhouse_service_name="my service",clickpipe_id="642bb967-940b-459e-9f63-a2833f62ec44",clickpipe_name="my kafka pipe",clickpipe_source="confluent"} 477187967

# HELP ClickPipes_Errors_Total Total errors ingesting data.
# TYPE ClickPipes_Errors_Total counter
ClickPipes_Errors_Total{clickhouse_org="11dfa1ec-767d-43cb-bfad-618ce2aaf959",clickhouse_service="82b83b6a-5568-4a82-aa78-fed9239db83f",clickhouse_service_name="my service",clickpipe_id="642bb967-940b-459e-9f63-a2833f62ec44",clickpipe_name="my kafka pipe",clickpipe_source="confluent"} 0

# HELP ClickPipes_Latency Time in milliseconds between when the record was produced to when it was written to ClickHouse.
# TYPE ClickPipes_Latency gauge
ClickPipes_Latency{clickhouse_org="11dfa1ec-767d-43cb-bfad-618ce2aaf959",clickhouse_service="82b83b6a-5568-4a82-aa78-fed9239db83f",clickhouse_service_name="my service",clickpipe_id="642bb967-940b-459e-9f63-a2833f62ec44",clickpipe_name="my kafka pipe",clickpipe_source="confluent"} 340

Available metrics

Not every metric is emitted by every pipe type. In particular, CDC ClickPipes expose a different subset of metrics than streaming and object storage ClickPipes.

• Streaming, object storage — Kafka, Kinesis, Pub/Sub, S3, and GCS
• CDC — Postgres, MySQL, MongoDB and BigQuery
• All — every pipe type

Data transfer

Metric	Type	Available for	Description
ClickPipes_FetchedBytes_Total	Counter	Streaming, object storage	Total uncompressed bytes fetched from the source.
ClickPipes_FetchedBytesCompressed_Total	Counter	All	Total bytes fetched from the source, as read over the wire. If the source data is uncompressed, equivalent to ClickPipes_FetchedBytes_Total. For CDC ClickPipes, this excludes bytes fetched from initial load, which is emitted separately to ClickPipes_FetchedBytesInitialLoad_Total.
ClickPipes_FetchedBytesInitialLoad_Total	Counter	CDC	Total bytes fetched during the initial load phase, including resyncs.
ClickPipes_SentBytes_Total	Counter	Streaming, object storage	Total uncompressed bytes sent to ClickHouse.
ClickPipes_SentBytesCompressed_Total	Counter	Streaming, object storage	Total compressed bytes sent to ClickHouse.

Events and records

Metric	Type	Available for	Description
ClickPipes_FetchedEvents_Total	Counter	Streaming, object storage	Total number of records fetched from the source.
ClickPipes_SentEvents_Total	Counter	Streaming, object storage	Total number of records sent to ClickHouse.
ClickPipes_FetchedEvents_PerTable	Gauge	CDC	Number of records fetched per destination table and event type. Uses additional destination_table and event_type labels.

Errors

Metric	Type	Available for	Description
ClickPipes_Errors_Total	Counter	All	Total errors encountered during ingestion.

Latency

Metric	Type	Available for	Description
ClickPipes_Latency	Gauge (avg)	Streaming, object storage	Time in milliseconds between when a record was produced at the source and when it was written to ClickHouse.
ClickPipes_SourceReplicationLatency_MiB	Gauge (avg)	CDC (Postgres only)	Replication slot lag at the source in MiB. Only emitted by Postgres ClickPipes.

Resource utilization

ClickPipes_Replica_* metrics report resource utilization for streaming and object storage ClickPipes. Each ClickPipe has dedicated compute, so these values reflect utilization for individual ClickPipes.

Metric	Type	Available for	Description
ClickPipes_Replica_CPUUsage	Gauge (avg)	Streaming, object storage	Average CPU usage of the ingestion replicas, in millicores.
ClickPipes_Replica_CPULimit	Gauge (last)	Streaming, object storage	Configured CPU limit of the ingestion replicas, in millicores.
ClickPipes_Replica_MemoryUsage	Gauge (avg)	Streaming, object storage	Average memory usage of the ingestion replicas, in bytes.
ClickPipes_Replica_MemoryLimit	Gauge (last)	Streaming, object storage	Configured memory limit of the ingestion replicas, in bytes.

ClickPipes_CDC_* metrics report resource utilization for CDC ClickPipes. All CDC ClickPipes in the same service share compute, so these values reflect the combined usage of every CDC ClickPipe in the service.

Metric	Type	Available for	Description
ClickPipes_CDC_CPUUsage	Gauge (avg)	CDC	Average CPU usage, in millicores. Shared across all CDC ClickPipes in the service.
ClickPipes_CDC_CPULimit	Gauge (last)	CDC	Configured CPU limit, in millicores. Shared across all CDC ClickPipes in the service.
ClickPipes_CDC_MemoryUsage	Gauge (avg)	CDC	Average memory usage, in bytes. Shared across all CDC ClickPipes in the service.
ClickPipes_CDC_MemoryLimit	Gauge (last)	CDC	Configured memory limit, in bytes. Shared across all CDC ClickPipes in the service.
ClickPipes_CDC_NetworkReceiveBytes	Gauge	CDC	Inbound network bandwidth, in bytes (delta per scrape window). Shared across all CDC ClickPipes in the service.

State and progress

Metric	Type	Available for	Description
ClickPipes_Info	Gauge	All	Always equal to 1. Use the clickpipe_state label to filter or alert on pipe state.
ClickPipes_LastFetchedBatchId	Gauge	CDC	ID of the last batch fetched from the source.
ClickPipes_LastSentBatchId	Gauge	CDC	ID of the last batch sent to ClickHouse.

Related pages

• Prometheus endpoint — Endpoint reference, authentication, and setup for external tools (e.g., Grafana, Datadog)
• Error reporting — Where record and system errors are stored