Skip to main content
client_python
GitHub Toggle Dark/Light/Auto mode Toggle Dark/Light/Auto mode Toggle Dark/Light/Auto mode Back to homepage

Custom Collectors

Sometimes it is not possible to directly instrument code, as it is not in your control. This requires you to proxy metrics from other systems.

To do so you need to create a custom collector, for example:

from prometheus_client.core import GaugeMetricFamily, CounterMetricFamily, REGISTRY
from prometheus_client.registry import Collector

class CustomCollector(Collector):
    def collect(self):
        yield GaugeMetricFamily('my_gauge', 'Help text', value=7)
        c = CounterMetricFamily('my_counter_total', 'Help text', labels=['foo'])
        c.add_metric(['bar'], 1.7)
        c.add_metric(['baz'], 3.8)
        yield c

REGISTRY.register(CustomCollector())

SummaryMetricFamily, HistogramMetricFamily and InfoMetricFamily work similarly.

A collector may implement a describe method which returns metrics in the same format as collect (though you don’t have to include the samples). This is used to predetermine the names of time series a CollectorRegistry exposes and thus to detect collisions and duplicate registrations.

Usually custom collectors do not have to implement describe. If describe is not implemented and the CollectorRegistry was created with auto_describe=True (which is the case for the default registry) then collect will be called at registration time instead of describe. If this could cause problems, either implement a proper describe, or if that’s not practical have describe return an empty list.

Collector protocol

A collector is any object that implements a collect method. Optionally it can also implement describe.

collect()

Returns an iterable of metric family objects (GaugeMetricFamily, CounterMetricFamily, etc.). Called every time the registry is scraped.

describe()

Returns an iterable of metric family objects used only to determine the metric names the collector produces. Samples on the returned objects are ignored. If not implemented and the registry has auto_describe=True, collect is called at registration time instead.

value vs labels

Every metric family constructor accepts either inline data or labels, but not both. The inline data parameter name varies by type: value for Gauge, Counter, and Info; count_value/sum_value for Summary; buckets for Histogram.

  • Pass inline data to emit a single unlabelled metric directly from the constructor.
  • Pass labels (a sequence of label names) and then call add_metric one or more times to emit labelled metrics.
# single unlabelled value
GaugeMetricFamily('my_gauge', 'Help text', value=7)

# labelled metrics via add_metric
g = GaugeMetricFamily('my_gauge', 'Help text', labels=['region'])
g.add_metric(['us-east-1'], 3)
g.add_metric(['eu-west-1'], 5)

API Reference

GaugeMetricFamily

GaugeMetricFamily(name, documentation, value=None, labels=None, unit='')
ParameterTypeDefaultDescription
namestrrequiredMetric name.
documentationstrrequiredHelp text shown in the /metrics output.
valuefloatNoneEmit a single unlabelled sample with this value. Mutually exclusive with labels.
labelsSequence[str]NoneLabel names. Use with add_metric. Mutually exclusive with value.
unitstr''Optional unit suffix appended to the metric name.

add_metric(labels, value, timestamp=None)

Add a labelled sample to the metric family.

ParameterTypeDescription
labelsSequence[str]Label values in the same order as the labels constructor argument.
valuefloatThe gauge value.
timestampfloat or TimestampOptional Unix timestamp for the sample.
g = GaugeMetricFamily('temperature_celsius', 'Temperature by location', labels=['location'])
g.add_metric(['living_room'], 21.5)
g.add_metric(['basement'], 18.0)
yield g

CounterMetricFamily

CounterMetricFamily(name, documentation, value=None, labels=None, created=None, unit='', exemplar=None)

If name ends with _total, the suffix is stripped automatically so the metric is stored without it and the _total suffix is added on exposition.

ParameterTypeDefaultDescription
namestrrequiredMetric name. A trailing _total is stripped and re-added on exposition.
documentationstrrequiredHelp text.
valuefloatNoneEmit a single unlabelled sample. Mutually exclusive with labels.
labelsSequence[str]NoneLabel names. Use with add_metric. Mutually exclusive with value.
createdfloatNoneUnix timestamp the counter was created at. Only used when value is set.
unitstr''Optional unit suffix.
exemplarExemplarNoneExemplar for the single-value form. Only used when value is set.

add_metric(labels, value, created=None, timestamp=None, exemplar=None)

ParameterTypeDescription
labelsSequence[str]Label values.
valuefloatThe counter value.
createdfloatOptional Unix timestamp the counter was created at.
timestampfloat or TimestampOptional Unix timestamp for the sample.
exemplarExemplarOptional exemplar. See Exemplars.
c = CounterMetricFamily('http_requests_total', 'HTTP requests by status', labels=['status'])
c.add_metric(['200'], 1200)
c.add_metric(['404'], 43)
c.add_metric(['500'], 7)
yield c

SummaryMetricFamily

SummaryMetricFamily(name, documentation, count_value=None, sum_value=None, labels=None, unit='')

count_value and sum_value must always be provided together or not at all.

ParameterTypeDefaultDescription
namestrrequiredMetric name.
documentationstrrequiredHelp text.
count_valueintNoneObservation count for a single unlabelled metric. Must be paired with sum_value.
sum_valuefloatNoneObservation sum for a single unlabelled metric. Must be paired with count_value.
labelsSequence[str]NoneLabel names. Use with add_metric. Mutually exclusive with count_value/sum_value.
unitstr''Optional unit suffix.

add_metric(labels, count_value, sum_value, timestamp=None)

ParameterTypeDescription
labelsSequence[str]Label values.
count_valueintThe number of observations.
sum_valuefloatThe sum of all observed values.
timestampfloat or TimestampOptional Unix timestamp for the sample.
s = SummaryMetricFamily('rpc_duration_seconds', 'RPC duration', labels=['method'])
s.add_metric(['get'], count_value=1000, sum_value=53.2)
s.add_metric(['put'], count_value=400, sum_value=28.7)
yield s

HistogramMetricFamily

HistogramMetricFamily(name, documentation, buckets=None, sum_value=None, labels=None, unit='')
ParameterTypeDefaultDescription
namestrrequiredMetric name.
documentationstrrequiredHelp text.
bucketsSequenceNoneBucket data for a single unlabelled metric. Each entry is a (le, value) pair or (le, value, exemplar) triple. Must include a +Inf bucket. Mutually exclusive with labels.
sum_valuefloatNoneObservation sum. Cannot be set without buckets. Omitted for histograms with negative buckets.
labelsSequence[str]NoneLabel names. Use with add_metric. Mutually exclusive with buckets.
unitstr''Optional unit suffix.

add_metric(labels, buckets, sum_value, timestamp=None)

ParameterTypeDescription
labelsSequence[str]Label values.
bucketsSequenceBucket data. Each entry is a (le, value) pair or (le, value, exemplar) triple. Must be sorted and include +Inf.
sum_valuefloat or NoneThe sum of all observed values. Pass None for histograms with negative buckets.
timestampfloat or TimestampOptional Unix timestamp.
h = HistogramMetricFamily('request_size_bytes', 'Request sizes', labels=['handler'])
h.add_metric(
    ['api'],
    buckets=[('100', 5), ('1000', 42), ('+Inf', 50)],
    sum_value=18350.0,
)
yield h

InfoMetricFamily

InfoMetricFamily(name, documentation, value=None, labels=None)
ParameterTypeDefaultDescription
namestrrequiredMetric name. The _info suffix is added automatically on exposition.
documentationstrrequiredHelp text.
valueDict[str, str]NoneKey-value label pairs for a single unlabelled info metric. Mutually exclusive with labels.
labelsSequence[str]NoneLabel names for the outer grouping. Use with add_metric. Mutually exclusive with value.

add_metric(labels, value, timestamp=None)

ParameterTypeDescription
labelsSequence[str]Outer label values (from the labels constructor argument).
valueDict[str, str]Key-value label pairs that form the info payload.
timestampfloat or TimestampOptional Unix timestamp.
# single unlabelled info metric
yield InfoMetricFamily('build', 'Build metadata', value={'version': '1.2.3', 'commit': 'abc123'})

# labelled: one info metric per service
i = InfoMetricFamily('service_build', 'Per-service build info', labels=['service'])
i.add_metric(['auth'], {'version': '2.0.1', 'commit': 'def456'})
i.add_metric(['api'], {'version': '1.9.0', 'commit': 'ghi789'})
yield i

Real-world example

Proxying metrics from an external source:

from prometheus_client.core import CounterMetricFamily, GaugeMetricFamily, REGISTRY
from prometheus_client.registry import Collector
from prometheus_client import start_http_server

# Simulated external data source
_QUEUE_STATS = {
    'orders': {'depth': 14, 'processed': 9821},
    'notifications': {'depth': 3, 'processed': 45210},
}

class QueueCollector(Collector):
    def collect(self):
        depth = GaugeMetricFamily(
            'queue_depth',
            'Current number of messages waiting in the queue',
            labels=['queue'],
        )
        processed = CounterMetricFamily(
            'queue_messages_processed_total',
            'Total messages processed from the queue',
            labels=['queue'],
        )
        for name, stats in _QUEUE_STATS.items():
            depth.add_metric([name], stats['depth'])
            processed.add_metric([name], stats['processed'])
        yield depth
        yield processed

REGISTRY.register(QueueCollector())

if __name__ == '__main__':
    start_http_server(8000)
    import time
    while True:
        time.sleep(1)