Exposing HikariCP Metrics with Micrometer and Prometheus

This guide is part of Prometheus and Grafana Pool Metrics, and it solves one specific, frustrating failure: you built the scrape config and the dashboards, but curl localhost:8080/actuator/prometheus | grep hikaricp returns nothing. Every other metric is there — JVM, HTTP, even jdbc_connections_* from the DataSource autoconfiguration — yet the hikaricp_connections_active, hikaricp_connections_pending, and hikaricp_connections_acquire_seconds series your Grafana panels query are simply absent. Without them, the saturation and wait-latency dashboards render “No data” and your pending-threads alert can never fire. The root cause is almost always that the running HikariDataSource was never wired to a Micrometer MeterRegistry.

Rapid incident diagnosis

Start by confirming the endpoint works at all and that the gap is HikariCP-specific:

curl -s localhost:8080/actuator/prometheus | grep -c '^'      # endpoint is live, returns a count
curl -s localhost:8080/actuator/prometheus | grep hikaricp    # the empty set is the symptom
curl -s localhost:8080/actuator/prometheus | grep jdbc_connections   # often present even when hikaricp is not

The presence of jdbc_connections_* but absence of hikaricp_* is the key clue: Spring Boot’s generic DataSource metrics bound, but HikariCP’s own metric tracker did not. There are three usual root causes, in order of frequency:

1. The Prometheus registry dependency is missing. Without micrometer-registry-prometheus on the classpath, /actuator/prometheus will not even exist — but if you added it after wiring a different registry, or the endpoint is exposed by a fallback, you can get partial output. Confirm the artifact is resolved.
2. The pool was built outside Spring’s autoconfiguration. If you construct new HikariDataSource(config) yourself in a @Bean method, Spring’s HikariDataSourceMetricsConfiguration does not see it, so it never calls setMetricRegistry/setMetricsTrackerFactory. The pool runs fine; it just publishes nothing.
3. A metricRegistry (Dropwizard) was set instead of Micrometer. HikariCP can publish to either Dropwizard MetricRegistry (setMetricRegistry) or Micrometer (setMetricsTrackerFactory(new MicrometerMetricsTrackerFactory(registry))). Setting the Dropwizard one sends metrics to a registry Prometheus never scrapes, and HikariCP refuses both at once.

A fast confirmation that the binding, not the scrape, is at fault: hit /actuator/metrics/hikaricp.connections.active directly. A 404 there means the meter was never registered; a JSON body with a value means the meter exists and the problem is in the Prometheus exposition or scrape config instead.

The third cause deserves a closer look because the error is silent until startup. HikariCP stores its metrics integration in a single field and guards against double-wiring. If your codebase migrated from Dropwizard and left a setMetricRegistry(dropwizardRegistry) call in place, then later added setMetricsTrackerFactory(...), the pool throws IllegalStateException: cannot use setMetricRegistry and setMetricsTrackerFactory together at construction. If only the Dropwizard call survives, there is no exception — the pool happily publishes to a registry that Prometheus has no idea exists, and /actuator/prometheus stays empty. Grep the codebase for setMetricRegistry before anything else; on a Prometheus stack it should never appear.

Mathematical sizing / parameter formula

There is no sizing math for exposition itself, but the metric you most want to alert on — hikaricp_connections_pending — has a clean interpretation worth fixing in your head before you build the alert. Pending is the count of application threads blocked inside getConnection() waiting for a connection to free up. By Little’s Law applied to the borrow queue, the expected pending depth is L = λ × W, where λ is the connection-request arrival rate and W is the mean wait time once the pool is saturated. The practical consequence: any sustained pending > 0 means arrival rate has exceeded the pool’s service capacity, and wait time is now accumulating linearly toward connectionTimeout.

A worked example: a service receives 800 query requests/second, each holding a connection for 5 ms, against a pool of maximumPoolSize = 10. Required concurrent connections ≈ 800 × 0.005 = 4, comfortably under 10, so pending should sit at 0. If a slow query pushes hold time to 20 ms, demand becomes 800 × 0.020 = 16 — above the ceiling of 10 — and roughly 6 requests queue at any instant. That surplus is exactly what hikaricp_connections_pending reports, which is why alerting on it catches the slow-query regression before the connectionTimeout errors arrive.

Exact remediation & configuration

First, ensure both the actuator and the Prometheus registry are present. With Spring Boot starters:

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
  <groupId>io.micrometer</groupId>
  <artifactId>micrometer-registry-prometheus</artifactId>
</dependency>

Expose the endpoint and tag every metric with the application name so multi-service dashboards can filter:

management:
  endpoints:
    web:
      exposure:
        include: health,info,prometheus,metrics
  endpoint:
    prometheus:
      enabled: true
  metrics:
    tags:
      application: ${spring.application.name}
spring:
  application:
    name: orders-platform
  datasource:
    hikari:
      pool-name: orders-primary
      maximum-pool-size: 10
      register-mbeans: true

If you let Spring Boot autoconfigure the DataSource (the recommended path), this is all you need — HikariDataSourceMetricsConfiguration binds the pool to the Micrometer registry automatically, and hikaricp_* series appear after the first scrape. The pool-name becomes the pool label; set it explicitly so the series are readable.

If you must build the pool programmatically — for example a second datasource or a custom factory — wire the Micrometer registry yourself with setMetricsTrackerFactory, never setMetricRegistry:

@Bean
public HikariDataSource ordersDataSource(MeterRegistry registry) {
    HikariConfig config = new HikariConfig();
    config.setJdbcUrl(env.get("DB_URL"));
    config.setPoolName("orders-primary");
    config.setMaximumPoolSize(10);

    HikariDataSource ds = new HikariDataSource(config);
    // Micrometer binding — produces the hikaricp_* family.
    ds.setMetricsTrackerFactory(new MicrometerMetricsTrackerFactory(registry));
    return ds;
}

Inject the shared MeterRegistry so the pool publishes into the same registry Prometheus scrapes. Do not also call setMetricRegistry(...) — HikariCP throws IllegalStateException: cannot use setMetricRegistry and setMetricsTrackerFactory together. Apply this change with a normal rolling restart; metric wiring takes effect at pool construction, so there is no zero-downtime in-place toggle — drain and replace instances one at a time.

Add the alerting rule on pending, the leading exhaustion indicator:

groups:
  - name: hikaricp.alerts
    rules:
      - alert: HikariPendingThreads
        expr: max_over_time(hikaricp_connections_pending{application="orders-platform"}[1m]) > 0
        for: 2m
        labels:
          severity: warning
        annotations:
          summary: "Threads pending on pool {{ $labels.pool }}"
          description: "Requests are queuing for a connection; pool may be undersized or a query is slow."

Validation & verification

After redeploying, verify the full path end to end. First confirm the meters exist on the application:

curl -s localhost:8080/actuator/prometheus | grep 'hikaricp_connections'
# expect: hikaricp_connections_active{application="orders-platform",pool="orders-primary"} 0.0
#         hikaricp_connections_pending{...} 0.0
#         hikaricp_connections_max{...} 10.0

Then confirm Prometheus actually ingested them. In the Prometheus expression browser or via the HTTP API:

hikaricp_connections_max{application="orders-platform"}
histogram_quantile(0.99, sum by (le, pool) (rate(hikaricp_connections_acquire_seconds_bucket[5m])))

Cross-check the live pool against the database to be sure the numbers are real, not stale: hikaricp_connections should track the backend session count for that application. On PostgreSQL:

SELECT application_name, count(*)
FROM pg_stat_activity
WHERE datname = 'orders'
GROUP BY application_name;

The session count should sit at or below hikaricp_connections_max. A backend count that exceeds it means connections are leaking outside the pool’s accounting — a separate problem the metrics now make visible. For the dashboard and recording-rule layer that consumes these verified series, return to Prometheus and Grafana Pool Metrics.

Frequently Asked Questions

Related

• Prometheus and Grafana Pool Metrics — the parent guide covering scrape config, PromQL, dashboards, and recording rules.
• HikariCP Configuration Deep Dive — the pool parameters these metrics measure and the JMX alternative.
• Detecting Connection Pool Saturation — how to read pending and acquire-wait series during an incident.
• Isolating Connection Pools for Multiple DataSources in Spring Boot — keeping per-pool labels distinct across datasources.