Load Testing Connection Pools with JMeter and Gatling

This guide is part of Java Connection Pool Benchmarks. Micro-benchmarks tell you how fast a pool hands out a connection. They tell you nothing about what happens to your service when the pool runs out, which is the failure you actually need to understand before production does it for you.

A load test answers different questions: at what request rate does the pool saturate, how does latency behave past that point, does the acquisition timeout fire before the request deadline, and does the service degrade or collapse. Getting useful answers requires one specific design decision that most load tests get wrong.

Closed-model load generation hides pool saturation

The decision is the load model. A closed model runs a fixed number of virtual users, each sending a request and waiting for the response before sending the next. An open model injects requests at a fixed rate regardless of whether previous ones have finished.

Closed-model tests are self-limiting. When the pool saturates and responses slow down, the virtual users slow down with them, so the arrival rate falls and the queue never grows. You get elevated latency and a comfortable-looking throughput plateau, and you never see the cascade. Real traffic does not behave that way — real users and upstream services keep arriving at the same rate while your service is slow, which is precisely what makes queues explode.

Every meaningful pool load test must use an open model. In Gatling that is constantUsersPerSec or rampUsersPerSec with injectOpen. In JMeter it is the Concurrency Thread Group or the Throughput Shaping Timer driving arrival rate rather than a fixed thread count.

A Gatling scenario for pool saturation

class PoolSaturationSimulation extends Simulation {

  val httpProtocol = http
    .baseUrl("https://orders.staging.internal")
    .acceptHeader("application/json")
    .shareConnections                       // do not confound with HTTP pooling

  // A read that touches the database on every call and is cheap server-side,
  // so the pool — not the query — is the constraint under test.
  val readScenario = scenario("order lookup")
    .exec(
      http("GET /orders/{id}")
        .get("/orders/#{orderId}")
        .check(status.in(200, 503))         // 503 is a valid, expected outcome
    )
    .feed(csv("order-ids.csv").random)

  setUp(
    readScenario.inject(
      // Warm the pool and the JIT before measuring anything.
      constantUsersPerSec(20).during(2.minutes),
      // Then ramp past the expected saturation point.
      rampUsersPerSec(20).to(600).during(8.minutes),
      constantUsersPerSec(600).during(3.minutes)
    )
  ).protocols(httpProtocol)
   .assertions(
     global.responseTime.percentile4.lt(2000),   // p99 under 2 s
     global.failedRequests.percent.lt(1)
   )
}

Four things in that scenario matter more than the numbers.

.check(status.in(200, 503)) treats shed load as a success from the generator’s point of view. If a 503 counts as a failure, Gatling’s failure percentage conflates “the service correctly rejected excess load” with “the service broke”, and the assertion becomes meaningless. Count them separately and assert on each.

The two-minute constant warm-up is not optional on the JVM. A cold pool has no established connections and a cold JIT interprets your data-access code. Measuring the first two minutes gives you numbers about class loading, not about the pool.

.shareConnections prevents the HTTP client’s own connection behaviour from confounding the result. Without it you may be measuring TLS handshakes on the load generator rather than database pool behaviour on the service.

The ramp is long — eight minutes — because pool saturation is a threshold effect and a fast ramp blows past the interesting region in seconds. You want enough samples on each side of the knee to locate it.

The JMeter equivalent

JMeter’s default Thread Group is a closed model and should not be used for this. Use the Concurrency Thread Group from the Custom Thread Groups plugin, paired with the Throughput Shaping Timer to drive arrival rate:

Concurrency Thread Group
  Target Concurrency: 800          # a ceiling, not the load level
  Ramp Up Time: 8 min
  Hold Target Rate Time: 3 min

Throughput Shaping Timer
  start_rps  end_rps  duration
     20        20       120       # warm-up
     20       600       480       # ramp
    600       600       180       # hold

The Concurrency Thread Group’s target concurrency here is a ceiling that must be high enough not to bind — if it binds, you are back to a closed model. Set it well above the arrival rate multiplied by your worst expected response time, then let the Throughput Shaping Timer determine the actual rate.

Add a Backend Listener writing to InfluxDB or Prometheus rather than reading the summary at the end. The interesting behaviour is a transition, and a single aggregate row across the whole run averages it away.

What to instrument on the service side

The load generator tells you what the client saw. Pool diagnosis needs the server side of the same window, and four series are enough:

The relationship between them is the whole analysis. Pending waiters rising while borrow-hold time stays flat means the pool is too small for the arrival rate — raising maximumPoolSize will help. Pending waiters rising because hold time rose means the database slowed down, and a larger pool sends more concurrent work at an already-struggling database. Same client-side symptom, opposite correct action.

Capture the server-side view too. If active backends are pinned at the database’s useful concurrency with wait events showing lock or buffer contention, the pool is already as large as it should be — that reading is described in interpreting pg_stat_activity wait events for pool issues.

Designing the test environment

A load test that runs against a database three times smaller than production produces a saturation point three times lower and a conclusion you cannot use. Three environment properties must match production or the result is decorative:

Database instance class and storage. Useful concurrency comes from CPU count and I/O parallelism. A db.t3.medium staging instance will saturate at a completely different pool size than a db.r6g.4xlarge.

Data volume and distribution. A query against 10,000 rows uses a different plan and holds a connection for a different length of time than the same query against 100 million. Restore a production-shaped snapshot, or generate data with realistic cardinality.

Network path. If production goes through a proxy or a load balancer and staging connects directly, you are testing a different system — the extra hop changes latency, and the proxy imposes its own connection ceiling.

Replica count matters too, and in a specific way: total connections against the database are replicas x maximumPoolSize, so testing one instance and multiplying is only valid up to the point where the database’s own ceiling binds. Test at production replica count if you want to observe that ceiling. Deriving the per-replica share is covered in Connection Pool Sizing Formulas.

Common failure patterns and remediation

Reading the result

The output you want from the run is three numbers, not a pass or a fail:

1. The saturation rate — the arrival rate at which pending waiters first become persistently non-zero. This is your service’s real capacity per replica.
2. The degradation shape past it — does p99 rise smoothly, or does the service fall over? A smooth rise with clean 503s means your timeout and breaker configuration is working; a vertical cliff with connection resets means it is not.
3. The recovery time — after the load drops back below the saturation rate, how long until p99 returns to baseline. A pool that takes minutes to recover from a 30-second overload has a queue that grew unbounded, which argues for a circuit breaker as described in implementing circuit breakers for connection acquisition failures.

Run the whole thing again after each configuration change, one change at a time. A test that changes pool size and timeout together tells you the combination is better or worse and nothing about which change did it.

Operational boundary

A load test measures a synthetic workload against a synthetic environment. It is excellent at finding the saturation knee and the degradation shape, and poor at predicting absolute production numbers, because real traffic has a query mix, a cache hit rate, and a diurnal shape you did not reproduce. Treat the knee as a relative measure — useful for comparing configurations — and validate the absolute number against production metrics once deployed.

It also cannot find a slow leak. A connection leaked once per thousand requests exhausts a pool over hours, and a twelve-minute test will not show it. Leak detection is a separate mechanism entirely, covered in configuring the HikariCP leak detection threshold.

Frequently Asked Questions

Related

• Java Connection Pool Benchmarks — the parent guide on benchmark rig design.
• Connection Pool Sizing Formulas — turning the measured knee into a pool size.
• Implementing Circuit Breakers for Connection Acquisition Failures — the behaviour to verify past the knee.
• Detecting Connection Pool Saturation — the metrics to record during the run.
• Interpreting pg_stat_activity Wait Events for Pool Issues — the server-side view of the same window.