Connection Pool Sizing Formulas

This guide is part of Pool Architecture & Algorithm Fundamentals, and it answers the question every other page eventually defers to: what number should the pool ceiling actually be? Pool size is the single most consequential connection-pooling decision and the one most often made by copying a value from a blog post. It is derivable — from measured concurrency, from the database’s parallel execution capacity, and from your share of a fixed connection budget — and the derivation takes about ten minutes once you know which three numbers to collect.

The formulas below are ordered by how much they require of you. Little’s Law needs two measurements and gives a floor. The knee test needs a load generator and gives a ceiling. The budget division needs an inventory of everything that connects and gives a hard constraint that overrides both. A correct configuration satisfies all three.

Key operational takeaways:

  • Required connections equal throughput multiplied by hold time — Little’s Law — and that product, not request rate, is what the pool must serve.
  • The database’s parallel execution capacity is a hard ceiling above which extra connections add latency and no throughput; find it with a step test rather than a formula.
  • The connection budget divided by every process that connects is a constraint that overrides both, and it is the one most frequently omitted.
  • Hold time is the lever with the best return: halving it halves the required pool, costs nothing at the database, and improves latency at the same time.
  • Every derived number needs the surge replica count as its divisor, not the current one, or the first scale-out invalidates it.
Three constraints on pool size Little's Law sets a floor from demand, the database's parallel capacity sets a ceiling, and the shared connection budget sets a hard limit. The correct size satisfies all three, and there is not always a value that does. floor — Little's Law λ × W throughput × hold time below this, queueing is guaranteed measure both from production ceiling — the knee database parallel capacity found by step test, not by formula above this, latency rises, throughput does not moves with data volume and query mix hard limit — the budget share ÷ processes replicas × workers, at surge exceeding it breaks other services too overrides the other two When the floor exceeds the hard limit, no pool size works That is the signal to change something structural: shorten hold time, reduce process count, or put a multiplexing proxy in the path. Continuing to tune the number at that point is how a capacity problem becomes a recurring incident.
Three constraints bound the answer from different directions. When they are mutually unsatisfiable, the correct response is structural rather than another attempt at the number.

Little’s Law: The Floor

Little’s Law states that the average number of items in a stable system equals the arrival rate multiplied by the average time each item spends in it. Applied to a connection pool, the “items” are connections in use, the arrival rate is borrows per second, and the time in system is hold time — the interval between checkout and check-in.

L = λ × W

L  concurrent connections in use
λ  borrows per second
W  mean hold time in seconds

A service handling 2,000 queries per second with a mean hold time of 5 ms needs 2000 × 0.005 = 10 connections in use on average. That is the floor: a pool smaller than 10 will queue continuously, no matter how the timeouts are set.

Two adjustments turn the floor into a working number. Add headroom for variance — hold time is a distribution, not a constant, and 20–30% is a reasonable allowance for ordinary jitter. Then check the tail: if p99 hold time is fifty times the mean, occasional slow queries will need more connections than the mean-based figure, and the choice is either to size for that or to let the acquisition timeout absorb it. Sizing for the p99 of a heavy-tailed distribution is usually wasteful; letting the timeout absorb it, with a circuit breaker behind, is usually correct.

The measurement that most often goes wrong is hold time. It is not query duration — it is the whole interval a connection is checked out, which includes result-set processing, any application work done inside a transaction, and the framework’s own release timing. On a per-request-checkout framework it is close to request duration; on a per-statement one it is close to query duration. Measuring it directly from pool telemetry rather than inferring it from query latency is the difference between a correct and a badly wrong answer.

Input Where To Measure It Common Mistake
λ — borrows/second Pool checkout counter, not request rate Using requests/s when each request makes several queries
W — hold time Interval between checkout and check-in events Using query duration, which omits application time
Headroom 20–30% over the product Sizing for p99 hold time instead
Result Concurrent connections in use Treating it as a per-service rather than per-process number

Operational Boundary: Little’s Law gives the demand-side floor. It says nothing about whether the database can serve that many connections in parallel, which is the next constraint.

The Knee: The Ceiling

Adding connections increases throughput only while the database has spare capacity to execute them in parallel. Past that point the queue moves from the pool into the database’s own scheduler, where each waiting query holds a backend process rather than a queue slot — strictly worse, because it is invisible to pool metrics and more expensive per waiter.

There is no formula for the knee. It is a property of the database’s cores, storage, lock contention, and working-set size, all of which change with data volume. Older guidance suggesting cores × 2 + spindles predates SSDs and connection-heavy cloud databases; treat it as a starting point for the first step of a test rather than an answer.

The step test is straightforward. Hold offered load constant at roughly 80% of your target peak. Set the pool to 4, record throughput and acquisition-wait p99 for five minutes, then double it — 8, 16, 32, 64 — letting each step run long enough for the pool to actually reach the new ceiling. Plot throughput against pool size. The knee is the last size at which throughput increased by more than a few percent.

Three conditions make the result trustworthy. Run against production-scale data, because a database serving entirely from cache parallelises very differently from one reading from disk. Keep every other parameter fixed, especially the acquisition timeout, or you are measuring two variables. And run from the same number of processes you will deploy, since the knee is a property of the aggregate connection count rather than the per-process one.

Locating the knee by step test Throughput rises with connection count until the database's parallel capacity is reached, after which it flattens while tail latency climbs steeply. throughput p99 latency knee — size here connections buy throughput connections buy only latency 4 16 48 128 aggregate connections across the fleet Past the knee the queue has moved into the database, where each waiter costs a backend process rather than a queue slot.
The knee is where throughput stops responding. Everything to its right converts connections into latency and into backend processes that pool metrics cannot see.

Operational Boundary: The knee bounds the aggregate connection count. Translating it into a per-process value is the next step, and it is where most well-run tests are wasted.

The Budget: The Hard Limit

Both preceding numbers describe aggregate demand. What gets configured is a per-process ceiling, and the conversion is a division whose divisor is routinely wrong.

Start from the database’s total. Subtract superuser_reserved_connections, the replication slots, the monitoring agents, and a working allowance for migrations and interactive sessions during an incident — collectively 10–20% on most systems. Divide what remains between services in proportion to measured demand rather than evenly, because their demands are rarely similar. Then divide each service’s share by the number of processes that will hold a pool, at maximum scale.

per_process_ceiling = floor(
    (max_connections × 0.8 − other_services)
    ÷ (max_replicas_at_surge × workers_per_replica)
)

The divisor is the term that goes wrong. It must include the autoscaler’s maximum, not the current replica count; the rollout surge on top of that; the worker processes inside each replica; and every background tier that imports the same configuration. A service that computes its ceiling from today’s four replicas will exhaust the budget the first time the autoscaler reaches sixteen — under exactly the load that triggered the scale-out.

Term Where To Find It Frequently Omitted
max_connections Database parameter, or provider tier Assumed rather than checked
Reserved / operational superuser_reserved_connections, agents, migrations Almost always
Other services’ share Inventory of what connects Yes — the shared-database blind spot
Max replicas HPA maxReplicas, not current Yes — the most common error
Rollout surge Deployment maxSurge Yes
Workers per replica Gunicorn, Puma, cluster module Sometimes
Background tiers Celery, Sidekiq, cron, consumers Frequently

When the floor from Little’s Law exceeds this limit, no pool size satisfies both, and that is genuinely useful information rather than a failure of the method. It says the deployment needs a structural change: shorter hold times, fewer processes, or a multiplexing proxy that decouples process count from backend count. Choosing between those is the subject of PgBouncer vs RDS Proxy vs pgpool-II.

From database total to per-process ceiling The database total is reduced by reserved and operational connections, split between services by demand, and divided by the surge process count to give the number that goes in the configuration file. max_connections = 500 reserved 60 other services 180 this service — 260 divide by processes at maximum scale HPA maxReplicas 16 + maxSurge 25% 4 × workers per replica 2 = 40 processes 260 ÷ 40 = 6 Write the divisor into the config as a comment The next person to raise maxReplicas needs to know that this number depends on it — otherwise the scale-out is the incident
The configured number is the end of a chain of divisions. Recording the divisor next to the value is what stops an unrelated autoscaler change from becoming an outage.

Operational Boundary: Budget arithmetic determines what you may take. Deciding how the budget is split between teams is a platform-governance question, not a pool-configuration one.

Reducing the Requirement Instead

Every formula above takes hold time as an input. It is also the term you can most easily change, and reducing it improves all three constraints at once — a smaller floor, more headroom under the knee, and a smaller share of the budget.

The reductions that pay best, roughly in order:

Move non-database work outside the transaction. An HTTP call, a large serialisation, or a template render inside the checkout window adds its full duration to hold time. A handler holding a connection for 120 ms of which 8 ms is querying needs fifteen times more connections than one that releases between queries.

Eliminate N+1 access patterns. A request issuing fifty sequential queries occupies fifty consecutive hold windows. Batching them into one or two collapses both the borrow count and the aggregate hold time, and it is usually the single largest available reduction.

Narrow the checkout scope. Per-request checkout holds a connection through the whole request; per-transaction holds it for the transaction; per-statement holds it only while a query is on the wire. Moving one step down the list is frequently a 5–15× reduction, with no database change at all.

Make the queries faster. Indexes and better plans reduce hold time directly, and unlike the others this also reduces load on the database itself. It is listed last only because it is usually the most work.

Change Typical Hold-Time Reduction Cost
External call out of the transaction 5–20× Atomicity trade, needs idempotency
N+1 elimination 10–50× on affected endpoints Moderate refactor
Per-request → per-transaction scope 2–5× Framework configuration
Per-transaction → per-statement 2–10× Only for non-transactional reads
Query optimisation 1.5–10× Highest effort, best side effects

The reason this section exists is that raising the ceiling is the reflex and shrinking the requirement is almost always cheaper. A pool that needs 60 connections after these changes is a different capacity conversation from one that needs 600.

A Worked Example, End to End

Numbers make the interaction concrete. Consider an order service on PostgreSQL, sharing a 500-connection database with three other services.

Step 1 — measure. Pool telemetry over a busy week gives 2,400 borrows per second at peak across the fleet, and a mean hold time of 6 ms with a p99 of 45 ms. Little’s Law gives 2400 × 0.006 = 14.4 connections in use on average; with 25% headroom, an aggregate floor of 18.

Step 2 — find the ceiling. A step test at 80% of peak load shows throughput rising through 8, 16 and 32 aggregate connections, and flattening between 32 and 64 — a 3% gain for a doubling, with acquisition p99 rising from 2 ms to 14 ms. The knee is 32. Anything above that buys latency.

Step 3 — check the budget. max_connections is 500. Reserving 3 superuser slots, 12 for monitoring agents and replication, and 25 for migrations and interactive sessions leaves 460. The four services’ measured demand splits roughly 40/30/20/10, giving this service 184.

Step 4 — divide. The autoscaler’s maxReplicas is 12; maxSurge at 25% adds 3; each replica runs 2 worker processes. The divisor is 15 × 2 = 30. 184 ÷ 30 = 6.1, so 6 per process — an aggregate of 180 at maximum scale.

Step 5 — reconcile. The floor is 18 aggregate, the knee is 32 aggregate, the budget permits 180. The binding constraint here is the knee, not the budget: sizing to the full 180 would put the service far past the point where connections stop helping. The correct per-process value is 32 ÷ 30 ≈ 1, which is uncomfortably small — and that discomfort is the real finding.

What it reveals is a topology problem. Thirty processes sharing an aggregate optimum of 32 connections means each process can hold roughly one, which leaves no headroom for a burst on any individual replica. The options are fewer, larger processes — 6 replicas of 2 workers, giving 12 processes and a per-process value of 2–3 — or a multiplexing proxy that decouples the process count from the backend count entirely.

Step Result Interpretation
Little’s Law floor 18 aggregate Demand-side minimum
Knee 32 aggregate Database-side maximum
Budget 184 aggregate Not binding here
Divisor 30 processes Surge replicas × workers
Naive per-process 6 From the budget alone — 5× past the knee
Correct per-process 1–2 From the knee — and too small to be comfortable
Conclusion Reduce process count or add a proxy The number was never the problem

The value of doing all four steps is exactly this: sizing from the budget alone gives 6, which is five times past the point where connections help, and no measurement would have contradicted it because the service would appear to work. The knee is what reveals that the deployment shape, not the pool setting, needs to change.

Common Failure Patterns & Remediation

Symptom Root Cause Exact Fix Validation
Formula says 10, service still queues Hold time measured as query duration Measure checkout-to-check-in directly Recomputed value matches observed concurrency
Raising the pool does not help Already past the knee Reduce hold time, or accept the ceiling Throughput responds again after the reduction
Works at current scale, fails on scale-out Divisor was the current replica count Recompute using maxReplicas + maxSurge Scale-out test reaches maximum cleanly
Fine in isolation, breaks the shared database Other services omitted from the budget Inventory all connecting processes Total backends stay under 80% of the limit
Pool sized correctly, still exhausts overnight Background tiers inherited the same config Separate configuration per tier Trough backend count matches expectation
No value satisfies floor and budget together Structural mismatch Shorten hold time or add a multiplexing proxy A satisfiable range exists after the change

Frequently Asked Questions

Is cores × 2 + effective_spindle_count still a useful formula?
As a starting point for the first step of a test, yes. As an answer, no — it was derived for spinning disks and single-database servers, and it takes no account of the connection budget, of hold time, or of how many processes share the database. Use it to pick where to begin measuring.
Should the pool be sized for average load or peak?
For peak concurrency, with the acquisition timeout absorbing anything beyond it. Sizing for the average guarantees queueing during every busy period; sizing for the absolute maximum wastes budget that another service needs. Peak plus 20–30% headroom is the usual balance.
How often should sizing be revisited?
When query mix, data volume, or replica count changes materially — and always when the autoscaler’s maximum is raised, because that changes the divisor even if nothing about the workload changed. In practice, a review each time the deployment’s shape changes is enough.
Does an async runtime change the formulas?
Not the formulas, only the inputs. Little’s Law still applies, but hold time in an async runtime is close to query duration rather than request duration, so the resulting number is much smaller. Applying a thread-per-request intuition to an async service typically overshoots by five to ten times.
What if hold time varies by two orders of magnitude across endpoints?
Split the traffic across two pools sized independently — a small pool with a tight timeout for interactive work, and a separate one for reporting. A single pool sized for the slow path guarantees the fast path queues behind it, and a single pool sized for the fast path guarantees the slow path times out.