Sizing Connection Pools for Kubernetes Horizontal Pod Autoscaling
This guide is part of Connection Pool Sizing Formulas. Autoscaling breaks pool sizing in a specific and predictable way: the number that divides the connection budget is the replica count, and under an HPA that number is not a constant. A pool sized against today’s four replicas is correct until the autoscaler reaches sixteen — which happens under exactly the load that made the scale-out necessary.
The failure has a characteristic shape. Everything works for months. Traffic rises, the HPA adds pods, and the newly created pods fail to start serving while the existing ones continue normally:
2026-07-26T14:22:11Z FATAL: sorry, too many clients already
2026-07-26T14:22:11Z at Connection.parseE (/app/node_modules/pg/lib/connection.js:614:13)
# readiness probe fails, pod never becomes ready, HPA sees latency rise and scales further
The autoscaler then responds to the resulting latency by adding more pods, each of which fails the same way. The scale-out is the incident.
Rapid Incident Diagnosis
Four checks, in order, confirm the diagnosis and give the inputs for the fix.
- Does the failure correlate with pod count rather than traffic? Plot the database’s total backend count against the deployment’s replica count. If they move together and the errors begin at a specific replica number, the divisor is wrong.
- What is the HPA’s ceiling?
kubectl get hpa <name> -o jsonpath='{.spec.maxReplicas}'. This, not the current count, is the number that matters. - What surge does the deployment allow?
kubectl get deploy <name> -o jsonpath='{.spec.strategy.rollingUpdate.maxSurge}'. A percentage applies to the desired count, so atmaxReplicasit adds that fraction again. - How many pools per pod? Worker processes, sidecars, and any second data source each hold their own.
The product of those numbers, multiplied by the configured pool size, is the demand at maximum scale. Compare it to the service’s share of max_connections.
The Divisor Formula
processes_at_max = ceil(maxReplicas × (1 + maxSurge_fraction)) × pools_per_pod
per_process_pool = floor(service_allowance ÷ processes_at_max)
Each term needs care.
maxReplicas comes from the HPA, not from kubectl get pods. If several HPAs target the same workload — a KEDA scaler alongside a CPU-based HPA, for instance — the effective ceiling is the higher of them.
maxSurge applies during every rollout and stacks on top of the current count. Expressed as a percentage it is a fraction of the desired replicas, so at maxReplicas it adds that fraction again. A maxSurge: 25% with maxReplicas: 16 means 20 pods can exist simultaneously.
pools_per_pod covers worker processes inside the pod — Gunicorn, Puma, the Node cluster module — and any additional data source. A pod running four Gunicorn workers against a writer and a replica holds eight pools.
# deployment.yaml — the numbers that determine the divisor
spec:
strategy:
rollingUpdate:
maxSurge: 25% # ← multiplies the ceiling during every rollout
maxUnavailable: 0
---
# hpa.yaml
spec:
minReplicas: 3
maxReplicas: 16 # ← the divisor's base, not the current count
With a service allowance of 180, maxReplicas 16, maxSurge 25% and 2 workers per pod: ceil(16 × 1.25) = 20 pods, × 2 = 40 processes, 180 ÷ 40 = 4 per process.
Exact Remediation and Configuration
Three changes make the sizing durable rather than a one-time correction.
Set the pool from the derived value, with the derivation recorded. The comment is not decoration — it is what stops the next maxReplicas change from silently invalidating the number.
env:
# 180 allowance ÷ (16 maxReplicas × 1.25 maxSurge × 2 workers) = 4
# RAISING maxReplicas REQUIRES RECOMPUTING THIS.
- name: DB_POOL_MAX
value: "4"
- name: DB_POOL_MIN
value: "1"
- name: DB_ACQUIRE_TIMEOUT_MS
value: "2500"
Set maxSurge: 0 where the budget is tight. It trades rollout speed for a lower peak, and on a service that sits close to its allowance it removes the surge term from the divisor entirely. The cost is that a rollout briefly reduces capacity rather than briefly increasing it.
Add a PodDisruptionBudget and readiness gating. A pod that cannot obtain a connection should fail readiness rather than accept traffic, so the load balancer keeps routing to healthy pods while the situation resolves. Combined with a minAvailable budget this prevents a partial connection failure from becoming a full outage.
The structural alternative, for a service whose maximum scale genuinely needs more connections than its allowance permits, is a multiplexing proxy — which removes the replica count from the arithmetic entirely. That decision is covered in Choosing a Connection Proxy for Serverless Postgres, whose reasoning applies equally to an aggressively autoscaled deployment.
Validation and Verification
The test that matters is a deliberate scale-out to the HPA’s maximum, run before traffic forces one.
# Force the deployment to its ceiling and watch the backend count.
kubectl scale deploy/orders-api --replicas=16
watch -n2 'kubectl get pods -l app=orders-api --no-headers | grep -c Running'
# From a psql session against the database, in parallel:
SELECT count(*) AS backends, max_conn.setting::int AS limit
FROM pg_stat_activity,
LATERAL (SELECT setting FROM pg_settings WHERE name = 'max_connections') max_conn
WHERE application_name = 'orders-api'
GROUP BY max_conn.setting;
Two conditions must hold. Every pod reaches Running and passes readiness — no pod fails to obtain a connection. And the backend count settles at or below the derived allowance, with the whole database still under 80% of max_connections.
Then repeat with a rollout in progress, which is the case the surge term exists for:
kubectl rollout restart deploy/orders-api # at maxReplicas, with maxSurge active
If the backend count exceeds the allowance only during the rollout, the surge term was omitted from the divisor.
Frequently Asked Questions
Should the pool scale with the pod count automatically?
What about a VerticalPodAutoscaler alongside the HPA?
2 × cores + 1 will change worker count when VPA changes the CPU request, silently altering the multiplier.Do init containers and jobs count toward the budget?
How does KEDA change this?
Is it safe to raise maxReplicas without touching the pool?
maxReplicas from 16 to 32 doubles the demand, and if the pool value is unchanged, the crossing point simply moves closer.Where should the derived value live — the manifest or the application config?
maxReplicas it was derived from. Keeping the two in the same file means a change to one is visible when reviewing the other, which is the only reliable mechanism for keeping them consistent over time.Should the HPA scale on a database-aware metric?
What about a StatefulSet rather than a Deployment?
Does the readiness probe need its own connection?
Related
- Connection Pool Sizing Formulas — the parent guide covering all three sizing constraints.
- Applying Little’s Law to Connection Pool Sizing — deriving the aggregate demand this divisor is applied to.
- Tuning Spring Boot HikariCP for Microservices — the same arithmetic applied to a JVM fleet.
- Serverless Function Connection Management — the extreme case, where the platform sets the multiplier entirely.