Tuning SQLAlchemy pool_size and max_overflow

This guide is part of FastAPI SQLAlchemy Pool Configuration. pool_size and max_overflow are the two numbers that decide how many database connections a SQLAlchemy engine can hold, and the relationship between them is not the one most people assume. They are not a minimum and a maximum — they are a retained count and a disposable allowance, and connections in each behave differently.

The default is pool_size=5, max_overflow=10, which permits 15 concurrent connections per engine and retains 5. That default is fine for a single-process script and wrong for almost every production deployment, in one direction or the other.

What each parameter caps

QueuePool, SQLAlchemy’s default for most drivers, works like this:

  • Up to pool_size connections are created on demand and kept in the pool when returned.
  • Beyond that, up to max_overflow further connections may be created. When an overflow connection is returned, it is closed immediately, not retained.
  • Past pool_size + max_overflow, a checkout waits up to pool_timeout seconds and then raises TimeoutError — surfaced through SQLAlchemy as QueuePool limit of size X overflow Y reached, connection timed out.

The asymmetry between retained and overflow connections is the part that matters operationally. Overflow connections pay full connection establishment cost — TCP handshake, TLS, authentication, PostgreSQL backend fork — on every checkout and get thrown away on release. A workload that sits permanently in the overflow band is reconnecting constantly, which shows up as latency rather than errors, and is one of the more common invisible performance problems in SQLAlchemy deployments.

Retained connections versus overflow connections A capacity bar for one engine. The first band is pool_size, whose connections are retained on release. The second band is max_overflow, whose connections are closed on release. Beyond both, checkouts wait for pool_timeout and then raise. Overflow connections are created and destroyed on every use pool_size — retained kept on release, reused by the next checkout no handshake cost after the first max_overflow — disposable closed on release, recreated next time full handshake cost every checkout beyond both wait up to pool_timeout then TimeoutError A service whose steady-state concurrency sits inside the amber band is reconnecting on every request. The symptom is added latency with no errors, which is easy to attribute to the database instead. rule of thumb pool_size should cover steady-state concurrency; max_overflow covers bursts only Both together must fit the fleet inside the database ceiling: processes x (pool_size + max_overflow).
The green band is your working set; the amber band is emergency capacity that costs a handshake every time it is used.

Sizing them

Set pool_size from steady-state concurrency, and max_overflow from burst headroom:

from sqlalchemy.ext.asyncio import create_async_engine

engine = create_async_engine(
    "postgresql+asyncpg://orders_app@db.internal/orders",

    # Retained: covers normal concurrent database work in this process.
    pool_size=10,

    # Disposable: absorbs bursts without becoming the steady state.
    max_overflow=5,

    # Fail fast rather than hanging a request.
    pool_timeout=5,

    # Age cap, below the shortest idle timer in the network path.
    pool_recycle=280,

    # Validate a connection before handing it out.
    pool_pre_ping=True,

    echo_pool=False,
)

The number to start from is concurrent database operations, not request concurrency and not thread count. In an async service most of a request is spent awaiting things other than the database, so concurrent database work is typically a small fraction of concurrent requests. Measuring it is better than guessing: instrument checkout duration and checked-out count, and take the p95.

The fleet arithmetic then constrains you from the other direction:

worst case = processes x (pool_size + max_overflow)
           =     8     x (   10     +      5      ) = 120 backends

That total must fit inside the database’s effective ceiling alongside every other service. If it does not, the fix is a smaller pool per process or a proxy, not a larger max_overflow — see tuning PostgreSQL max_connections and superuser_reserved_connections.

A useful default shape for a web service is max_overflow at roughly half of pool_size. Large overflow allowances are tempting because they prevent timeouts, but they convert a visible capacity problem into an invisible latency problem and let the fleet quietly exceed the database ceiling during a spike.

The parameters that go with them

Parameter Purpose Typical value
pool_timeout Seconds to wait for a connection before raising 5–10
pool_recycle Maximum connection age in seconds below the shortest external idle timer
pool_pre_ping Test a connection before checkout, replacing dead ones True for most deployments
pool_use_lifo Reuse the most recently returned connection True when the database can shrink
echo_pool Log checkout and return events "debug" temporarily, never in production

pool_pre_ping deserves a note. It issues a trivial round trip before handing a connection to the caller, so a connection severed by a load balancer is detected and replaced transparently rather than raising a OperationalError in the middle of a request. The cost is one round trip per checkout, typically well under a millisecond on a private network. Enable it unless you have measured that cost and it matters; combine it with pool_recycle rather than treating either as a substitute for the other.

pool_use_lifo=True is worth setting when the database benefits from letting idle connections age out — for example, behind a proxy or on a managed database whose capacity scales with load. LIFO checkout concentrates traffic on a few connections and lets the rest idle past pool_recycle, whereas the default FIFO round-robins across all of them and keeps them all warm.

Async engines and event loops

With create_async_engine and asyncpg, the pool is AsyncAdaptedQueuePool and the same parameters apply. Two async-specific rules matter:

One engine per event loop. An engine created in one loop and used from another produces confusing failures. In practice this means creating the engine inside the application’s lifespan handler rather than at module import, so it binds to the loop that will actually use it:

@asynccontextmanager
async def lifespan(app: FastAPI):
    app.state.engine = create_async_engine(DSN, pool_size=10, max_overflow=5)
    app.state.sessionmaker = async_sessionmaker(app.state.engine, expire_on_commit=False)
    yield
    await app.state.engine.dispose()

dispose() on shutdown is required. Without it, connections are closed by garbage collection at an unpredictable time, which during a rolling deploy means the old pod’s backends linger while the new pod’s pool is opening — briefly doubling the fleet’s connection count.

For a forking server, dispose() also matters after the fork. A pool created before fork() is inherited by every child, and several processes then share the same TCP sockets, which corrupts the protocol. Create the engine after forking, or call engine.dispose() in a post-fork hook.

Common failure patterns and remediation

Symptom Underlying cause Remediation
QueuePool limit ... connection timed out Concurrency exceeds pool_size + max_overflow Raise pool_size if the database has headroom; otherwise shorten holds
Latency high, no pool errors Steady state sits in the overflow band, reconnecting constantly Raise pool_size so overflow is only for bursts
OperationalError: server closed the connection after quiet periods pool_recycle above the shortest external idle timer Lower pool_recycle; enable pool_pre_ping
Database ceiling exhausted during a scale-up max_overflow large, multiplied by process count Lower max_overflow; derive autoscaling limits from the connection budget
Sessions hold connections for whole requests Session opened at request start, closed at request end Scope the session tightly around database work
Errors after forking workers Pool inherited across fork() Create the engine post-fork, or dispose in a post-fork hook
Connections linger after shutdown dispose() never called Dispose in the lifespan or shutdown handler
InterfaceError: another operation is in progress One session shared across concurrent tasks One session per task; never share an AsyncSession

Confirming the sizing from metrics

SQLAlchemy exposes pool state directly, and four numbers answer the sizing question:

p = engine.pool
logger.info(
    "pool size=%s checked_in=%s checked_out=%s overflow=%s",
    p.size(), p.checkedin(), p.checkedout(), p.overflow(),
)

overflow() is the diagnostic one. It returns the number of connections created beyond pool_size — and if it is persistently above zero at steady state, pool_size is too small. A brief positive value during a traffic spike is exactly what overflow is for; a value that never returns to zero means you are paying a handshake per request.

checkedout() at pool_size + max_overflow with requests waiting is outright saturation. Export both as gauges and alert on sustained saturation rather than on any single sample, following the approach in Detecting Connection Pool Saturation.

What the overflow counter tells you Three overflow patterns over time: always zero meaning pool_size is generous, brief spikes meaning the sizing is correct, and permanently positive meaning pool_size is too small and every request pays a handshake. The overflow counter is the sizing verdict always zero pool_size covers everything consider lowering it to free budget brief spikes only sizing is correct overflow is doing its job permanently positive pool_size too small a handshake on every request The third pattern produces no errors at all, which is why it survives in production for months. Graph the counter before concluding a latency problem lives in the database.
Persistent overflow is the sizing bug that never raises an exception. Graph it explicitly.
Session scope decides what the pool size means A session opened for the whole request holds a connection through serialisation and external calls, so pool size caps concurrent requests. A session scoped to the database work caps concurrent queries instead. The same pool size supports very different concurrency session spans the request connection held during rendering held during external HTTP calls pool size caps concurrent requests pool of 10 serves 10 requests session scoped to queries connection held only while querying released before slow work pool size caps concurrent queries pool of 10 serves far more requests Tightening session scope often removes the need to enlarge the pool at all, and costs nothing at the database. It is also the change that makes pool sizing predictable, because hold time stops depending on response size.
Pool size means different things depending on how long a session holds its connection.

Operational boundary

These parameters cap connection count. They do nothing about how long a session holds a connection, and in SQLAlchemy that is usually the larger lever. A session opened at the start of a request and closed at the end holds a connection through template rendering, serialisation, and any external call — so a pool of 10 supports 10 concurrent requests rather than 10 concurrent queries. Scoping sessions tightly around database work often removes the need to enlarge the pool at all.

They also cannot help across processes. If your process count is large, per-process sizing eventually cannot fit inside the database ceiling at any value, and a proxy is required. NullPool combined with an external pooler is a legitimate configuration in that topology — and the correct one for serverless, where the process may be frozen between invocations, as covered in Serverless Function Connection Management.

Frequently Asked Questions

Is pool_size a minimum or a maximum?
Neither exactly. It is the number of connections the pool will retain. Connections are created lazily up to that number and kept when returned; beyond it, overflow connections are created and discarded.
What is the real maximum for one engine?
pool_size + max_overflow. Past that, checkouts wait pool_timeout seconds and then raise.
Why is my service slow with no pool errors?
Check pool.overflow(). If it is persistently positive, every request past pool_size is paying a full connection handshake, which adds latency without ever raising an exception.
Should I enable pool_pre_ping?
Yes for most deployments. It costs a trivial round trip and eliminates the class of errors caused by connections severed while idle. Pair it with pool_recycle rather than choosing between them.
How does this differ for async engines?
The parameters are identical. The additional rules are that the engine must be created in the event loop that uses it, disposed on shutdown, and never shared across a fork().
What should I use in serverless?
NullPool, or a very small pool with an external pooler. A retained pool inside a function that freezes between invocations holds connections it cannot use.
Does max_overflow count toward the database ceiling?
Yes, whenever those connections exist. Budget the fleet as processes x (pool_size + max_overflow), because that is the peak the database will actually see.