Limiting Database Connections from Queue-Triggered Lambda Functions

This guide is part of Serverless Function Connection Management. An API-triggered function scales with user traffic, which usually ramps. A queue-triggered function scales with backlog, which can appear all at once: a downstream outage ends, a batch producer dumps a day of events, or a replay job re-enqueues a million messages. Lambda’s SQS poller then adds concurrent invocations as fast as it is allowed, every new execution environment opens its own database connection, and the database that was comfortably serving the steady trickle is flooded:

2026-09-10T02:14:07Z ERROR  consumer  error: sorry, too many clients already
    at Parser.parseErrorMessage (node_modules/pg-protocol/dist/parser.js:283:98)
  code: '53300'
2026-09-10T02:14:09Z WARN   sqs  ApproximateNumberOfMessagesVisible=412,880  ConcurrentExecutions=610

The obvious fix — reserved concurrency on the function — caps connections but makes Lambda throttle the poller, and throttled messages go back to the queue, count towards maxReceiveCount, and end up in the dead-letter queue without ever being processed. The objective is to cap concurrency at the event source instead, size it from the database’s connection budget, and keep batches small enough that each invocation holds its connection briefly.

Rapid incident diagnosis

Correlate concurrency with backlog. Pull both series for the incident window. A step in ConcurrentExecutions that follows a jump in queue depth confirms backlog-driven scale-out:

aws cloudwatch get-metric-data --start-time 2026-09-10T01:30:00Z --end-time 2026-09-10T03:00:00Z \
  --metric-data-queries '[
    {"Id":"c","MetricStat":{"Metric":{"Namespace":"AWS/Lambda","MetricName":"ConcurrentExecutions","Dimensions":[{"Name":"FunctionName","Value":"orders-consumer"}]},"Period":60,"Stat":"Maximum"}},
    {"Id":"q","MetricStat":{"Metric":{"Namespace":"AWS/SQS","MetricName":"ApproximateNumberOfMessagesVisible","Dimensions":[{"Name":"QueueName","Value":"orders-events"}]},"Period":60,"Stat":"Maximum"}},
    {"Id":"t","MetricStat":{"Metric":{"Namespace":"AWS/Lambda","MetricName":"Throttles","Dimensions":[{"Name":"FunctionName","Value":"orders-consumer"}]},"Period":60,"Stat":"Sum"}}
  ]'

Read the event source mapping. The settings that bound scale-out live on the mapping, not on the function:

aws lambda list-event-source-mappings --function-name orders-consumer \
  --query "EventSourceMappings[].{src:EventSourceArn,batch:BatchSize,window:MaximumBatchingWindowInSeconds,max:ScalingConfig.MaximumConcurrency,par:ParallelizationFactor,fr:FunctionResponseTypes}"
  • SQS mapping with no MaximumConcurrency: concurrency is bounded only by account and function limits.
  • Kinesis or DynamoDB Streams mapping: concurrency is shards × ParallelizationFactor, so a resharding event or a raised factor changes it.
  • Throttles above zero with reserved concurrency set: messages are being returned to the queue unprocessed. Check the dead-letter queue depth next.

Count connections per invocation. If the handler creates a pool with a default size, each environment may open more than one connection while processing a batch in parallel. PostgreSQL shows it directly:

SELECT client_addr, count(*) FROM pg_stat_activity
WHERE application_name = 'orders-consumer'
GROUP BY client_addr ORDER BY count(*) DESC LIMIT 10;

More than one connection per client address means per-environment pools larger than 1.

Concurrent Lambda executions during a queue backlog After a backlog of 400,000 messages appears at 02:00, concurrent executions of the queue consumer rise steadily and cross the database's usable connection budget of 180 within minutes, reaching over 600. ConcurrentExecutions queue depth (scaled) usable connections 180 01:50 backlog arrives 02:40 610 concurrent: too many clients
A backlog, not user traffic, drives the scale-out. Concurrency climbs until connections exceed the database ceiling long before the queue drains.

Mathematical sizing or parameter formula

Connection budget to concurrency. With one connection per execution environment:

MaximumConcurrency = floor( usable_connections_for_consumer ÷ connections_per_environment )

If the database has 180 connections available to this consumer and each environment opens exactly one, MaximumConcurrency = 180. If a proxy sits in front of the database, the budget is the proxy’s client limit and the backend pool is sized separately from transaction time.

Throughput at that cap. Check that the cap still drains the backlog in acceptable time:

drain_rate       = MaximumConcurrency × batch_size ÷ seconds_per_invocation
time_to_drain    = backlog ÷ (drain_rate − arrival_rate)

With 180 concurrent invocations, batches of 10 and 1.2 seconds per invocation, the drain rate is 180 × 10 ÷ 1.2 = 1,500 messages per second. A 400,000-message backlog with 200 messages per second still arriving drains in 400000 ÷ 1300 ≈ 308 seconds — about five minutes, with the database never above its budget.

Hold time per connection. Larger batches reduce invocations but lengthen each one. If a batch is written in one transaction, the transaction and its locks last the whole batch. Keep:

batch_size × per_message_db_seconds  ≤  a few hundred milliseconds

Stream sources. For Kinesis and DynamoDB Streams, concurrency is structural:

concurrency = open_shards × ParallelizationFactor

Twenty shards with a factor of 10 is 200 concurrent invocations. Resharding to 40 doubles it without any change to the function.

Reserved concurrency versus event source maximum concurrency With reserved concurrency of 180, the SQS poller keeps invoking beyond 180, invocations are throttled, messages return to the queue and after maxReceiveCount move to the dead-letter queue. With MaximumConcurrency of 180 on the mapping, the poller never exceeds 180 invocations and messages wait in the queue. Reserved concurrency 180 SQS poller scales past 180 Throttled invocations above 180 Messages return receive count rises dead-letter queue MaximumConcurrency 180 on mapping SQS poller stops at 180 180 running no throttles Remaining messages wait in the queue, receive count unchanged
Reserved concurrency caps connections by throttling, which returns messages to the queue and drives them towards the dead-letter queue. Maximum concurrency on the mapping caps polling instead, so nothing is throttled.

Exact remediation & configuration

Cap concurrency on the SQS mapping. MaximumConcurrency accepts values from 2 to 1,000 and applies per event source mapping:

aws lambda update-event-source-mapping --uuid 3f2c9e1a-8b7d-4c1e-9a55-0d6e2b1f7c40 \
  --scaling-config MaximumConcurrency=180 \
  --batch-size 10 --maximum-batching-window-in-seconds 2 \
  --function-response-types ReportBatchItemFailures

In infrastructure as code, keep the value next to the database budget it is derived from:

# AWS SAM
OrdersConsumer:
  Type: AWS::Serverless::Function
  Properties:
    Handler: consumer.handler
    Runtime: nodejs20.x
    Timeout: 30
    Events:
      Orders:
        Type: SQS
        Properties:
          Queue: !GetAtt OrdersQueue.Arn
          BatchSize: 10
          MaximumBatchingWindowInSeconds: 2
          ScalingConfig:
            MaximumConcurrency: 180          # = usable connections for this consumer
          FunctionResponseTypes: [ReportBatchItemFailures]

If several mappings invoke the same function, their maximums add up; the sum must fit the budget.

Remove reserved concurrency used only as a connection cap, or set it above MaximumConcurrency so it never throttles the mapping. Check the queue’s redrive policy: maxReceiveCount should comfortably exceed the number of times a message can be retried for genuine errors.

One connection per environment, created outside the handler. Process the batch sequentially on that connection, and report per-message failures instead of failing the whole batch:

import pg from "pg";

const pool = new pg.Pool({
  connectionString: process.env.DATABASE_URL,
  max: 1,                              // one connection per execution environment
  idleTimeoutMillis: 60_000,
  connectionTimeoutMillis: 3_000,
  application_name: "orders-consumer",
});

export const handler = async (event) => {
  const batchItemFailures = [];
  for (const record of event.Records) {
    try {
      const msg = JSON.parse(record.body);
      await pool.query(
        "INSERT INTO order_events (id, order_id, type, payload) VALUES ($1, $2, $3, $4) ON CONFLICT (id) DO NOTHING",
        [msg.id, msg.orderId, msg.type, msg],
      );
    } catch (err) {
      batchItemFailures.push({ itemIdentifier: record.messageId });
    }
  }
  return { batchItemFailures };
};

ON CONFLICT DO NOTHING makes redelivery harmless, which matters because the cap increases the time messages spend queued and visibility timeouts can expire.

For stream sources, lower ParallelizationFactor rather than the shard count, since shards are sized for the producer:

aws lambda update-event-source-mapping --uuid 7a1d0c55-2e4b-4f0a-b1c9-6e8d3a2f9b10 --parallelization-factor 4

Put a proxy in front when the budget is too small for acceptable drain times. With RDS Proxy or PgBouncer in transaction mode, MaximumConcurrency bounds proxy clients, and backend connections follow transaction time instead, as discussed in Choosing a Connection Proxy for Serverless Postgres.

Backlog drain with maximum concurrency 180 A backlog of 400,000 messages drains at a net 1,300 messages per second, reaching zero after about 308 seconds, while connections stay flat at 180. queue depth database connections (flat at 180) drained at ~308 s 0 s 450 s drain = 180 × 10 messages ÷ 1.2 s = 1,500/s, minus 200/s still arriving
The drain time at the capped concurrency is the number to check before choosing the cap. Here the backlog clears in about five minutes without exceeding the connection budget.

Validation & verification

Replay a backlog in staging. With the mapping disabled, enqueue a large burst, then enable the mapping and watch concurrency and connections together:

import json, os, uuid, boto3
from concurrent.futures import ThreadPoolExecutor

sqs = boto3.client("sqs")
queue_url = os.environ["QUEUE_URL"]

def send_batch(_):
    entries = [{"Id": str(i), "MessageBody": json.dumps({"id": str(uuid.uuid4()), "orderId": i, "type": "replay"})}
               for i in range(10)]
    sqs.send_message_batch(QueueUrl=queue_url, Entries=entries)

with ThreadPoolExecutor(max_workers=16) as pool:
    list(pool.map(send_batch, range(20_000)))          # 200,000 messages

During the drain, ConcurrentExecutions must plateau at MaximumConcurrency, Throttles must stay at zero, and the database session count for orders-consumer must stay at or below the cap:

SELECT count(*) FROM pg_stat_activity WHERE application_name = 'orders-consumer';

Nothing reaches the dead-letter queue that did not fail for a genuine reason: its ApproximateNumberOfMessagesVisible should be unchanged after the replay.

Drain time matches the formula. Record the time from peak depth to zero and compare it with backlog ÷ (drain_rate − arrival_rate). A much longer drain means invocations take longer than assumed — usually database latency rising under the capped load — and the batch size or query cost needs attention before raising the cap.

Idempotency. Force a partial failure in one batch and confirm only the failed messages are redelivered, and that redelivered messages do not create duplicate rows.

Protecting the database from retries and poison messages

Capping concurrency bounds how many connections the consumer can open at once. It does not bound how much work retries create, and a batch that fails repeatedly consumes connections and database time on every attempt.

Partial batch responses stop whole-batch retries. Without ReportBatchItemFailures, one failing message fails the invocation and all ten messages in the batch return to the queue. Each retry redoes nine successful writes. With it, only the failed message returns. That is the difference between a poison message costing one connection-hold per retry and costing ten.

Bound retries with the queue’s redrive policy. Set maxReceiveCount so a message that fails consistently reaches the dead-letter queue after a few attempts rather than cycling for the full retention period:

aws sqs set-queue-attributes --queue-url "$QUEUE_URL" --attributes '{
  "RedrivePolicy": "{\"deadLetterTargetArn\":\"arn:aws:sqs:eu-west-1:123456789012:orders-events-dlq\",\"maxReceiveCount\":\"5\"}",
  "VisibilityTimeout": "180"
}'

Set the visibility timeout to at least six times the function timeout, as AWS recommends for SQS event sources, so a message is not redelivered while a slow invocation is still processing it — a duplicate delivery doubles its connection use.

Distinguish database back-pressure from bad messages. When the database is overloaded, every message fails, and sending them all to the dead-letter queue loses work. Detect connection and timeout errors and fail the whole batch in that case, so messages return to the queue with their receive count increasing slowly, while genuine data errors are reported per item:

const TRANSIENT = new Set(["53300", "57P01", "08006", "08001"]);   // too many clients, admin shutdown, connection failures

export const handler = async (event) => {
  const batchItemFailures = [];
  for (const record of event.Records) {
    try {
      await processRecord(record);
    } catch (err) {
      if (TRANSIENT.has(err.code) || err.message?.includes("timeout")) {
        throw err;                                       // whole batch retries later
      }
      batchItemFailures.push({ itemIdentifier: record.messageId });
    }
  }
  return { batchItemFailures };
};

Throwing on transient errors also slows the poller: Lambda reduces concurrency for SQS event sources when invocations error, which is a useful form of back-pressure while the database recovers.

Failure type Handling Effect on connections
Bad message data Report item failure; DLQ after maxReceiveCount One short retry per attempt
Database overloaded or connection refused Throw; whole batch retried later Poller backs off, fewer concurrent connections
Slow invocation beyond visibility timeout Raise visibility timeout Prevents duplicate processing

Frequently Asked Questions

Why not rely on the pool’s connection timeout to shed load?
A connection timeout fails the invocation after the database is already saturated, the message is retried, and the retries keep concurrency high. Capping concurrency before invocations start prevents the saturation instead of reacting to it.
Does MaximumConcurrency work with FIFO queues?
FIFO queue scaling is bounded by the number of active message groups, since messages in one group are processed in order. MaximumConcurrency still applies as an upper bound, which is useful when a producer creates many groups at once.
How does this combine with provisioned concurrency?
Provisioned concurrency keeps environments initialised to reduce cold starts; it does not change how many invocations the poller runs. Connections are opened by whichever environments are running, so the cap on the mapping still governs connection count.
Should the consumer use more than one connection per environment for large batches?
Rarely. Parallel writes within one invocation multiply connections by the batch parallelism, and the same throughput is usually achieved more safely by raising MaximumConcurrency by the same factor within the budget, or by writing the batch in a single multi-row statement.
Does MaximumConcurrency apply per queue or per function?
Per event source mapping. A function with two SQS mappings, each capped at 100, can run 200 concurrent invocations and open 200 connections. Sum the caps across mappings when comparing with the database budget.