Configuring Retry Policies for Azure SQL Transient Faults

This guide is part of Azure SQL Connection Management. Azure SQL Database is a multi-tenant service that will close your connections on purpose: during a planned failover, when the service reconfigures, when a database is moved between nodes, and when you exceed a resource governor limit. These are not faults in your application, and the platform’s documented expectation is that clients retry.

Without a retry policy, every one of those events becomes a user-visible error. With a badly designed one, they become duplicate orders. The difference is entirely in which errors you retry and whether the operation is safe to repeat.

The errors worth retrying

Azure SQL has a documented set of transient error numbers. The important ones:

Number Meaning Retry
40613 Database is not currently available (reconfiguration) Yes, with backoff
40501 Service is busy — engine throttling Yes, with longer backoff
40197 Error processing the request; service reconfiguring Yes
49918, 49919, 49920 Cannot process request, not enough resources Yes, with longer backoff
4060 Cannot open database (may be transient during failover) Yes, limited attempts
10928 Resource ID limit reached — session or worker limit Yes, but investigate; this is usually a pool sizing bug
10929 Resource ID minimum guarantee exceeded Yes, with backoff
10053, 10054, 10060 Transport-level errors, connection reset or timeout Yes
233 Connection initialization failure Yes
64 Connection failed during login Yes
18456 Login failed No — credentials, not transience
2627, 2601 Primary key or unique constraint violation No — data, not transience
1205 Deadlock victim Yes, but only for idempotent work

The two rows marked no are the ones a naive catch (SqlException) { retry(); } gets wrong. Retrying a login failure hammers the server with credentials that will never work, and retrying a constraint violation cannot succeed.

10928 deserves particular attention because it is often misclassified. It means you have hit the tier’s session or worker limit — a connection sizing problem wearing a transient error’s clothing. Retrying works, but the real fix is a smaller pool per instance or a smaller fleet, worked out from the tier’s documented limits.

Not every SqlException is transient An exception is classified into three groups: transient platform errors which should be retried with backoff, resource limit errors which may be retried but indicate a sizing problem, and permanent errors such as login failures and constraint violations which must not be retried. Retry the platform, never the data transient platform 40613, 40197, 40501 10053, 10054, 233 retry with exponential backoff and jitter, up to a small count expected during planned failover resource limits 10928, 10929 49918, 49919, 49920 retry, but treat as a signal that the pool or tier is wrong retrying alone hides a sizing bug permanent 18456 login failed 2627, 2601 constraint never retry — it cannot succeed and it wastes the deadline surface these to the caller A blanket catch on SqlException collapses all three columns into one, which is why the default hand-rolled retry loop retries logins that will never succeed.
Three classes of error, three different responses. Only the first is what retry logic is for.

Using SqlClient’s built-in retry provider

Microsoft.Data.SqlClient 3.0 and later ships a configurable retry provider, which is preferable to a hand-rolled loop because it sits below your code and covers connection opening as well as command execution:

var connectionOptions = new SqlRetryLogicOption
{
    NumberOfTries   = 5,
    DeltaTime       = TimeSpan.FromSeconds(1),
    MaxTimeInterval = TimeSpan.FromSeconds(20),
    // Defaults cover the documented transient list; add tier-specific ones.
    TransientErrors = new[] { 40613, 40501, 40197, 49918, 49919, 49920,
                              10928, 10929, 10053, 10054, 10060, 233, 64, 4060 },
};

var provider = SqlConfigurableRetryFactory
    .CreateExponentialRetryProvider(connectionOptions);

await using var conn = new SqlConnection(connectionString);
conn.RetryLogicProvider = provider;   // retries Open()
await conn.OpenAsync(ct);

Command execution needs its own provider, and it should be more conservative — a retried command may have already run:

var commandOptions = new SqlRetryLogicOption
{
    NumberOfTries   = 3,
    DeltaTime       = TimeSpan.FromSeconds(1),
    MaxTimeInterval = TimeSpan.FromSeconds(10),
    // Only retry SELECTs automatically; writes need idempotency.
    AuthorizedSqlCondition = cmd =>
        cmd is not null &&
        Regex.IsMatch(cmd.CommandText, @"^\s*SELECT", RegexOptions.IgnoreCase),
};

cmd.RetryLogicProvider =
    SqlConfigurableRetryFactory.CreateExponentialRetryProvider(commandOptions);

AuthorizedSqlCondition is the safety valve, and it is the part most implementations skip. Without it, a transport-level error on a command that already committed causes the retry to run it again — and a transport error is exactly the case where the client does not know whether the server applied the change.

For Entity Framework Core, the equivalent is the built-in execution strategy:

services.AddDbContext<OrdersContext>(options =>
    options.UseSqlServer(connectionString, sql =>
    {
        sql.EnableRetryOnFailure(
            maxRetryCount: 5,
            maxRetryDelay: TimeSpan.FromSeconds(20),
            errorNumbersToAdd: new[] { 10928, 10929 });
        sql.CommandTimeout(15);
    }));

EnableRetryOnFailure has a documented restriction worth internalising: with a retrying execution strategy, user-initiated transactions must be wrapped in strategy.ExecuteAsync(...), because the strategy needs to be able to replay the whole transaction rather than a single command mid-transaction. EF throws a clear exception if you forget, which is one of the friendlier failure modes in this area.

Backoff, jitter, and the deadline

Three properties make a retry policy safe rather than harmful.

Exponential backoff. Immediate retries against a service that is reconfiguring add load at the worst moment. Start at roughly one second and double.

Jitter. Without it, an entire fleet retries in lockstep and produces a synchronised thundering herd every backoff interval. The CreateExponentialRetryProvider factory applies jitter; a hand-rolled loop usually does not.

A total deadline. Five retries at exponential backoff can consume 30 seconds or more. If the caller’s deadline is 10 seconds, the later retries are pure waste — they run against a request nobody is waiting for any more. Bound the whole operation with a CancellationToken and pass it everywhere:

using var cts = CancellationTokenSource.CreateLinkedTokenSource(httpContext.RequestAborted);
cts.CancelAfter(TimeSpan.FromSeconds(10));
await using var conn = new SqlConnection(cs) { RetryLogicProvider = provider };
await conn.OpenAsync(cts.Token);

Retries interact with the connection pool too, and not favourably: each attempt holds a pooled connection for the length of its wait. A fleet retrying with long backoff can saturate its own pool without a single query executing. Keep the retry count small and the deadline short, and pair the policy with a circuit breaker so a sustained platform outage sheds load instead of queueing — the pattern is in implementing circuit breakers for connection acquisition failures.

Retries spend the deadline and hold a connection A request deadline bar with five retry attempts and their backoff intervals drawn inside it. The last two attempts fall outside the deadline and are wasted, while every attempt holds a pooled connection for the length of its wait. Every attempt spends deadline and occupies a pooled connection caller deadline — 10 s nobody is waiting any more try 1 try 2 after 1 s try 3 after 2 s try 4 after 4 s try 5 after 8 s — past the deadline pool impact one pooled connection held for the whole sequence, not just during the attempts A fleet retrying with long backoff can saturate its own pool without executing a single query. Bound the sequence with a linked cancellation token so attempts stop when the deadline does.
Retries are not free: they consume the caller's deadline and hold a pooled connection for the whole backoff sequence.

Idempotency for writes

Retrying a read is always safe. Retrying a write is safe only if repeating it has the same effect as doing it once, and a transport-level error gives no information about whether the server committed.

The reliable approach is an idempotency key carried by the operation and enforced by a unique constraint:

CREATE TABLE order_submissions (
    idempotency_key uniqueidentifier NOT NULL PRIMARY KEY,
    order_id        bigint           NOT NULL,
    created_at      datetime2        NOT NULL DEFAULT SYSUTCDATETIME()
);
try
{
    await using var tx = await conn.BeginTransactionAsync(ct);
    await InsertOrderAsync(conn, tx, order, ct);
    await RecordSubmissionAsync(conn, tx, request.IdempotencyKey, order.Id, ct);
    await tx.CommitAsync(ct);
}
catch (SqlException e) when (e.Number is 2627 or 2601)
{
    // The key already exists: a previous attempt committed. Return that result.
    return await LookupBySubmissionKeyAsync(conn, request.IdempotencyKey, ct);
}

The constraint violation becomes the signal that the first attempt succeeded, which turns an unsafe retry into a safe one. Note the pattern deliberately catches 2627/2601 here even though the classification table above says never retry them — the distinction is that we are not retrying, we are interpreting the violation as evidence of a prior success.

Turning an ambiguous write into a safe retry The first attempt commits but the response is lost to a transport error, so the client does not know the outcome. The retry inserts the same idempotency key, hits the unique constraint, and reads back the result of the first attempt instead of duplicating the work. A transport error never tells you whether the server committed attempt 1 insert order + idempotency key COMMIT succeeds on the server transport error 10054 response never reaches the client outcome unknown to the caller retry fires same idempotency key as attempt 1 without an idempotency key the retry inserts a second order the customer is charged twice with an idempotency key the unique constraint rejects the duplicate and the first attempt's result is returned The constraint violation is not an error here — it is the evidence that the earlier attempt committed. That is what makes retrying a write safe rather than merely convenient.
The unique constraint turns an ambiguous outcome into a definite one, which is what makes the retry safe.

Common failure patterns and remediation

Symptom Underlying cause Remediation
Errors surface to users on every Azure maintenance window No retry policy at all Enable the SqlClient retry provider or EnableRetryOnFailure
Duplicate records after a network blip Writes retried without idempotency Add an idempotency key with a unique constraint
Login failures retried repeatedly Blanket catch (SqlException) Classify by Number; never retry 18456
10928 under normal load Session or worker limit for the tier reached Reduce pool size per instance, or scale the tier
Retries exhaust the pool Each attempt holds a connection while backing off Lower retry count and total deadline; add a circuit breaker
Whole fleet retries in lockstep No jitter in a hand-rolled loop Use the built-in exponential provider
EF throws about user-initiated transactions Explicit transaction with a retrying strategy Wrap the transaction in strategy.ExecuteAsync
Retries continue past the request deadline No linked cancellation token Bound the whole operation with CancelAfter

Connection string settings that reduce the need

Several settings shorten the window in which transient errors can occur at all:

Server=tcp:orders.database.windows.net,1433;
Database=orders;
Encrypt=True;
TrustServerCertificate=False;
Connect Timeout=15;
Max Pool Size=40;
Min Pool Size=2;
Connection Lifetime=280;
ConnectRetryCount=3;
ConnectRetryInterval=10;
Application Name=orders-api;

ConnectRetryCount and ConnectRetryInterval enable SqlClient’s idle connection resiliency, which transparently re-establishes a broken idle connection before your code sees an error. It is distinct from the retry provider and complements it.

Connection Lifetime caps connection age so connections are re-established periodically and follow any topology change — the same reasoning as everywhere else, and covered generally in tuning HikariCP maxLifetime and idleTimeout.

Application Name costs nothing and makes sys.dm_exec_sessions attributable during an incident, which is the difference between naming the offending service in a minute and in an hour.

Operational boundary

A retry policy converts transient platform events into latency. It does not convert a sustained outage into availability, and it does not fix a pool that is too large for the tier’s session limit. If 10928 appears regularly, the correct action is to reduce connections per instance or move to a tier with a higher limit — retrying is a way of tolerating that condition, not of resolving it.

Retries also do not help with a saturated pool. When every pooled connection is checked out, a retry queues behind the same wait; if anything it makes the queue longer. Bound the pool wait separately from the retry policy, and treat sustained saturation as a capacity problem.

Frequently Asked Questions

Which errors should I never retry?
Login failures (18456), constraint violations (2627, 2601), permission errors, and syntax errors. None of them can succeed on a second attempt.
Is SqlClient’s retry provider enough, or do I need Polly?
The built-in provider covers connection opening and command execution with exponential backoff and jitter, and is enough for most services. Polly is useful when you want retries composed with a circuit breaker and a bulkhead in one policy.
Why does EF Core complain about my transaction?
A retrying execution strategy must be able to replay the entire transaction, not a single command inside it. Wrap the transaction in strategy.ExecuteAsync(...).
How many retries is right?
Three to five, with exponential backoff bounded by a total deadline shorter than the caller’s. More attempts mostly consume a deadline that has already expired.
What is error 10928 telling me?
You have reached the session or worker limit for your service tier. Retrying works, but the underlying cause is usually too many connections per instance across the fleet.
Do retries make writes unsafe?
They can. A transport error gives no information about whether the server committed. Use an idempotency key enforced by a unique constraint so a repeat is detectable.
Should retries and circuit breakers coexist?
Yes, and they must be designed together. Retry inside a closed breaker; treat an open breaker as non-retryable so a sustained outage sheds load instead of queueing.