Every call to an external API fails sooner or later: the network drops, the other system goes into maintenance, the quota fills up or the data you send is rejected. In the field, the error that does the most damage is often not the initial failure but the reaction to it: a permanent error is retried forever, hundreds of requests per second go to a system that has crashed, a timed-out POST is blindly repeated and a duplicate order is created. For calls to systems such as ERP, CRM, payments and Microsoft 365, the retry, backoff, rate limit, timeout and circuit breaker policy should be written down from the start.
Separating temporary and permanent errors
The first rule: only temporary errors are retried. A temporary error is one where the same request, sent unchanged after a while, has a chance of succeeding: a connection timeout, a DNS resolution error, the other system being overloaded, a quota being exceeded. A permanent error is one whose result won’t change unless the request changes: an invalid field, unauthorized access, a record not found.
Retrying a permanent error does two kinds of harm. First, you notice the real problem (a missing field mapping, an expired certificate) late; the log fills with thousands of repeats of the same error and the real line gets lost. Second, in a bulk transfer, each of thousands of records being tried several times wastes the other system’s quota, and healthy requests are affected too. A single bad record locks up the whole flow.
The second rule: the operation being retried must be idempotent. RFC 9110 defines the GET, PUT and DELETE methods as idempotent; POST isn’t. A POST that timed out may have been processed on the other side. Don’t assume the operation didn’t happen because the client didn’t get a response. POST calls that create orders or payments should be retried automatically only if the other API supports an idempotency key (an Idempotency-Key header at many providers) or a duplicate check by external reference number.
Retry table by HTTP status code
| Status | Meaning | Retry? |
|---|---|---|
| Connection error, timeout | No response received | Yes; for POST only with an idempotency key |
| 400, 422 | Request invalid | No; fix the data |
| 401 | Authentication failed | Once after refreshing the token; then no |
| 403, 404 | No permission / no resource | No |
| 408 | Server timed out waiting for the request | Yes |
| 409 | Conflict | Usually no; read the state and decide |
| 429 | Too many requests | Yes; respect the Retry-After value |
| 500 | Internal server error | A limited number of times |
| 502, 503, 504 | Gateway / service unavailable | Yes, with backoff |
This table is a default, not a contract. Some APIs return 500 on validation errors, and some report quota with 403. For every API you integrate with, read the documentation, configure the table for that API and include the error code in the response body in the classification too. The reason 500 is retried only a limited number of times is that this code can indicate both a temporary failure and a software bug that a specific request triggers every time. If the same record gets a 500 every time, the problem isn’t on the other side but in that record.
Exponential backoff, jitter and Retry-After
Retrying immediately and at fixed intervals after an error reloads the troubled system without giving it a chance to recover. Exponential backoff increases the wait time multiplicatively with each attempt. It isn’t enough on its own: if hundreds of clients that got an error at the same time retry at the same intervals, the load comes back in waves. Jitter breaks up this synchronization by adding randomness to the wait time. In the common “full jitter” approach, the wait is chosen randomly between zero and the exponential upper limit.
BASE = 1.0 # seconds
CAP = 30.0 # upper limit of a single wait
MAX_RETRIES = 5
for attempt in range(MAX_RETRIES + 1):
response = call(request, timeout=remaining_time_budget())
if response.ok:
return response
if not is_retryable(response) or attempt == MAX_RETRIES:
raise PermanentError(response)
if response.retry_after is not None: # 429 / 503
wait = response.retry_after
else:
wait = random.uniform(0, min(CAP, BASE * 2 ** attempt))
if wait > remaining_time_budget():
raise TimeBudgetExceeded()
sleep(wait)
An example calculation: With a base of 1 second, the upper limits of the waits before five retries are 1, 2, 4, 8 and 16 seconds respectively; the total wait is 31 seconds in the worst case, and more once the timeouts of the six calls including the first are added. The 30-second cap doesn’t come into play with this setting; if the number of attempts is increased, every wait from the sixth retry on is capped at 30 seconds. For a call running inside a user request, this is far too long. For background synchronization, it’s reasonable. Set the parameters separately according to the call’s context; a single global setting breaks both sides.
The Retry-After header is defined in RFC 9110 and can carry a number of seconds or an HTTP date. The client must parse both forms and not wait less than the server suggests. When this header arrives, use the server’s value instead of your own backoff calculation; this avoids unnecessary rejections in quota windows. A mistake we often see in the field is a client that parses only the number form waiting zero seconds on the date form and retrying immediately.
Rate limits and quota management
The 429 (Too Many Requests) status code is defined in RFC 6585. Instead of only reacting to quota errors, manage the quota on the client side in advance:
- Learn the quota from the documentation and limit it in the client. Keep the number of calls per second or minute below the quota with a rate limiter such as a token bucket.
- Read the remaining quota headers. Many APIs report the number of remaining requests and the time the window resets with headers like
X-RateLimit-Remaining. These headers aren’t standard; their names and meanings vary by API. - Consider all processes sharing the quota together. A nightly transfer, a webhook worker and the website using the same API key consume the same quota. The nightly product sync uses up the quota, and in the morning orders get 429s. Give priority to critical flows (orders) and low priority to bulk flows (product sync).
- Prefer bulk endpoints. If the API offers a batch endpoint that updates several records in a single call, use it instead of one call per record.
Microsoft Graph also returns 429 and Retry-After when throttling; in Graph API integrations, respecting this header is the basic way to keep throttling from lasting longer. A client that ignores the header and insists waits longer.
Timeout budget and circuit breaker
An HTTP client without a defined timeout can wait a very long time for a server that doesn’t respond; many libraries’ defaults are either very long or nonexistent. This is the first place to look in a code review. Set two separate values: a short connect timeout (TCP/TLS setup) and a read timeout based on the API’s normal response time.
Alongside the individual timeouts, define a total time budget. If a web request the user is waiting on has, say, a 3-second budget and the first attempt took 2.5 seconds, a second attempt shouldn’t be made, or should be limited to the remaining time. The remaining_time_budget() function in the pseudo-code above is for this.
When the other system has completely failed, every request waiting for its timeout exhausts your own application’s threads and connection pool. That’s how a failure spreads. A circuit breaker manages this situation with a three-state mechanism:
| State | Behavior | Transition condition |
|---|---|---|
| Closed | Calls are made normally, errors are counted | Open if the error rate threshold is exceeded |
| Open | An error is returned immediately without making the call | Half-open when the wait period ends |
| Half-open | A limited number of trial calls are allowed | Closed if successful, otherwise open |
While the breaker is open, offer the user a meaningful fallback behavior: such as showing the last cached price with a “to be confirmed” note, or queueing the order and leaving it in “processing” status. A detailed description of the pattern is in Microsoft’s Circuit Breaker pattern document. When retry and circuit breaker are used together, the retry loop should stop trying while the breaker is open; otherwise the two mechanisms undermine each other.
Dead-letter for failed messages
In background integrations, a message whose retries are exhausted isn’t silently deleted. A dead-letter queue (DLQ) is a separate area where messages that couldn’t be processed wait together with the reason for the error. Brokers such as RabbitMQ, Azure Service Bus and Amazon SQS offer this concept built in; in simple queues working with a database table, a status = 'dead' field does the same job. A deleted message doesn’t come back. A message in the dead-letter queue gets fixed and replayed.
Checklist for dead-letter management:
- Are the last error code, error message, attempt count and first/last attempt time stored with the message?
- Do permanent errors (400, 422) go straight to dead-letter without waiting for retries?
- Is an alert defined for the number of messages in dead-letter, and is its owner clear?
- Is there a documented way to send messages back to the main queue (replay) once the problem is fixed?
- Has it been verified that replayed messages are processed idempotently?
The size and age of the dead-letter queue are among the main metrics showing an integration’s health; alert design for these metrics is explained in the integration monitoring and reconciliation guide. For the equivalent of the same principles on the incoming traffic side, see the webhook receiver design article; for which flow should be synchronous and which queued, see ERP–web integration patterns. If you’d like to review the error policy of your existing integrations, you can find more on our ERP, CRM and API integrations page.
Frequently asked questions
How many times should a request be retried?
There’s no universal number; what decides is the total time budget. For requests the user is waiting on, one or two short attempts are enough. Background transfers can retry more, but when the attempts run out the message isn’t deleted; it’s moved to a dead-letter area.
Is it safe to resend a POST request that timed out?
Only if the other API supports an idempotency key or a duplicate check by external reference number. Otherwise the first request may have been processed on the other side, and resending creates a duplicate order or payment. In that case, first query whether the record exists.
Why is jitter needed; isn’t exponential backoff enough?
If many clients that got an error at the same time retry at the same exponential intervals, the requests pile up at the same moments again and load the recovering system in waves. Jitter spreads these requests over time by adding randomness to the wait time.
How should circuit breaker and retry work together?
Retry tolerates a single request’s temporary error; a circuit breaker stops calls entirely when the other system has generally failed. The retry loop should check the breaker’s state before every attempt; if the breaker is open, it should return an error without waiting and switch to fallback behavior.