By the end of the lesson, you can explain the difference between timeout and deadline, say when it is safe to repeat a call, and how idempotency makes repetition safe.
Why is it needed?
The outcome of a remote call can be unknown (6.1). From the point of view of fault tolerance the question is how long to wait, when to repeat, and how to preserve the correctness of the business state when the outcome is uncertain. Repetition is the most common defence and also the most common source of errors.
Timeout and deadline
A timeout is a duration of waiting. A deadline is a point in time: until when the caller will still wait for the response. According to the gRPC documentation a timeout is the maximum duration of the call, and it becomes a deadline by adding it to the start time of the call. gRPC sets no deadline by default, so a realistic deadline must always be set explicitly in the client.
In chained calls the deadline must be passed on: if the client's time limit is finite, an inner call cannot get the whole limit again. gRPC forwards the remaining time, not the absolute time, so there is no clock-skew problem. After expiry the call may be cancelled on the server side, but the application must stop the work it started; cancellation does not undo a change that has already been carried out.
Which failure is retried?
A transient network failure or overload may improve later. Invalid input, missing authorisation and a business rejection do not improve with repetition. Automatic repetition should happen only for the agreed class of errors, in a limited number, and within the time limit.
According to RFC 9110 a method is idempotent if the intended effect of several identical requests is the same as that of one; in the standard PUT, DELETE and the safe methods are such. An idempotent request can be repeated automatically if a connection failure occurs before the response is read. A client should not automatically retry a request with a non-idempotent method unless it knows that the semantics of the request are in fact idempotent, or has some means to detect that the original request was never applied.
Idempotency key
For a modifying request the client can send the same idempotency key for the same logical operation. The server stores the key, the meaning of the request and the result durably; on repetition it does not execute it again but returns the earlier result. The AWS description also stresses: an API with side effects is safe to retry only if it is idempotent.
flowchart TD
A["Modifying request + key"] --> B{"Is the key known?"}
B -->|"No"| C["Atomic recording and execution"]
C --> D["Record the result durably"]
B -->|"Yes"| F{"Same request?"}
F -->|"No"| G["Key conflict: reject"]
F -->|"Yes"| H["Return the earlier result"]
The scope of the key matters (it can be tied to a user and an operation type). A different payload with the same key must be rejected. Parallel first requests are protected by an atomic uniqueness key, not by a prior read. The "in progress" state must be designed too, and after the key expires there is no protection against repetition any more, so the retention time is part of the contract. The IETF Idempotency-Key header specification is currently a draft, not a published standard.
Retrying with delay: backoff and jitter
Immediate, unlimited repetition makes the situation worse. With exponential backoff the wait between repetitions grows with each attempt; jitter spreads the waits randomly so that clients do not retry at the same time. According to the AWS description, both the wait and the number of attempts should be capped. One possible illustrative rule: a random wait between 0 and min(maximum, base × 2^attempt); the parameters should be chosen based on the dependency and the need, not from a general recipe.
Retry storm and the responsible layer
If the client, the gateway and the service all repeat, the load multiplies. According to the AWS example, three attempts per layer across five layers can multiply the load on the deep database up to 3⁵ = 243 times. That is why, according to AWS, for low-cost operations it is worth retrying at a single point in the call chain, and only if the dependency looks healthy. If the server returns 503 because of overload, the Retry-After header can suggest a waiting time.
Common misconceptions
Statement
Clarification
"Retry always increases reliability."
Under overload it increases the load; it multiplies across layers.
"After a timeout the request did not happen."
It may have been executed.
"An idempotent operation gives the same response."
The effect is the same, the response can differ (for example 204 first, 404 later on a delete).
"The request identifier is also an idempotency key."
Only if the server implements such a rule.
In short
A deadline is a point in time, a timeout a duration; in a chain the remaining time is passed on.
Only transient errors and idempotent operations are retried automatically.
The idempotency key is durable, atomic, scoped and expiring.
Backoff and jitter spread the retries; a single layer is responsible for retrying.
Review questions
Why can a non-idempotent request not be repeated automatically?
What should happen with the same key and different input?
How does the load multiply with the independent retries of several layers?
Time Limits, Retries and Idempotency
Objectives
By the end of the lesson, you can explain the difference between timeout and deadline, say when it is safe to repeat a call, and how idempotency makes repetition safe.
Why is it needed?
The outcome of a remote call can be unknown (6.1). From the point of view of fault tolerance the question is how long to wait, when to repeat, and how to preserve the correctness of the business state when the outcome is uncertain. Repetition is the most common defence and also the most common source of errors.
Timeout and deadline
A timeout is a duration of waiting. A deadline is a point in time: until when the caller will still wait for the response. According to the gRPC documentation a timeout is the maximum duration of the call, and it becomes a deadline by adding it to the start time of the call. gRPC sets no deadline by default, so a realistic deadline must always be set explicitly in the client.
In chained calls the deadline must be passed on: if the client's time limit is finite, an inner call cannot get the whole limit again. gRPC forwards the remaining time, not the absolute time, so there is no clock-skew problem. After expiry the call may be cancelled on the server side, but the application must stop the work it started; cancellation does not undo a change that has already been carried out.
Which failure is retried?
A transient network failure or overload may improve later. Invalid input, missing authorisation and a business rejection do not improve with repetition. Automatic repetition should happen only for the agreed class of errors, in a limited number, and within the time limit.
According to RFC 9110 a method is idempotent if the intended effect of several identical requests is the same as that of one; in the standard PUT, DELETE and the safe methods are such. An idempotent request can be repeated automatically if a connection failure occurs before the response is read. A client should not automatically retry a request with a non-idempotent method unless it knows that the semantics of the request are in fact idempotent, or has some means to detect that the original request was never applied.
Idempotency key
For a modifying request the client can send the same idempotency key for the same logical operation. The server stores the key, the meaning of the request and the result durably; on repetition it does not execute it again but returns the earlier result. The AWS description also stresses: an API with side effects is safe to retry only if it is idempotent.
flowchart TD A["Modifying request + key"] --> B{"Is the key known?"} B -->|"No"| C["Atomic recording and execution"] C --> D["Record the result durably"] B -->|"Yes"| F{"Same request?"} F -->|"No"| G["Key conflict: reject"] F -->|"Yes"| H["Return the earlier result"]The scope of the key matters (it can be tied to a user and an operation type). A different payload with the same key must be rejected. Parallel first requests are protected by an atomic uniqueness key, not by a prior read. The "in progress" state must be designed too, and after the key expires there is no protection against repetition any more, so the retention time is part of the contract. The IETF
Idempotency-Keyheader specification is currently a draft, not a published standard.Retrying with delay: backoff and jitter
Immediate, unlimited repetition makes the situation worse. With exponential backoff the wait between repetitions grows with each attempt; jitter spreads the waits randomly so that clients do not retry at the same time. According to the AWS description, both the wait and the number of attempts should be capped. One possible illustrative rule: a random wait between 0 and min(maximum, base × 2^attempt); the parameters should be chosen based on the dependency and the need, not from a general recipe.
Retry storm and the responsible layer
If the client, the gateway and the service all repeat, the load multiplies. According to the AWS example, three attempts per layer across five layers can multiply the load on the deep database up to 3⁵ = 243 times. That is why, according to AWS, for low-cost operations it is worth retrying at a single point in the call chain, and only if the dependency looks healthy. If the server returns
503because of overload, theRetry-Afterheader can suggest a waiting time.Common misconceptions
204first,404later on a delete).In short
Review questions
Learning resources