By the end of the lesson, you can explain how the circuit breaker, the bulkhead and load limits restrict the spread of failures, and when to use which.
Why is it needed?
A failed or slow dependency spoils not only the given call: the requests waiting for it can occupy connections, threads and memory, and then the failure spreads to other functions. The goal of fault tolerance is to limit the spread of failures and to support recovery.
Circuit breaker
The circuit breaker observes the results of calls to a dependency, and on persistent failure temporarily stops new attempts. According to Fowler's description it has three states:
Closed: calls go through; failures increase a counter, a success resets it.
Open: after the threshold is reached, the breaker returns an error immediately and does not call the protected operation.
Half-open: after a waiting time it lets one trial call through; on success it closes, on failure the waiting time starts again.
The threshold can be a fixed number of consecutive failures or a failure rate. A business rejection (for example "no stock") is not necessarily a dependency failure; if it is counted as a technical error, the breaker may disable a healthy service. The breaker does not carry out the operation later; it rejects quickly or gives a planned alternative. According to Fowler, the client must handle the open state: it can be an error, or an alternative solution (for example queueing the operation, or stale but acceptable data). The breaker is a good monitoring point: state changes should be logged.
Bulkhead
The bulkhead divides resources into separate pools so that the failure of one does not take down the others. The name refers to the compartmentalisation of a ship's hull. According to the Azure description a consumer can, for example, get a separate connection pool per service; if a service breaks, only the pool assigned to it fills up. With asynchronous messaging, jobs can also be isolated with separate queues.
The price of isolation is resource efficiency and complexity: too narrow a limit may reject even under normal load, too wide a limit does not protect.
Rate limit and concurrency limit
A rate limit restricts the number of accepted requests per unit of time. A concurrency limit restricts the work in progress at the same time. In a slowing system the same request rate per second results in more simultaneous work, so the two limits address different problems. A separate limit per user, tenant, endpoint or external dependency may be justified so that one client cannot take all the capacity. Under overload a clear rejection and a retry signal are needed (for example 429, 503 and Retry-After).
Degraded operation
A fallback can be cached data, a partial response or leaving out an optional function. The meaning of the result must be preserved: "the payment service is faulty, therefore a successful payment is assumed" is unacceptable; "reviews are temporarily unavailable" can be correct partial operation. A general fallback for every endpoint is a bad shortcut.
Common misconceptions
Statement
Clarification
"The circuit breaker carries out the operation later."
It rejects quickly; later execution needs a durable queue.
"The bulkhead always improves efficiency."
It reserves resources; the price of isolation is utilisation.
"Rate limit and concurrency limit are the same."
One limits the rate, the other the simultaneous work.
"A fallback is always good."
Only if the meaning is preserved.
In short
The circuit breaker rejects quickly on persistent failure; it has three states.
The bulkhead separates resources so the failure does not spread.
The rate limit restricts the request rate, the concurrency limit the simultaneous work.
A fallback is good only if it preserves the meaning of the result.
Review questions
What is the difference between a circuit breaker and a retry?
Why are separate limits needed for the request rate and for simultaneous work?
When is a fallback unacceptable in the case of a payment failure?
Circuit Breaker, Bulkhead and Load Limits
Objectives
By the end of the lesson, you can explain how the circuit breaker, the bulkhead and load limits restrict the spread of failures, and when to use which.
Why is it needed?
A failed or slow dependency spoils not only the given call: the requests waiting for it can occupy connections, threads and memory, and then the failure spreads to other functions. The goal of fault tolerance is to limit the spread of failures and to support recovery.
Circuit breaker
The circuit breaker observes the results of calls to a dependency, and on persistent failure temporarily stops new attempts. According to Fowler's description it has three states:
The threshold can be a fixed number of consecutive failures or a failure rate. A business rejection (for example "no stock") is not necessarily a dependency failure; if it is counted as a technical error, the breaker may disable a healthy service. The breaker does not carry out the operation later; it rejects quickly or gives a planned alternative. According to Fowler, the client must handle the open state: it can be an error, or an alternative solution (for example queueing the operation, or stale but acceptable data). The breaker is a good monitoring point: state changes should be logged.
Bulkhead
The bulkhead divides resources into separate pools so that the failure of one does not take down the others. The name refers to the compartmentalisation of a ship's hull. According to the Azure description a consumer can, for example, get a separate connection pool per service; if a service breaks, only the pool assigned to it fills up. With asynchronous messaging, jobs can also be isolated with separate queues.
The price of isolation is resource efficiency and complexity: too narrow a limit may reject even under normal load, too wide a limit does not protect.
Rate limit and concurrency limit
A rate limit restricts the number of accepted requests per unit of time. A concurrency limit restricts the work in progress at the same time. In a slowing system the same request rate per second results in more simultaneous work, so the two limits address different problems. A separate limit per user, tenant, endpoint or external dependency may be justified so that one client cannot take all the capacity. Under overload a clear rejection and a retry signal are needed (for example
429,503andRetry-After).Degraded operation
A fallback can be cached data, a partial response or leaving out an optional function. The meaning of the result must be preserved: "the payment service is faulty, therefore a successful payment is assumed" is unacceptable; "reviews are temporarily unavailable" can be correct partial operation. A general fallback for every endpoint is a bad shortcut.
Common misconceptions
In short
Review questions
Learning resources