By the end of the lesson, you can say how latency adds up in sequential and parallel calls, what fan-out and tail latency are, and how the number of mandatory dependencies affects availability.
Why is it needed?
The performance of communication between services is described not by the average response time of a single endpoint, but by the whole request path: the number, order and parallelism of the calls and the amount of data transferred. A system made of many small services may spend more time on communication than on business computation.
Sequential and parallel calls
If a request makes n dependent calls in sequence, the latencies add up. If the calls are independent and start together, the total time approaches that of the slowest, but more simultaneous resources are needed. Parallelisation does not remove data dependence: if the parameter of the second call is the result of the first, they cannot start together.
sequenceDiagram
participant G as Assembler
participant A as Service A
participant B as Service B
participant C as Service C
par Independent data
G->>A: request
A-->>G: data
and
G->>B: request
B-->>G: data
and
G->>C: request
C-->>G: data
end
G->>G: Assemble the result
Fan-out and tail latency
Fan-out: one incoming request starts several internal calls. If all results must be awaited, the slowest dependency determines completion; the more calls, the more likely that one of them is slow. The average hides this: the range of slow requests is shown by high percentiles (for example p95, p99). This phenomenon is called tail latency. Percentiles cannot simply be added up; the whole path must be measured, which a distributed trace (tracing of the call chain) makes visible.
Availability with mandatory dependencies
If the success of an operation needs each of n independent services, and each is 99% available, the combined availability is 0.99ⁿ. For three dependencies that is 0.99³ ≈ 97.03%. Real dependencies often correlate, so this is not an SLA calculation, only an illustration that stacking mandatory dependencies lowers the success of the whole operation.
Chatty APIs and batching
A connection is chatty if one business operation needs many small back-and-forth calls. A summary endpoint, batching or a targeted read model (4.3) can reduce the cost. An overly large response, however, carries needless data and creates a tight contract. The goal is a boundary fitted to the given consumption need, not merging all calls.
Mandatory and optional dependencies
Not all data is equally critical. If optional data is missing, the page can still be shown; if the result of a payment is unknown, the purchase must not be declared successful. Degraded operation must be decided on the basis of business meaning, not with a general "return an empty list on error" rule (6.6).
Common misconceptions
Statement
Clarification
"If every endpoint is fast, the system is fast."
The total time of the path, sequentiality and the slowest dependency matter.
"The average response time shows the user experience."
The range of slow requests is shown by percentiles.
"Parallelisation always helps."
With data dependence it is not possible, and it needs more resources.
"Smaller services are faster."
More network calls can appear.
In short
In sequential calls latencies add up, in parallel ones the slowest counts.
Fan-out increases the likelihood of tail latency; percentiles show it.
Mandatory dependencies mean multiplying availability.
A chatty connection can be reduced with batching, a summary endpoint or a read model.
Review questions
Why do three separately fast endpoints not prove a fast system?
When can fan-out be reduced with a read model?
Without which data can a page keep working, and what must it signal?
Latency, Fan-Out and Communication Coupling
Objectives
By the end of the lesson, you can say how latency adds up in sequential and parallel calls, what fan-out and tail latency are, and how the number of mandatory dependencies affects availability.
Why is it needed?
The performance of communication between services is described not by the average response time of a single endpoint, but by the whole request path: the number, order and parallelism of the calls and the amount of data transferred. A system made of many small services may spend more time on communication than on business computation.
Sequential and parallel calls
If a request makes n dependent calls in sequence, the latencies add up. If the calls are independent and start together, the total time approaches that of the slowest, but more simultaneous resources are needed. Parallelisation does not remove data dependence: if the parameter of the second call is the result of the first, they cannot start together.
sequenceDiagram participant G as Assembler participant A as Service A participant B as Service B participant C as Service C par Independent data G->>A: request A-->>G: data and G->>B: request B-->>G: data and G->>C: request C-->>G: data end G->>G: Assemble the resultFan-out and tail latency
Fan-out: one incoming request starts several internal calls. If all results must be awaited, the slowest dependency determines completion; the more calls, the more likely that one of them is slow. The average hides this: the range of slow requests is shown by high percentiles (for example p95, p99). This phenomenon is called tail latency. Percentiles cannot simply be added up; the whole path must be measured, which a distributed trace (tracing of the call chain) makes visible.
Availability with mandatory dependencies
If the success of an operation needs each of n independent services, and each is 99% available, the combined availability is 0.99ⁿ. For three dependencies that is 0.99³ ≈ 97.03%. Real dependencies often correlate, so this is not an SLA calculation, only an illustration that stacking mandatory dependencies lowers the success of the whole operation.
Chatty APIs and batching
A connection is chatty if one business operation needs many small back-and-forth calls. A summary endpoint, batching or a targeted read model (4.3) can reduce the cost. An overly large response, however, carries needless data and creates a tight contract. The goal is a boundary fitted to the given consumption need, not merging all calls.
Mandatory and optional dependencies
Not all data is equally critical. If optional data is missing, the page can still be shown; if the result of a payment is unknown, the purchase must not be declared successful. Degraded operation must be decided on the basis of business meaning, not with a general "return an empty list on error" rule (6.6).
Common misconceptions
In short
Review questions
Learning resources