By the end of the lesson, you can tell time measured on the server side from time measured on the user side, you can explain why response time worsens nonlinearly near capacity, and you know why we think in percentiles instead of averages.
Why is a single server-side time not enough?
The performance of a system cannot be described with a single server-side time. The network, the server, the downloaded resources, and the browser's work together determine when the user can act meaningfully (9.4). This lesson deepens the server-side question: what happens to the response time when more users arrive at once?
It is worth thinking of three separate questions: how fast does the server respond; how fast does useful content appear; and how fast does the interface react to an action. A good measurement names which question it answers.
flowchart LR
B[Browser] --> H[Network]
H --> S[Server response]
S --> L[Downloading resources]
L --> R[Rendering and interaction]
Response time and network latency
Response time is the time that passes from starting a request to the arrival of the response. It consists of several phases (4.6): the request has to reach the server, the server has to process it, the response has to come back. Network latency is partly a physical given (distance, a mobile connection, a congested network), and server-side processing can depend on a database query, the response of another service, or the complexity of a computation.
If an API prepares the response in 80 milliseconds from the server's point of view, the user may still experience half a second by the time the request travels there and back and the client processes the result. For this reason "it's fast for me" is not enough: the same page behaves differently from another network, device, and geographic location.
Backend measurement and user measurement
From the server side often the processing time of a request is visible. This is valuable, but it is only one half of the story: it does not include name resolution, the establishment of the connection, the fluctuation of the mobile network, the later download of resources, or how much the user's phone struggles with rendering. User measurement looks at what is actually lived through in the browser (Core Web Vitals, 9.4). The two viewpoints are not competitors: if the server-side time is good but the user-side is bad, the investigation turns toward the client, the network, and the resources; if both worsen, there is probably a common cause.
Load, capacity, bottleneck
Load can mean the number of incoming requests, active users, messages to be processed, database operations, or network traffic. Capacity is the quantity that the system can handle at the expected quality. A bottleneck is the component that saturates first (a processor, a pool of database connections, an external service), and thereby determines the performance of the whole system.
Near capacity the response time worsens not linearly but "knee-like": requests queue up waiting for each other, and every further request waits even longer.
Tried: the knee
We wrote a small trial server that works on a single thread and uses 10 milliseconds of processor time for every request, so its capacity is in theory about 100 requests per second. A client sent requests at different rates (requests per second), for 8 seconds at each rate, and we measured the response time (Node.js 24, on one machine, on a local network):
Incoming requests/s
Mean (ms)
p50 (ms)
p95 (ms)
p99 (ms)
20
12
12
15
28
50
10
10
10
11
80
11
10
10
20
90
10
10
11
11
95
12
10
20
54
105
229
217
463
1265
What do we see?
Below capacity (20–90 requests/s) the response time is essentially constant, about 10 ms, that is, the request's own processing time.
At 95 requests/s the mean is still good, but p95 and p99 already rise (20 and 54 ms): the weak signal appears in the slow exceptions earlier than in the mean.
Above capacity (105 requests/s) the queue grows: the p99 is more than a second, the mean jumped to ten times as much, although the load grew by only 5–10%. Because arrival is faster than service, the waiting queue keeps lengthening until the end of the measurement.
(The mean of the first row is slightly distorted by the JavaScript runtime "warming up" during the first requests. The real capacity of the server depends on the environment; the point is the shape of the phenomenon, not the concrete numbers.)
This relationship means that even a small excess can trigger a spectacular collapse: that is why capacity must be run with a reserve, and why the system must not always be run at its limit. The next lessons (10.4, 10.5) show what can be done if the load crosses the limit anyway.
Loading time: not a single moment
Loading time is often characterized by a single number, yet the visitor forms an opinion gradually. In the first moments they sense whether anything happened; later whether the main content is visible; finally whether the controls work. On a news portal the title and the beginning of the article should be visible soon; on a shopping page the price, availability, and the cart button; on an administration interface the first useful part of the table. Speed is thus not merely a matter of bytes but of priority (3.4, 9.4).
Resource demand: what do we ask of the visitor?
Every downloaded image, font, script, and measuring code requires data traffic, memory, processor time, and sometimes battery. On a modern desktop machine a large application may only seem slower; on an older phone or a slow network the same can be unusable. Performance is therefore also a question of fairness (9.3). JavaScript is especially expensive, because it is not only downloaded: it must also be processed and run. The goal is proportionality: does the downloaded and run content serve the user's task?
Distributions instead of averages
A percentile gives the value below which a given percentage of the measurements lie: p50 is the median (the typical case), p95 and p99 the "slow tail". According to Google's SRE book a high percentile shows the plausible worst case, and the distribution of latencies is often skewed, so the mean and the median are not the same. Many users meet the slow tail: if loading a page needs several dozen requests, each has a chance of being slow, and the user feels the slowest.
The pattern over time matters too. A system can be fast on a weekday morning and slow when everyone buys a ticket. The interpretation of performance should always be tied to load, device, network, and user task.
Worked example: the fast API and the slow enrolment page
The developers of Kurzuspont see that the API of the course list responds in 120 ms on average. From this they conclude that the enrolment page is fast. Students nevertheless complain: on mobile it takes many seconds before they can select the course.
The investigation reveals: the course photos are too large (3.4, 9.4), several external measuring codes slow the loading, and a large amount of client-side code has to be processed for the filter to work. The API measurement was not false, it just did not measure the whole user flow. On the other side, at the moment of opening the server gets near the knee: the mean stays acceptable, but the p99 takes off. The solution is not a single "speed-up trick": it is deciding which content is primary (9.4), what can be deferred, and how we prepare the system for the peak (10.4, 10.5).
Common misconceptions
Statement
Clarification
"Loading time is the same as the server's response time."
The server's response time is only one component.
"A test on the developer's fast machine is enough."
Users' devices and connections are varied.
"Response time grows in proportion to the load."
Near capacity it worsens nonlinearly, knee-like.
"The average response time describes quality well."
The average hides the slow but real cases; p95/p99 shows them.
"The more content on the page, the better."
Early usefulness and reaction are worsened by needless initial load.
In brief
Server-side response time is only one component of the experience; user measurement looks at the whole chain.
Near capacity the response time worsens knee-like; the weak signal appears first in p95/p99.
On the tried server at 90 requests/s p99 ≈ 11 ms, at 105 requests/s p99 ≈ 1265 ms.
It is worth thinking in percentiles (p50, p95, p99) instead of the average.
Capacity must be planned with a reserve, and the bottleneck must be found.
Review questions
What does a bottleneck mean, and what can it be in a web application?
Why does p99 signal saturation earlier than the mean?
Why does the response time suddenly worsen when capacity is exceeded?
What is the difference between time measured on the server side and on the user side?
Response time, load, and capacity
Objectives
By the end of the lesson, you can tell time measured on the server side from time measured on the user side, you can explain why response time worsens nonlinearly near capacity, and you know why we think in percentiles instead of averages.
Why is a single server-side time not enough?
The performance of a system cannot be described with a single server-side time. The network, the server, the downloaded resources, and the browser's work together determine when the user can act meaningfully (9.4). This lesson deepens the server-side question: what happens to the response time when more users arrive at once?
It is worth thinking of three separate questions: how fast does the server respond; how fast does useful content appear; and how fast does the interface react to an action. A good measurement names which question it answers.
flowchart LR B[Browser] --> H[Network] H --> S[Server response] S --> L[Downloading resources] L --> R[Rendering and interaction]Response time and network latency
Response time is the time that passes from starting a request to the arrival of the response. It consists of several phases (4.6): the request has to reach the server, the server has to process it, the response has to come back. Network latency is partly a physical given (distance, a mobile connection, a congested network), and server-side processing can depend on a database query, the response of another service, or the complexity of a computation.
If an API prepares the response in 80 milliseconds from the server's point of view, the user may still experience half a second by the time the request travels there and back and the client processes the result. For this reason "it's fast for me" is not enough: the same page behaves differently from another network, device, and geographic location.
Backend measurement and user measurement
From the server side often the processing time of a request is visible. This is valuable, but it is only one half of the story: it does not include name resolution, the establishment of the connection, the fluctuation of the mobile network, the later download of resources, or how much the user's phone struggles with rendering. User measurement looks at what is actually lived through in the browser (Core Web Vitals, 9.4). The two viewpoints are not competitors: if the server-side time is good but the user-side is bad, the investigation turns toward the client, the network, and the resources; if both worsen, there is probably a common cause.
Load, capacity, bottleneck
Load can mean the number of incoming requests, active users, messages to be processed, database operations, or network traffic. Capacity is the quantity that the system can handle at the expected quality. A bottleneck is the component that saturates first (a processor, a pool of database connections, an external service), and thereby determines the performance of the whole system.
Near capacity the response time worsens not linearly but "knee-like": requests queue up waiting for each other, and every further request waits even longer.
Tried: the knee
We wrote a small trial server that works on a single thread and uses 10 milliseconds of processor time for every request, so its capacity is in theory about 100 requests per second. A client sent requests at different rates (requests per second), for 8 seconds at each rate, and we measured the response time (Node.js 24, on one machine, on a local network):
What do we see?
(The mean of the first row is slightly distorted by the JavaScript runtime "warming up" during the first requests. The real capacity of the server depends on the environment; the point is the shape of the phenomenon, not the concrete numbers.)
This relationship means that even a small excess can trigger a spectacular collapse: that is why capacity must be run with a reserve, and why the system must not always be run at its limit. The next lessons (10.4, 10.5) show what can be done if the load crosses the limit anyway.
Loading time: not a single moment
Loading time is often characterized by a single number, yet the visitor forms an opinion gradually. In the first moments they sense whether anything happened; later whether the main content is visible; finally whether the controls work. On a news portal the title and the beginning of the article should be visible soon; on a shopping page the price, availability, and the cart button; on an administration interface the first useful part of the table. Speed is thus not merely a matter of bytes but of priority (3.4, 9.4).
Resource demand: what do we ask of the visitor?
Every downloaded image, font, script, and measuring code requires data traffic, memory, processor time, and sometimes battery. On a modern desktop machine a large application may only seem slower; on an older phone or a slow network the same can be unusable. Performance is therefore also a question of fairness (9.3). JavaScript is especially expensive, because it is not only downloaded: it must also be processed and run. The goal is proportionality: does the downloaded and run content serve the user's task?
Distributions instead of averages
A percentile gives the value below which a given percentage of the measurements lie: p50 is the median (the typical case), p95 and p99 the "slow tail". According to Google's SRE book a high percentile shows the plausible worst case, and the distribution of latencies is often skewed, so the mean and the median are not the same. Many users meet the slow tail: if loading a page needs several dozen requests, each has a chance of being slow, and the user feels the slowest.
The pattern over time matters too. A system can be fast on a weekday morning and slow when everyone buys a ticket. The interpretation of performance should always be tied to load, device, network, and user task.
Worked example: the fast API and the slow enrolment page
The developers of Kurzuspont see that the API of the course list responds in 120 ms on average. From this they conclude that the enrolment page is fast. Students nevertheless complain: on mobile it takes many seconds before they can select the course.
The investigation reveals: the course photos are too large (3.4, 9.4), several external measuring codes slow the loading, and a large amount of client-side code has to be processed for the filter to work. The API measurement was not false, it just did not measure the whole user flow. On the other side, at the moment of opening the server gets near the knee: the mean stays acceptable, but the p99 takes off. The solution is not a single "speed-up trick": it is deciding which content is primary (9.4), what can be deferred, and how we prepare the system for the peak (10.4, 10.5).
Common misconceptions
In brief
Review questions
Learning resources