By the end of the lesson, you can explain why overload worsens by itself, you know the tools of defense (rate limiting, queue, graceful degradation), and you can tell the three signals of observability apart: log, metric, and trace.
Why is "starting more servers" not enough?
On a normal weekday a web shop receives a few dozen requests per second. At the start of a sale or the ticket sales of a popular concert, however, within a few minutes thousands may try to do the same operation. It is not only the number of page loads that grows: many refresh, log in, put things in the cart, start a payment. Different parts of the system react differently. In lesson 10.3 we saw that near capacity the response time worsens knee-like. Now we examine what we can do if the load nevertheless reaches or exceeds the limit.
Peak traffic: why does the fault worsen by itself?
On a slow page the user often clicks again or refreshes. A client may automatically retry a failed request. This is understandable behavior, but it loads an already overloaded system further. If the backend service is slow, the application server keeps more open requests alive, consumes more memory, and in the end other, still working functions slow down too.
flowchart LR
U[Many simultaneous requests] --> G[Entry limit]
G --> Q[Queue]
Q --> S[Primary operation]
S --> M[Metrics and logs]
M --> D[Decision about capacity]
Rate limiting
Rate limiting regulates how many requests a client, user, IP address, or access token may send in a given time. It is needed not only because of attacks: a faulty program can call an API a thousand times a minute; and at the moment of a popular opening the same person can spoil others' chances by constant refreshing.
When designing a limit two questions must be separated: whom do we identify (user, token, IP address), and what do we protect (the login endpoint is stricter because of password guessing, a public image may be more permissive). The response must be clear: according to RFC 6585 429 Too Many Requests signals that the client has sent too many requests within a given time, and the response may contain a Retry-After header, which says when it can try again. Tried on a server that allows 5 requests per client every 10 seconds:
requests 1–5 : 200
request 6 : 429
request 7 : 429
HTTP/1.1 429 Too Many Requests
Content-Type: application/problem+json
Retry-After: 10
{"type":"about:blank","title":"Túl sok kérés","status":429,
"detail":"10 másodpercenként legfeljebb 5 kérés engedélyezett."}
Another client (bela) at the same time: 200
(The Hungarian texts mean "Too many requests" and "At most 5 requests are allowed every 10 seconds.")
The limit works per client: Béla's requests do not count into Anna's limit. The response uses the error format seen in 10.2, so the client also understands by machine how long it has to wait.
A rate limit is not a punishment but the organization of a shared resource. With a bad design, however, it is unfair: many students of a university network can appear behind the same external IP address, so an IP-address-based limit hits them together. The designer must understand the consequence, not just flip the switch.
Queues
For some operations the user expects an immediate response: login, putting into the cart, confirmation of payment. Others can be delayed: sending an invoice or a confirmation email, processing a large image, sending notifications, refreshing a search index. A queue detaches these latter jobs from the direct request. The web application quickly accepts the task, and a background process works through it later, at its own pace.
With an uploaded video it would be a bad experience if the browser waited until all the versions were ready. The system can rather signal: "The upload succeeded, processing is in progress." This improves responsiveness and smooths the load. In exchange, the delay, a possible failed processing, and making sure a task running twice does not cause a double effect (idempotency, 2.3 and 10.2) must be designed.
Graceful degradation
Graceful degradation means that on a fault or overload the system does not become unusable in a single moment. First it limits the less important functions, gives a simpler view, or offers later processing. It tries to preserve the most important tasks. In a ticket-selling system, for example, the personalized recommender can be temporarily switched off, the page can load fewer images, and instead of search predefined categories can appear. The basic goal (the correct selection and purchase of tickets) takes precedence. If the stock information cannot be guaranteed, the system must communicate honestly, and must not ask for payment with a false promise.
This is not giving up quality, but making priorities conscious. At design time we must ask: what is the minimally acceptable operation of the service? Which data must be accurate? What can wait? What can be hidden temporarily? The answer is a business, user, and ethical question as well.
Observability: how do we know what is happening?
The state of a complex web system is not directly visible. The user says only: "slow" or "the payment didn't work". According to the definition of OpenTelemetry, observability means that we can understand the system from the outside, without knowing its inside, by asking questions, and can investigate new, unexpected problems too. For this the application must emit telemetry signals. The three basic signals:
Signal
What is it?
What is it good for?
Example
Log
A record of time-stamped, concrete events
Finding out the detailed cause of a given event
"14:02:11 – saving the enrolment failed with a timeout, request: 8841"
Metric
A numeric value aggregated over time
Detecting trends and anomalies, alerting
requests/s, error rate, p95 response time, queue length
Trace
The path of a request through the processing services
Where does the time go?
the enrolment: application 20 ms, database 400 ms, payment provider 1.2 s
According to OpenTelemetry a trace consists of spans (a unit of work, with a name, timings, and attributes); the root span covers the whole request, the child spans the steps. Logs are most useful when they can be linked to a trace or span.
A log must not irresponsibly contain personal data or passwords (9.6, 8.6): usefulness and privacy need a balance. Metrics are excellent for trends: if the error rate jumps from 0.1% to 8%, it stands out even if we do not yet know in which user's request the error occurred. That becomes clear from the log and the trace. Observability connects to chapter 8 too: the OWASP Top 10:2025 has a separate category for logging and alerting failures (8.7).
Alerting: not every graph asks for attention
An alert must trigger action. If a team receives a dozen insignificant messages every minute, it gets used to them and ignores them (alert fatigue). A good alert therefore does not merely say that the processor of a machine is high, but is tied to user impact: the proportion of failed payments is persistently rising, or the p95 response time of logins exceeds a threshold (10.1: SLI/SLO). An alert is often justified by several signals together: a short traffic peak can be normal, while rising latency, error rate, and queue length together are already a strong warning. The thresholds are not eternal values: they must be reviewed as the system and the expectation change.
Worked example: the opening of course registration in Kurzuspont
At nine o'clock course registration opens.
Tool
What does it do at opening?
CDN (4.4, 10.4)
The static elements of the home page do not load the application.
Rate limiting
Login is limited, so password guessing and aggressive retrying do not block everyone; the limit gives a 429 + Retry-After response.
Careful handling
The subject list and the capacity are critical: the server decides, without a false "there is room" promise.
Queue
The confirmation emails go into a queue: the student sees the success on the screen first, the email arrives later.
Graceful degradation
On overload the non-vital recommender and statistics can be switched off.
Observation
The metrics: error rate, p95, number of database connections, queue length; the problematic request can be investigated in a trace; the details can be searched in the log.
Alerting
The proportion of successful enrolments falls below the SLO (10.1) → the on-call person is notified.
This way the system not only reacts, but can also be diagnosed.
Common misconceptions
Statement
Clarification
"If there is monitoring, we will surely notice the fault."
Only what we measure can be noticed, if the signal is interpretable and someone reacts to it.
"More servers solve every overload."
Not if the database, an external API, or a shared lock is the bottleneck.
"A rate limit affects only malicious attackers."
It can affect a well-meaning but faulty client too; the fairness of the message and the rule matters.
"Graceful degradation means the system is working wrongly."
It is a predefined, safe limitation instead of an uncontrolled failure.
"The more alerts, the safer."
Many insignificant alerts cause fatigue and drown out the important one.
In brief
Overload reinforces itself (reloading, retrying); first the bottleneck must be found.
Load peaks and observability
Objectives
By the end of the lesson, you can explain why overload worsens by itself, you know the tools of defense (rate limiting, queue, graceful degradation), and you can tell the three signals of observability apart: log, metric, and trace.
Why is "starting more servers" not enough?
On a normal weekday a web shop receives a few dozen requests per second. At the start of a sale or the ticket sales of a popular concert, however, within a few minutes thousands may try to do the same operation. It is not only the number of page loads that grows: many refresh, log in, put things in the cart, start a payment. Different parts of the system react differently. In lesson 10.3 we saw that near capacity the response time worsens knee-like. Now we examine what we can do if the load nevertheless reaches or exceeds the limit.
Peak traffic: why does the fault worsen by itself?
On a slow page the user often clicks again or refreshes. A client may automatically retry a failed request. This is understandable behavior, but it loads an already overloaded system further. If the backend service is slow, the application server keeps more open requests alive, consumes more memory, and in the end other, still working functions slow down too.
flowchart LR U[Many simultaneous requests] --> G[Entry limit] G --> Q[Queue] Q --> S[Primary operation] S --> M[Metrics and logs] M --> D[Decision about capacity]Rate limiting
Rate limiting regulates how many requests a client, user, IP address, or access token may send in a given time. It is needed not only because of attacks: a faulty program can call an API a thousand times a minute; and at the moment of a popular opening the same person can spoil others' chances by constant refreshing.
When designing a limit two questions must be separated: whom do we identify (user, token, IP address), and what do we protect (the login endpoint is stricter because of password guessing, a public image may be more permissive). The response must be clear: according to RFC 6585
429 Too Many Requestssignals that the client has sent too many requests within a given time, and the response may contain aRetry-Afterheader, which says when it can try again. Tried on a server that allows 5 requests per client every 10 seconds:(The Hungarian texts mean "Too many requests" and "At most 5 requests are allowed every 10 seconds.")
The limit works per client: Béla's requests do not count into Anna's limit. The response uses the error format seen in 10.2, so the client also understands by machine how long it has to wait.
A rate limit is not a punishment but the organization of a shared resource. With a bad design, however, it is unfair: many students of a university network can appear behind the same external IP address, so an IP-address-based limit hits them together. The designer must understand the consequence, not just flip the switch.
Queues
For some operations the user expects an immediate response: login, putting into the cart, confirmation of payment. Others can be delayed: sending an invoice or a confirmation email, processing a large image, sending notifications, refreshing a search index. A queue detaches these latter jobs from the direct request. The web application quickly accepts the task, and a background process works through it later, at its own pace.
With an uploaded video it would be a bad experience if the browser waited until all the versions were ready. The system can rather signal: "The upload succeeded, processing is in progress." This improves responsiveness and smooths the load. In exchange, the delay, a possible failed processing, and making sure a task running twice does not cause a double effect (idempotency, 2.3 and 10.2) must be designed.
Graceful degradation
Graceful degradation means that on a fault or overload the system does not become unusable in a single moment. First it limits the less important functions, gives a simpler view, or offers later processing. It tries to preserve the most important tasks. In a ticket-selling system, for example, the personalized recommender can be temporarily switched off, the page can load fewer images, and instead of search predefined categories can appear. The basic goal (the correct selection and purchase of tickets) takes precedence. If the stock information cannot be guaranteed, the system must communicate honestly, and must not ask for payment with a false promise.
This is not giving up quality, but making priorities conscious. At design time we must ask: what is the minimally acceptable operation of the service? Which data must be accurate? What can wait? What can be hidden temporarily? The answer is a business, user, and ethical question as well.
Observability: how do we know what is happening?
The state of a complex web system is not directly visible. The user says only: "slow" or "the payment didn't work". According to the definition of OpenTelemetry, observability means that we can understand the system from the outside, without knowing its inside, by asking questions, and can investigate new, unexpected problems too. For this the application must emit telemetry signals. The three basic signals:
According to OpenTelemetry a trace consists of spans (a unit of work, with a name, timings, and attributes); the root span covers the whole request, the child spans the steps. Logs are most useful when they can be linked to a trace or span.
A log must not irresponsibly contain personal data or passwords (9.6, 8.6): usefulness and privacy need a balance. Metrics are excellent for trends: if the error rate jumps from 0.1% to 8%, it stands out even if we do not yet know in which user's request the error occurred. That becomes clear from the log and the trace. Observability connects to chapter 8 too: the OWASP Top 10:2025 has a separate category for logging and alerting failures (8.7).
Alerting: not every graph asks for attention
An alert must trigger action. If a team receives a dozen insignificant messages every minute, it gets used to them and ignores them (alert fatigue). A good alert therefore does not merely say that the processor of a machine is high, but is tied to user impact: the proportion of failed payments is persistently rising, or the p95 response time of logins exceeds a threshold (10.1: SLI/SLO). An alert is often justified by several signals together: a short traffic peak can be normal, while rising latency, error rate, and queue length together are already a strong warning. The thresholds are not eternal values: they must be reviewed as the system and the expectation change.
Worked example: the opening of course registration in Kurzuspont
At nine o'clock course registration opens.
429+Retry-Afterresponse.This way the system not only reacts, but can also be diagnosed.
Common misconceptions
In brief
429,Retry-After), queue, graceful degradation.Review questions
429andRetry-Aftermean?Learning resources
429.