Latency and errors on the path of the request

Objectives

By the end of the lesson, you can say what the loading time of a page is made up of, separate the errors before and after HTTP, and choose the right point of investigation from a symptom.

Why is "the internet is slow" not enough?

If a page is slow or does not open, "the internet is bad" is not a precise enough explanation. Name resolution, connection setup, TLS, the server's response, further resources, and the browser's work can each cause delay or error separately. The solution is different too: for a DNS error the naming system must be examined, for a slow server the application. Whoever can assign the right layer to a symptom reaches the cause faster.

The concept of latency

Latency is the time that passes between starting a request and the arrival of the result. It has several components:

• propagation time: the physical path of the signal; it grows with distance. It cannot be bypassed with faster hardware, only with content brought closer (CDN, 4.4).
• connection setup: the round trips of name resolution and the TCP and TLS handshakes (4.2).
• server-side processing: the application looks up data, waits for another service.
• transfer: the size of the response divided by the transfer speed.
• processing in the browser: DOM, style, layout, JavaScript (3.3).

The measurement of the previous lesson showed that reaching the same page in two ways gives different proportions of the steps. A measurement of a more distant service (three measurements from www.wikipedia.org from the same machine, in seconds):

dns=0.0041 tcp=0.0461 tls=0.0979 ttfb=0.1517 total=0.2615
dns=0.0047 tcp=0.1341 tls=0.1894 ttfb=0.2475 total=0.3782
dns=0.0069 tcp=0.0514 tls=0.1105 ttfb=0.2522 total=0.3678

The time of the TCP connection (tcp − dns) fluctuated between 42 and 129 ms, while for example.com it was 9–11 ms. The effect of distance and network state is thus clearly visible, and so is the fact that the same measurement does not repeat exactly. This is why it is worth looking not at one measurement but at a distribution (see 10.3).

The segments do not always add up simply: some resources load in parallel, an earlier connection or a cached response saves steps, and at other times a resource can be discovered only after processing a previous file. So the timeline of the specific page must be examined, rather than reciting a general rule.

The same symptom, a different cause

A student would open the course page but sees an empty screen for a long time. The cause may be that DNS gives no answer, the connection is established slowly, the server works for a long time, or the HTML has arrived but the browser is waiting for an important style sheet or program.

flowchart TD
    T[Slow or faulty page] --> D[DNS answer?]
    D --> C[Connection and TLS?]
    C --> H[HTTP response?]
    H --> R[Resources and rendering?]
    R --> U[Does the user action work?]

The diagram is not an automatic decision tree. The steps can partly overlap. The order is useful for separating the problems that arise before, during, and after the response.

Errors before HTTP

If the process gets stuck before the HTTP request, no status code arrives. The real curl errors below show where the chain broke:

Command	Error	Where did it get stuck?
curl https://kurzuspont.example/	curl: (6) Could not resolve host: kurzuspont.example	DNS: the name cannot be resolved
curl http://localhost:9/	curl: (7) Failed to connect to localhost port 9 ... Couldn't connect to server	TCP: nobody accepts connections on this port
curl --max-time 2 http://10.255.255.1/	curl: (28) Connection timed out after 2005 milliseconds	TCP: the target does not respond, the request times out
curl https://expired.badssl.com/	curl: (60) SSL certificate problem: certificate has expired	TLS: the certificate has expired

(The error codes of curl also indicate the steps: 6, 7, 28, 60. In the browser the same errors appear with different text.)

Errors in HTTP

If the request reached a responding web system, we get an HTTP status code (2.4). A 404 is an HTTP response: the request reached the server, and the resource was not found there. A 500 denotes a class of server-side error. A curl call runs successfully in such a case:

$ curl -sS -o /dev/null -w 'http_code=%{http_code}\n' http://localhost:8080/nincs
http_code=404

This also means that the exit code of curl was 0 (the network operation succeeded), even though the page was not found. A network error and an HTTP error are therefore two separate questions.

Errors after HTTP

Next to a successful main document, a sub-resource can fail separately: a missing style sheet is betrayed by a 404 row in the network panel while the text of the HTML appears. A late-arriving image harms the stability of the page, and a long JavaScript task delays the response of the interface. The error therefore has to be interpreted per resource and also by its effect on the user.

Investigating in the right place

1. Address and host name: is it correct? (DNS)
2. Connection: is the service reachable? (curl, the browser's error page)
3. Main document: status, timing, content type in the network panel.
4. Sub-resources: which one is faulty or slow?
5. Current DOM and styles: in the Elements view.
6. User action: can the task be carried through?

A single measurement does not generalize to every user: the network, the location, the device, the state of the cache, and the server load also vary. A diagnosis is a conclusion built on observations, where it must be stated clearly which layer there is evidence for and which not.

Worked example: Kurzuspont is slow for the student

Complaint: "Kurzuspont does not open." Possible diagnoses by symptom:

Observation	Likely layer	What to do
"Server not found" error page	DNS	name server settings, TTL
Spins for a long time, then times out	TCP or network	reachability, firewall
Certificate error in the browser	TLS	renew the certificate
Opens, but 503	application or overload	capacity, chapter 10
Opens, without styles	sub-resource	access to the CSS
Opens, but the button does not respond	processing in the browser	JavaScript error

The complaint is the same, the action differs row by row. It is the provider's responsibility that the error message helps the user tell these apart too (see 10.1).

Common misconceptions

Statement	Clarification
"Slow page = slow internet."	The work of the server and the browser can also cause delay.
"A 404 means there is no network."	A 404 is already an HTTP response from a reached system.
"If the HTML is 200, everything is fine."	Further resources or the user action may fail.
"Delays always add up in sequence, in the same way."	A cache, parallelism, and a reused connection modify the timeline.
"One measurement is enough."	The values fluctuate from measurement to measurement.

In brief

• Latency consists of propagation time, connection setup, server-side processing, transfer, and processing in the browser.
• Errors before HTTP (DNS, TCP, TLS) give no status code; HTTP errors do.
• An error can be in the main document or in a sub-resource.
• One measurement does not tell everything: the distribution matters.
• Diagnosis assigns the symptom to the right layer.

Review questions

1. Which layer's error does "Could not resolve host" indicate?
2. Why is the exit code of curl 0 for a 404 response?
3. Why does the connection setup time fluctuate on the same page?
4. How can a certificate error be told apart from a 503 response?

Learning resources

• How the web works (MDN) — the steps.
• HTTP response status codes (MDN) — the list of error codes.