Acknowledgement, Delivery Guarantees and Duplication
Objectives
By the end of the lesson, you can distinguish the producer and the consumer acknowledgement, explain the at-most-once, at-least-once and exactly-once guarantees, and say why the consumer must expect duplication.
Why is it needed?
The reliability of messaging must be interpreted along the whole path: producer, broker, consumer and the business data store. A "reliable broker" does not mean that a business effect happens exactly once.
Two separate acknowledgements
According to the RabbitMQ documentation, a publisher confirm tells the producer that the broker has handled the message (for a durable queue this means it has been written to disk). A consumer acknowledgement relates to the consumer's processing. The documentation considers the automatic acknowledgement mode "unsafe" for work that needs reliable delivery: the message counts as delivered at the moment it is sent. With manual acknowledgement, an unacknowledged message automatically goes back into the queue when the channel or connection closes.
sequenceDiagram
participant P as Producer
participant B as Broker
participant C as Consumer
participant D as Business database
P->>B: Message
B-->>P: Producer acknowledgement
B->>C: Delivery
C->>D: Business modification
D-->>C: Commit
C-->>B: Consumer acknowledgement
If the consumer stops after the commit but before the acknowledgement, the message may be delivered again while the effect has already happened. If the acknowledgement goes out before the commit, the work may be lost after a failure. The order of acknowledgement is therefore part of the durability plan.
At-most-once, at-least-once, exactly-once
At-most-once: there is no redelivery, so the message may be lost. Acceptable for fast-expiring data (for example the latest value of a measuring instrument).
At-least-once: redelivery happens until success, so duplication is possible. For example Amazon SQS standard queues give at-least-once delivery, more than one copy of a message might be delivered, and messages may occasionally arrive out of order; the RabbitMQ documentation also asks consumers to be ready for redelivery and to be idempotent.
Exactly-once: a broker or processor can provide such a property within a given transactional scope, but the effect of an external e-mail, a bank API or another database does not become exactly-once because of it. The boundary of the claim must always be named.
Idempotent consumer
At-least-once delivery can be used safely together with an idempotent consumer (6.5). The consumer can durably record the processed event identifier in one transaction with the business modification (inbox, 5.3), so on redelivery it recognises the already-happened effect. The check and the save must not be two independent steps, because two parallel consumers may both see the message as "new". The retention time of deduplication identifiers should match the time window of repetition, otherwise a late repetition looks like a new operation. A fingerprint of the content can also belong to the identifier.
Characteristics (by guarantee)
Aspect
At-most-once
At-least-once
"Exactly-once"
Can it be lost?
Yes
No (under the agreed conditions)
No, within the scope
Can it repeat?
No
Yes
No, within the scope
What does it ask of the consumer?
Accepting loss
Idempotency
Knowing the boundary of the scope
When it fits
Fast-expiring data
Business commands, notifications
The transactional scope of narrow, closed systems
Common misconceptions
Statement
Clarification
"With a reliable broker there will be no duplication."
With redelivery there will be.
"Exactly-once executes everything exactly once."
Only within the named scope.
"A producer acknowledgement means it was processed."
Only that the broker handled it.
"Acknowledging before processing is always fine."
After a failure the work may be lost.
In short
The producer and the consumer acknowledgement are separate events.
With at-least-once the consumer must be idempotent.
Exactly-once is true only within a named boundary.
The deduplication identifier must be recorded in one transaction with the business modification.
Review questions
Why can a repeated message arrive after a successful database commit?
Where should the transactional boundary of the deduplication record be?
Why does a broker's exactly-once feature not prove an e-mail sent exactly once?
Acknowledgement, Delivery Guarantees and Duplication
Objectives
By the end of the lesson, you can distinguish the producer and the consumer acknowledgement, explain the at-most-once, at-least-once and exactly-once guarantees, and say why the consumer must expect duplication.
Why is it needed?
The reliability of messaging must be interpreted along the whole path: producer, broker, consumer and the business data store. A "reliable broker" does not mean that a business effect happens exactly once.
Two separate acknowledgements
According to the RabbitMQ documentation, a publisher confirm tells the producer that the broker has handled the message (for a durable queue this means it has been written to disk). A consumer acknowledgement relates to the consumer's processing. The documentation considers the automatic acknowledgement mode "unsafe" for work that needs reliable delivery: the message counts as delivered at the moment it is sent. With manual acknowledgement, an unacknowledged message automatically goes back into the queue when the channel or connection closes.
If the consumer stops after the commit but before the acknowledgement, the message may be delivered again while the effect has already happened. If the acknowledgement goes out before the commit, the work may be lost after a failure. The order of acknowledgement is therefore part of the durability plan.
At-most-once, at-least-once, exactly-once
Idempotent consumer
At-least-once delivery can be used safely together with an idempotent consumer (6.5). The consumer can durably record the processed event identifier in one transaction with the business modification (inbox, 5.3), so on redelivery it recognises the already-happened effect. The check and the save must not be two independent steps, because two parallel consumers may both see the message as "new". The retention time of deduplication identifiers should match the time window of repetition, otherwise a late repetition looks like a new operation. A fingerprint of the content can also belong to the identifier.
Characteristics (by guarantee)
Common misconceptions
In short
Review questions
Learning resources