Some might call this pedantic, but I think physical reality provided us the earliest idempotency keys, which were natural keys in specific systems. Some effects are idempotent because they address physical reality in a way that converges or provides mutual exclusion.
The pigeon-hole principle isn't an instruction guide for birds, it just describes the reality of occupancy. An egg-tooth keeps pecking, escaping at most one egg. A dog may circle, but occupies at most one bed.
The real value of idempotence is being truly end-to-end. It is about the final effect being controlled, rather than about some message hop in the middle. In my opinion, the impulse to isolate it to a messaging layer is often misguided. It presupposes that the other layers don't also have similar failure modes.
It makes for a more robust system when you can elicit the desired idempotence from natural keys and occupancy rules. For example, the way REST can make creation idempotent when the name can be established ahead of the creation request, or deletion is idempotent when names are not reused.
Natural idempotence requires deep understanding of the domain, which developers (and coding agents) do not always have. Also, it's still a limited solution, which doesn't solve the generic case. So it's each much easier to use technical idempotency solutions like:
- stamp every initial request / domain event with the UUID (idempotency key)
- use client-side IDs instead of using server-generated ones with RETURNING
- use PUT instead of POST
- use UPSERTs
- etc.
IMO, you migh as well use CQRS/ES which is a natural evolution of this idea of modeling the physical reality rather than trying to reduce it to CRUD/L
Here, we can draw an analogy between natural hedging and financial hedging (derivatives). Natural hedging requires planning, preparation, and hard work, and even that doesn't cover every scenario, but it's cheap. With derivatives, you pay someone else to take care of it. The other party may not even understand the underlying asset, but they have quant models (a technical solution)
RE: the question about the first mentions of idempotency keys, IMO cryptographic nonces (numbers used only once) are somewhat related. Idempotency keys are used to facilitate message replay, while nonces are used to prevent replay. But the basic idea is similar
I didn't really mean to emphasize CRUD when I mentioned REST. Just that it is an example of a natural mapping for some domains. I'm not a fan of anything trying to solve distribute system problems with only middleware. The end-to-end mapping needs to traverse the messaging layer, but cannot really be made robust when the endpoint are naive to the distributed reality.
A similar analogy is how 2-phase commit protocols enact (virtual) escrow.
It doesn't matter CRUD, REST, 2PC, or middleware in general. From my PoV they're all technical solutions used to avoid deep modeling of the domain and its workflows, and instead trying to model the real world using generic primitives without understanding it first, or maybe with surface-level understanding only
Also, learning data engineering and BI helps one better model the real world than generic methods. It's somewhat similar to CQRS/ES, but with much more prior art
Comments
Some might call this pedantic, but I think physical reality provided us the earliest idempotency keys, which were natural keys in specific systems. Some effects are idempotent because they address physical reality in a way that converges or provides mutual exclusion.
The pigeon-hole principle isn't an instruction guide for birds, it just describes the reality of occupancy. An egg-tooth keeps pecking, escaping at most one egg. A dog may circle, but occupies at most one bed.
The real value of idempotence is being truly end-to-end. It is about the final effect being controlled, rather than about some message hop in the middle. In my opinion, the impulse to isolate it to a messaging layer is often misguided. It presupposes that the other layers don't also have similar failure modes.
It makes for a more robust system when you can elicit the desired idempotence from natural keys and occupancy rules. For example, the way REST can make creation idempotent when the name can be established ahead of the creation request, or deletion is idempotent when names are not reused.
Natural idempotence requires deep understanding of the domain, which developers (and coding agents) do not always have. Also, it's still a limited solution, which doesn't solve the generic case. So it's each much easier to use technical idempotency solutions like:
- stamp every initial request / domain event with the UUID (idempotency key)
- use client-side IDs instead of using server-generated ones with RETURNING
- use PUT instead of POST
- use UPSERTs
- etc.
IMO, you migh as well use CQRS/ES which is a natural evolution of this idea of modeling the physical reality rather than trying to reduce it to CRUD/L
Here, we can draw an analogy between natural hedging and financial hedging (derivatives). Natural hedging requires planning, preparation, and hard work, and even that doesn't cover every scenario, but it's cheap. With derivatives, you pay someone else to take care of it. The other party may not even understand the underlying asset, but they have quant models (a technical solution)
RE: the question about the first mentions of idempotency keys, IMO cryptographic nonces (numbers used only once) are somewhat related. Idempotency keys are used to facilitate message replay, while nonces are used to prevent replay. But the basic idea is similar
I didn't really mean to emphasize CRUD when I mentioned REST. Just that it is an example of a natural mapping for some domains. I'm not a fan of anything trying to solve distribute system problems with only middleware. The end-to-end mapping needs to traverse the messaging layer, but cannot really be made robust when the endpoint are naive to the distributed reality.
A similar analogy is how 2-phase commit protocols enact (virtual) escrow.
It doesn't matter CRUD, REST, 2PC, or middleware in general. From my PoV they're all technical solutions used to avoid deep modeling of the domain and its workflows, and instead trying to model the real world using generic primitives without understanding it first, or maybe with surface-level understanding only
The dichotomy is generic vs custom/bespoke:
- generic (CRUD / REST / RDBMS+SQL+ACID Tx / Tx Scripts / 2PC / BPMN / Admin UIs) - also SmartUIs/Fat Clients (VB, Delphi, SPAs, etc.)
vs
- custom (DDD / CQRS/ES / Sagas / Process Managers / Durable Executions / TBUI)
Also, learning data engineering and BI helps one better model the real world than generic methods. It's somewhat similar to CQRS/ES, but with much more prior art