Filesystems are nothing more than specialized databases but they don't expose the necessary interface to use them as such.
Most FS are not even transactional, so they can't expose the necessary interfaces (hell, most individual FS calls are not guaranteed to be transactional).
I assume you could build an application-level transactional system on ZFS, but I don't know that it exposes any such APIs either.
My point is that we need a new storage interface that is transactional. It's extremely hard to build a sound interface on the existing ones. And it would not be built on the existing FS because those are already built for the broken interfaces.
You don't need a new hardware interface. You only need a sensible API for write barriers. In particular, you want "semi-permeable" barriers, which is just another way of saying tagged commands, or grouped writes.
1) An fcntl() to set a numerical tag ID on an fd. All subsequent writes on the fd belong to this ID group.
2) No writes in a tagged group are allowed to proceed until all writes from a preceding group are completed.
3) Any untagged writes can complete at any time in any order, as usual.
Done. That's all you need to implement safe filesystem transactions, while eliminating the bottleneck of fsync().
And my point is that you’re working from the wrong end: you can’t have a transactional storage interface when you don’t have a widely available and deployed transactional storage.
It’s easy to build a non-transactional interface atop a transactional storage, but if you don’t have the latter the former makes no sense.
When you say transactional storage do you mean hardware support for transactions? I don't think that would be required. As long as the storage hardware interface (e.g. NVME) guarantees certain low level primitives like atomic sector writes.
I guess what you'd want from hardware is similar to what you get with RAM. Atomic reads and writes of certain size plus read and write barriers.
And then ditch the posix FS APIs, replace with a transactional interface that probably uses io_uring under the hood and build an FS that handles it in a correct and performant way.
Most CPU architectures provide atomic read, write and barrier primitives. Locking is a higher level interface and usually involves use of those atomic primitives.
The point is that once you have these low level guarantees, you can actually implement a good transactional system on top.
Historically what we got from storage in terms of guarantees was just "once you fsync your data will probably persist if you did it very carefully but not always".
Right, you're pretty much on the money: you want transactional storage to build upon.
If NVME or flash interfaces worked reliably as you suggest, then you could do this today by just treating them as memory, but right now there is an fsync (or equivalent) somewhere in the call stack. And even then, hardware complexities are wildly abstracted to the point where a lot of storage devices have no guarantees in the face of e.g. power loss.
Older hardware actually excels here! Tape can be used very, very reliably because the interface is simple and obvious in comparison to flash-based storage.
Comments
Most FS are not even transactional, so they can't expose the necessary interfaces (hell, most individual FS calls are not guaranteed to be transactional).
I assume you could build an application-level transactional system on ZFS, but I don't know that it exposes any such APIs either.
My point is that we need a new storage interface that is transactional. It's extremely hard to build a sound interface on the existing ones. And it would not be built on the existing FS because those are already built for the broken interfaces.
You don't need a new hardware interface. You only need a sensible API for write barriers. In particular, you want "semi-permeable" barriers, which is just another way of saying tagged commands, or grouped writes.
Done. That's all you need to implement safe filesystem transactions, while eliminating the bottleneck of fsync().And my point is that you’re working from the wrong end: you can’t have a transactional storage interface when you don’t have a widely available and deployed transactional storage.
It’s easy to build a non-transactional interface atop a transactional storage, but if you don’t have the latter the former makes no sense.
When you say transactional storage do you mean hardware support for transactions? I don't think that would be required. As long as the storage hardware interface (e.g. NVME) guarantees certain low level primitives like atomic sector writes.
I guess what you'd want from hardware is similar to what you get with RAM. Atomic reads and writes of certain size plus read and write barriers.
And then ditch the posix FS APIs, replace with a transactional interface that probably uses io_uring under the hood and build an FS that handles it in a correct and performant way.
RAM doesn’t really provide atomic reads and writes without additional semantics in most cases? (Locking)
Most CPU architectures provide atomic read, write and barrier primitives. Locking is a higher level interface and usually involves use of those atomic primitives.
The point is that once you have these low level guarantees, you can actually implement a good transactional system on top.
Historically what we got from storage in terms of guarantees was just "once you fsync your data will probably persist if you did it very carefully but not always".
Right, you're pretty much on the money: you want transactional storage to build upon.
If NVME or flash interfaces worked reliably as you suggest, then you could do this today by just treating them as memory, but right now there is an fsync (or equivalent) somewhere in the call stack. And even then, hardware complexities are wildly abstracted to the point where a lot of storage devices have no guarantees in the face of e.g. power loss.
Older hardware actually excels here! Tape can be used very, very reliably because the interface is simple and obvious in comparison to flash-based storage.