My vision for the operating system of the future is to build a microkernel that exclusively uses a modified ZFS that has a transactional API, that can run Linux drivers in userspace with zero or few changes, and that uses a uniform event API like Windows' handles or Plan 9's file descriptors.
But that modified ZFS is perhaps the most important part, and I would want to make it so `fsync()` on that platform behaves as people would want it to: transactional such that if it succeeds, the data was written, and if it doesn't, that data was not written.
The ability to run Linux drivers is so there isn't a chicken-and-egg problem with drivers. (It would also be nice to implement the POSIX API to solve that part of the chicken-and-egg problem.) The uniform event API is because the current OS API's are difficult to work with. OS's are based around resources and events. The resource API's are pretty good, but the event API's (select(), poll(), epoll(), io_uring, kqueue, WaitForMultipleObjects(), etc.) are still artificially constrained.
That's one of those things that sounds good in theory, but in practice has more to do with hardware than software semantics.
Any end-to-end guarantees pretty much requires control over hardware, which of course would be very interesting to a lot of people, but isn't really what mainstream operating system concern themselves with.
Already the block device hides too much of hardware, and any VFS layer will be even worse. Just because a write is done doesn't mean it's actually on the spinning rust. And even if it is, it could be done in several ways, maybe it was relocated, or parity hasn't been computed, or it's in the queue. Any attempt of abstracting different types of storage will pretty much have to resort to the least bad common denominator here.
I'm not saying it's a bad idea, just that it's not the same problem that operating systems try to solve. But in the post-Optane world maybe block devices are too low level anyway and we'll finally see higher level storage systems.
You're right, for the most part. We have to have hardware that doesn't lie when the data made it to disk, whatever that disk is. And we also need hardware that can do atomic updates to single sectors.
But once we have those things, and people are demanding them more and more, a copy-on-write filesystem should be perfect for emulating transactions, using that single sector atomic update to "commit" the transaction.
It would also be nice to implement the POSIX API to solve that part of the chicken-and-egg problem.
If you already have a microkernel capable of running Linux hardware drivers in userspace, it shouldn't be hard to also run userspace ABIs/personalities interfaces as userspace drivers; NetBSD rump drivers more or less officially support what you're doing, and I suspect you could modify user mode linux (UML) to provide ABI compatibility good enough to run unmodified Linux binaries.
Comments
I agree with you.
My vision for the operating system of the future is to build a microkernel that exclusively uses a modified ZFS that has a transactional API, that can run Linux drivers in userspace with zero or few changes, and that uses a uniform event API like Windows' handles or Plan 9's file descriptors.
But that modified ZFS is perhaps the most important part, and I would want to make it so `fsync()` on that platform behaves as people would want it to: transactional such that if it succeeds, the data was written, and if it doesn't, that data was not written.
The ability to run Linux drivers is so there isn't a chicken-and-egg problem with drivers. (It would also be nice to implement the POSIX API to solve that part of the chicken-and-egg problem.) The uniform event API is because the current OS API's are difficult to work with. OS's are based around resources and events. The resource API's are pretty good, but the event API's (select(), poll(), epoll(), io_uring, kqueue, WaitForMultipleObjects(), etc.) are still artificially constrained.
That's one of those things that sounds good in theory, but in practice has more to do with hardware than software semantics.
Any end-to-end guarantees pretty much requires control over hardware, which of course would be very interesting to a lot of people, but isn't really what mainstream operating system concern themselves with.
Already the block device hides too much of hardware, and any VFS layer will be even worse. Just because a write is done doesn't mean it's actually on the spinning rust. And even if it is, it could be done in several ways, maybe it was relocated, or parity hasn't been computed, or it's in the queue. Any attempt of abstracting different types of storage will pretty much have to resort to the least bad common denominator here.
I'm not saying it's a bad idea, just that it's not the same problem that operating systems try to solve. But in the post-Optane world maybe block devices are too low level anyway and we'll finally see higher level storage systems.
You're right, for the most part. We have to have hardware that doesn't lie when the data made it to disk, whatever that disk is. And we also need hardware that can do atomic updates to single sectors.
But once we have those things, and people are demanding them more and more, a copy-on-write filesystem should be perfect for emulating transactions, using that single sector atomic update to "commit" the transaction.
If you already have a microkernel capable of running Linux hardware drivers in userspace, it shouldn't be hard to also run userspace ABIs/personalities interfaces as userspace drivers; NetBSD rump drivers more or less officially support what you're doing, and I suspect you could modify user mode linux (UML) to provide ABI compatibility good enough to run unmodified Linux binaries.
Yeah, that's part of my idea too; just have different libc's capable of emulating various OS calls.
UML sounds interesting.