A coherent userland (not without occasional warts, but without stuff being deliberately, maliciously ignored, broken or replaced to foster someone's agenda).
ZFS as a first class citizen, without wondering if Oracle will show up and take the ball away.
Jails, although that's not super relevant to the desktop experience.
I won't mention systemd, although I truly dislike it (though I have no particular attachment to init scripts, I was quite fond of Solaris' SMF for example).
The absence of snap and whatever horrors Canonical keeps pushing is another plus, but I'm in not in a good position to judge its impact as I'm blissfully ignorant of how bad they are, I just stumble in the occasional horror story.
Biggest thing I'm missing and wish I had: Docker. Jails may be technically better, but the world went another way and they have no functional ecosystem.
For me, jails aren’t only technically better, but also have better ergonomics. With Docker:
- I always end up running out of disk space, because of Docker’s love of immutable layers for everything — my jails just live in a normal directory in the filesystem, no need to overthink it;
- I have no idea how to restrict /dev in Docker containers — I’m sure you can, but it’s a mystery to me;
- copying/moving files between host and container requires setting up volumes/ssh/whatever special ceremony; copying/moving files to/from jails is trivial, it’s the same as copying/moving files between any other directories;
- updating jails is easy; I use pkg and freebsd-update, just like with a normal system update, no need to update the host system one way, and the jails another way, like with Docker;
- jails don’t mess with my firewall rules, unlike Docker.
Overall, jails give me a much more cohesive experience.
I agree wholeheartedly. I like jails a lot (and jails+ZFS even more) and I'm not a fan of Docker at all. But the entire planet adopted it and isn't even aware jails exist.
You won't find a premade pihole jail, a downloadable HomeAssistant jail, etc. That's what I mean with ecosystem. Being the better solution is surprisingly unimportant when nobody knows you exist.
If I understand correctly, the licensing issue is around mixing GPL software and CDDL software. I don't think that happens in the ZoL tree, but may happen in distributions that ship the kernel and ZoL together.
Using the ZoL tree with FreeBSD doesn't have the same licensing conflict; BSD and CDDL software can mix.
I think ubuntu has a good argument[1] that it can still be shipped as a separate kernel module and satisfy both licenses.
"The CDDL applies to all files under the CDDL, while the GPLv2 applies to derivative works. The CDDL cannot apply to the Linux kernel because zfs.ko is a self-contained file system module — the kernel itself is quite obviously not a derivative work of this new file system."
The FreeBSD installer provides out of the box options to put the root filesystem on an encrypted ZFS partition. The last I looked into doing root-on-encrypted-ZFS on Linux it was a lot more involved.
That seems to be the one advantage separating kernel from user space has provided. At least in my experience, jails may break when the host OS gets updated and the jail isn’t upgraded in lock step. That means having 1+n upgrades for every minor version update.
Requiring a docker container to have a user space which matches the kernel kind of defeats the purpose.
This shouldn't normally happen, generally you can run a new kernel with an older userland (the documented upgrade process for the main root is upgrade kernel, reboot, upgrade userland, so at least part of userland needs to still work), and a chroot jail is mostly an independent userland. You may need to ensure the kernel is compiled with backwards compatability if your userland is very far off, and sometimes you see problems with syscall details: 13.0 released with a bug that some ifconfig arguments wouldn't work with a 12.x binary; and I've run into problems with the cpu pinning syscall when the bitmask size increased (this was recently fixed upstream, although I ran into it years ago), etc, so it's not unheard of, but it's fairly rare, in my experience.
Comments
A coherent userland (not without occasional warts, but without stuff being deliberately, maliciously ignored, broken or replaced to foster someone's agenda).
ZFS as a first class citizen, without wondering if Oracle will show up and take the ball away.
Jails, although that's not super relevant to the desktop experience. I won't mention systemd, although I truly dislike it (though I have no particular attachment to init scripts, I was quite fond of Solaris' SMF for example).
The absence of snap and whatever horrors Canonical keeps pushing is another plus, but I'm in not in a good position to judge its impact as I'm blissfully ignorant of how bad they are, I just stumble in the occasional horror story.
Biggest thing I'm missing and wish I had: Docker. Jails may be technically better, but the world went another way and they have no functional ecosystem.
For me, jails aren’t only technically better, but also have better ergonomics. With Docker:
- I always end up running out of disk space, because of Docker’s love of immutable layers for everything — my jails just live in a normal directory in the filesystem, no need to overthink it;
- I have no idea how to restrict /dev in Docker containers — I’m sure you can, but it’s a mystery to me;
- copying/moving files between host and container requires setting up volumes/ssh/whatever special ceremony; copying/moving files to/from jails is trivial, it’s the same as copying/moving files between any other directories;
- updating jails is easy; I use pkg and freebsd-update, just like with a normal system update, no need to update the host system one way, and the jails another way, like with Docker;
- jails don’t mess with my firewall rules, unlike Docker.
Overall, jails give me a much more cohesive experience.
I agree wholeheartedly. I like jails a lot (and jails+ZFS even more) and I'm not a fan of Docker at all. But the entire planet adopted it and isn't even aware jails exist.
You won't find a premade pihole jail, a downloadable HomeAssistant jail, etc. That's what I mean with ecosystem. Being the better solution is surprisingly unimportant when nobody knows you exist.
Is this still true now that FreeBSD is using the ZoL/OpenZFS tree?
If I understand correctly, the licensing issue is around mixing GPL software and CDDL software. I don't think that happens in the ZoL tree, but may happen in distributions that ship the kernel and ZoL together.
Using the ZoL tree with FreeBSD doesn't have the same licensing conflict; BSD and CDDL software can mix.
I think ubuntu has a good argument[1] that it can still be shipped as a separate kernel module and satisfy both licenses.
"The CDDL applies to all files under the CDDL, while the GPLv2 applies to derivative works. The CDDL cannot apply to the Linux kernel because zfs.ko is a self-contained file system module — the kernel itself is quite obviously not a derivative work of this new file system."
[1] https://ubuntu.com/blog/zfs-licensing-and-linux
The FreeBSD installer provides out of the box options to put the root filesystem on an encrypted ZFS partition. The last I looked into doing root-on-encrypted-ZFS on Linux it was a lot more involved.
Ubuntu 22.04 at least, supports root-on-encrypted-ZFS right from the installer and it worked just fine for me.
I guess my point was the Oracle could still show up and take the ball away -- Linux and FreeBSD are using the same ZFS code base now.
The CDDL license guarantees they can’t. (And it’s probably the only reason they hadn’t.)
That seems to be the one advantage separating kernel from user space has provided. At least in my experience, jails may break when the host OS gets updated and the jail isn’t upgraded in lock step. That means having 1+n upgrades for every minor version update.
Requiring a docker container to have a user space which matches the kernel kind of defeats the purpose.
This shouldn't normally happen, generally you can run a new kernel with an older userland (the documented upgrade process for the main root is upgrade kernel, reboot, upgrade userland, so at least part of userland needs to still work), and a chroot jail is mostly an independent userland. You may need to ensure the kernel is compiled with backwards compatability if your userland is very far off, and sometimes you see problems with syscall details: 13.0 released with a bug that some ifconfig arguments wouldn't work with a 12.x binary; and I've run into problems with the cpu pinning syscall when the bitmask size increased (this was recently fixed upstream, although I ran into it years ago), etc, so it's not unheard of, but it's fairly rare, in my experience.
Amen, brother.. Ubuntu 22 was my breaking point.
The absurdity of snap has gone too far.
Even Certbot for Let's Encrypt instructs you to install certbot from a _snap_.
Yes, let me load this additional package manager/software channel on a server install, that will go super well/s.