I'm falling behind on security updates because I built an all-in-one box like this. While I mostly agree with you, now I wish I had separated out the db on day 1 into a private network so I could maintain a stateless app tier where I can update the public-facing OS image with no downtime or risk.
You might want to check with your hosting provider. I know with Linode I can "clone" my running server, do the upgrades on it, and then swap the IP addresses of the clone and the original (so now the clone is my main server). It's not the most convenient process, but it should work with minimal downtime.
You could do the same thing without the IP swap trick using DNS, it just take a bit longer to propagate. The real catch is what happens to data that was added or changed between when you clone the server to when you make the switch? Ideally you'd want to put your site in a readonly mode if possible.
I think rolling release distros are your friend for these type of all-in-one setups. Small weekly updates. Easy to test too because you only need one test machine.
I think rolling release distros are your friend for these type of all-in-one setups. Small weekly updates. Easy to test too because you only need one test machine.
I think the important thing here is a distro that tests it's changes well, and one that doesn't force you into a major upgrade (where you have to change your configs) very often.
Releases like Debian that want you to do a rolling major upgrade every two years, I think, are more difficult to deal with, because the major upgrades, if you have anything at all custom and the config file format changed, are going to require work and testing for you to move the configs over, even if the developers test perfectly (and nothing is perfect.)
I think RHEL/CentOS is best, assuming that the latest RHEL/CentOS supports all the packages in-distro. (If you need to step outside of the distro repos, that kind of defeats the point. maintaining a package yourself on an ancient distro gets old fast, and most of the smaller 3rd party repos don't put as much effort into keeping the old package versions patched up.)
That's the thing, sure, you have to format and re-install for a major upgrade, but you have ten years before you have to worry about that.
Comments
I'm falling behind on security updates because I built an all-in-one box like this. While I mostly agree with you, now I wish I had separated out the db on day 1 into a private network so I could maintain a stateless app tier where I can update the public-facing OS image with no downtime or risk.
You might want to check with your hosting provider. I know with Linode I can "clone" my running server, do the upgrades on it, and then swap the IP addresses of the clone and the original (so now the clone is my main server). It's not the most convenient process, but it should work with minimal downtime.
You could do the same thing without the IP swap trick using DNS, it just take a bit longer to propagate. The real catch is what happens to data that was added or changed between when you clone the server to when you make the switch? Ideally you'd want to put your site in a readonly mode if possible.
I think rolling release distros are your friend for these type of all-in-one setups. Small weekly updates. Easy to test too because you only need one test machine.
the one server vs many servers debate aside,
I think the important thing here is a distro that tests it's changes well, and one that doesn't force you into a major upgrade (where you have to change your configs) very often.
Releases like Debian that want you to do a rolling major upgrade every two years, I think, are more difficult to deal with, because the major upgrades, if you have anything at all custom and the config file format changed, are going to require work and testing for you to move the configs over, even if the developers test perfectly (and nothing is perfect.)
I think RHEL/CentOS is best, assuming that the latest RHEL/CentOS supports all the packages in-distro. (If you need to step outside of the distro repos, that kind of defeats the point. maintaining a package yourself on an ancient distro gets old fast, and most of the smaller 3rd party repos don't put as much effort into keeping the old package versions patched up.)
That's the thing, sure, you have to format and re-install for a major upgrade, but you have ten years before you have to worry about that.
It might make sense to run nginx on a separate server / vps just for better security.
Docker is the answer to your challenges, IMO.
Is Docker production ready?