Bad notifications, sure. But all you've really demonstrated is that you can only handle planned outages. Planning around the unplanned outages are where we earn our stripes (and paychecks).
You have servers running on infrastructure inside a datacenter that you don't control. Surviving reboots and outages needs to be a part of your strategy if these systems are important.
Yup. However, that's hard when you've spent 99% of your budget on overpriced Rackspace servers. Great brand ("nobody got fired for choosing IBM"), but horrible bang for your buck. But hey, if it's not your money, can't blame them for choosing them.
Comments
Bad notifications, sure. But all you've really demonstrated is that you can only handle planned outages. Planning around the unplanned outages are where we earn our stripes (and paychecks).
You have servers running on infrastructure inside a datacenter that you don't control. Surviving reboots and outages needs to be a part of your strategy if these systems are important.
Oh, and there's this: https://github.com/Netflix/SimianArmy/wiki/Chaos-Monkey
Yup. However, that's hard when you've spent 99% of your budget on overpriced Rackspace servers. Great brand ("nobody got fired for choosing IBM"), but horrible bang for your buck. But hey, if it's not your money, can't blame them for choosing them.
Sorry, but if you can do that (spend 99% of budget on servers), then you have bigger problems. :(