I think I'm missing something. Why isn't Amazon sorting this out behind the scenes so that any failing hardware is seamlessly replaced and the user is none the wiser? Am I expecting too much?
EC2 instances don't come with vmotion. It's up to the customer to detect a failed/retired node and restart on another EC2 instance.
The first thing you discover when reading through the various options is that you need to treat ALL local storage like /tmp, subject to deletion at will. Keep your persistent storage on EBS/S3.
And even if you do keep your important stuff on EBS, make sure you take snapshots on a frequent basis. We have received this email a couple of times:
Your volume experienced a failure due to multiple failures of the
underlying hardware components and we were unable to recover it.
Although EBS volumes are designed for reliability, backed by multiple
physical drives, we are still exposed to durability risks caused by
concurrent hardware failures of multiple components, before our systems
are able to restore the redundancy. We publish our durability expectations
on the EBS detail page here (http://aws.amazon.com/ebs).
Sincerely,
EBS Support
Fortunately, we had recent snapshots and it was a matter of (manually) spinning up a new instance from those.
Windows Azure actually does this. If the host your virtual machine is on for some reason fails or needs to be replaced your entire VM is migrated to another host. The migration process can take a few minutes but all your data is safe.
My point being. On this topic AWS could learn from Microsoft on how to do cloud.
The one problem they have is that the majority of their instances include local storage, which would make migration impossible. So the best they can offer is a reboot so the server ends up on another host.
They could potentially do this on their second generation (M3) instances, as well as micro instances if they wanted to. However I'd guess that these instances are just a small percentage of the overall servers used.
The one problem they have is that the majority of their instances include local storage, which would make migration impossible
True for AWS. Using VMware vSphere then this could be done with a shared nothing migration which moves compute and storage (vMotion + storage vMotion combined).
While Xen should make live migrations technically possible, it would probably reduce EC2's provisioning flexibility and introduce undesirable complexity.
Migrations would be restricted to hosts running specific releases of the hypervisor [1], and AWS's SDN systems would need to handle these changes in very-near-realtime.
Comments
I think I'm missing something. Why isn't Amazon sorting this out behind the scenes so that any failing hardware is seamlessly replaced and the user is none the wiser? Am I expecting too much?
EC2 instances don't come with vmotion. It's up to the customer to detect a failed/retired node and restart on another EC2 instance.
The first thing you discover when reading through the various options is that you need to treat ALL local storage like /tmp, subject to deletion at will. Keep your persistent storage on EBS/S3.
And even if you do keep your important stuff on EBS, make sure you take snapshots on a frequent basis. We have received this email a couple of times:
Fortunately, we had recent snapshots and it was a matter of (manually) spinning up a new instance from those.Edit: proper quotation
Windows Azure actually does this. If the host your virtual machine is on for some reason fails or needs to be replaced your entire VM is migrated to another host. The migration process can take a few minutes but all your data is safe.
My point being. On this topic AWS could learn from Microsoft on how to do cloud.
The one problem they have is that the majority of their instances include local storage, which would make migration impossible. So the best they can offer is a reboot so the server ends up on another host.
They could potentially do this on their second generation (M3) instances, as well as micro instances if they wanted to. However I'd guess that these instances are just a small percentage of the overall servers used.
True for AWS. Using VMware vSphere then this could be done with a shared nothing migration which moves compute and storage (vMotion + storage vMotion combined).
Xen also does this, but AWS may still be using a very old fork of Xen 2 or 3.
It's unfortunate that there doesn't seem to be a large-scale consumer for the XenServer platform now that Citrix open-sourced the entire thing.
While Xen should make live migrations technically possible, it would probably reduce EC2's provisioning flexibility and introduce undesirable complexity.
Migrations would be restricted to hosts running specific releases of the hypervisor [1], and AWS's SDN systems would need to handle these changes in very-near-realtime.
[1] wiki.xen.org/wiki/Xen_Version_Compatibility