I once read a Twitter thread about the idea of an explicitly flaky AWS region.
It honestly sounds like it would be a really interesting itch to scratch - VMs with explicitly bad network, bad disk, etc. You could specify the severity and randomness of failures, or what days you want things to crash. You could even go nuts and corrupt memory, or randomly trigger GPFs or fault random instructions.
The best part is that the corruption could be cooperative, such that the initial handshakes of TLS connections always succeed, and things like database connections always establish correctly, and PID 1 never faults. So you always end up crashing somewhere obscure, infuriatingly annoying, and very interesting.
I understand that both NFS and rsync-without-SSH don't carry any form of authenticity checksum (IIUC NFS encryption does provide this if you set up Kerberos). Would be fun to corrupt random packets so they get the same TCP+IP checksum, eh...? ;)
And you could even go absolutely ballistic and borrow techniques from fuzzers like AFL to observe and trace long-running executions, and compute ways to trip a program up to get maximally divergent results. I wonder if you could build an inference model for that...
Comments
I once read a Twitter thread about the idea of an explicitly flaky AWS region.
It honestly sounds like it would be a really interesting itch to scratch - VMs with explicitly bad network, bad disk, etc. You could specify the severity and randomness of failures, or what days you want things to crash. You could even go nuts and corrupt memory, or randomly trigger GPFs or fault random instructions.
The best part is that the corruption could be cooperative, such that the initial handshakes of TLS connections always succeed, and things like database connections always establish correctly, and PID 1 never faults. So you always end up crashing somewhere obscure, infuriatingly annoying, and very interesting.
I understand that both NFS and rsync-without-SSH don't carry any form of authenticity checksum (IIUC NFS encryption does provide this if you set up Kerberos). Would be fun to corrupt random packets so they get the same TCP+IP checksum, eh...? ;)
And you could even go absolutely ballistic and borrow techniques from fuzzers like AFL to observe and trace long-running executions, and compute ways to trip a program up to get maximally divergent results. I wonder if you could build an inference model for that...
Yeah, this sounds unreasonably fun.