I typically don't tell other people how to run their businesses, but if a similar issue brought my website down and I were to post about the causes, I might focus more on my failures in capacity planning, vendor selection, and monitoring rather than on my vendor's lackluster customer service. User-visible failures are, ultimately, process failures on my part, regardless of the surface cause. A nice side effect of this philosophy is that improvements to my processes help with all sorts of surface causes whereas if I were to address surface causes individually it would be like playing whack-a-mole. Bad vendor whack, hard drive failed whack, traffic spike whack, poor customer service whack, out of memory exception whack whack whack -- why is everyone conspiring to keep me from getting any work done.
It didn't bring HN down. HN deliberately didn't rely on that server for anything except hosting static content that was also duplicated on this server. I planned in advance for the possibility that the other server wouldn't be usable, by writing the code so that I could switch to serving the same content off news by changing one variable, which I did. As a result service was barely affected.
In short, Pair flaked, but we had in fact planned the system in a way that protected us against it.
Comments
I typically don't tell other people how to run their businesses, but if a similar issue brought my website down and I were to post about the causes, I might focus more on my failures in capacity planning, vendor selection, and monitoring rather than on my vendor's lackluster customer service. User-visible failures are, ultimately, process failures on my part, regardless of the surface cause. A nice side effect of this philosophy is that improvements to my processes help with all sorts of surface causes whereas if I were to address surface causes individually it would be like playing whack-a-mole. Bad vendor whack, hard drive failed whack, traffic spike whack, poor customer service whack, out of memory exception whack whack whack -- why is everyone conspiring to keep me from getting any work done.
It didn't bring HN down. HN deliberately didn't rely on that server for anything except hosting static content that was also duplicated on this server. I planned in advance for the possibility that the other server wouldn't be usable, by writing the code so that I could switch to serving the same content off news by changing one variable, which I did. As a result service was barely affected.
In short, Pair flaked, but we had in fact planned the system in a way that protected us against it.
One obvious point is that HN is a peripheral part of YC's business, mostly aimed at recruiting, rather than something that's critical.