At a former SaaS startup my colleague rolled out some updates that affected an apache module's connection pooling persistence defaults.
Everything had worked fine in his testing, but once in production there was enough concurrency to make this change in caching behavior matter. One customer's web login would spuriously see another customer's content, it all seemed completely random.
A similar change (or rare bug) could easily result in one customer's boolean "yes they're allowed" reach another who should have received "no they're not". There's all sorts of opportunities in the backend to cross such streams, especially things that need to scale and do a lot of caching.
Comments
At a former SaaS startup my colleague rolled out some updates that affected an apache module's connection pooling persistence defaults.
Everything had worked fine in his testing, but once in production there was enough concurrency to make this change in caching behavior matter. One customer's web login would spuriously see another customer's content, it all seemed completely random.
A similar change (or rare bug) could easily result in one customer's boolean "yes they're allowed" reach another who should have received "no they're not". There's all sorts of opportunities in the backend to cross such streams, especially things that need to scale and do a lot of caching.
I sure hope that startup learned that dev testing is not sufficient!
Bankrupt after burning >100M of VC money, but from causes far deeper than lack of testing.