It highly likely that this person caused a $72,000 bill on all the websites he has crawled. It's just that the cost is spread out over multiple websites so it is not noticeable.
I don't think so, serving a page from cache is far cheaper than requesting, crawling, and storing that page in a database. Cloud comes with a premium, too.
While that’s true, not everything can be cached, and many websites run expensive code to assign a session to each new “user.” Larger sites generally learn to avoid that or have the infrastructure to accommodate it, but even moderate-sized blogs and forums probably can’t cope with that scenario all too well.
Comments
It highly likely that this person caused a $72,000 bill on all the websites he has crawled. It's just that the cost is spread out over multiple websites so it is not noticeable.
I don't think so, serving a page from cache is far cheaper than requesting, crawling, and storing that page in a database. Cloud comes with a premium, too.
While that’s true, not everything can be cached, and many websites run expensive code to assign a session to each new “user.” Larger sites generally learn to avoid that or have the infrastructure to accommodate it, but even moderate-sized blogs and forums probably can’t cope with that scenario all too well.