Comment on Ask HN: How are you dealing with scraping hits from EC2 machines?parentComments−bdcravens13yNot all scrapers spider a site's contents. Sometimes they are going after something specific.
Comments
Not all scrapers spider a site's contents. Sometimes they are going after something specific.