Comment on Ask HN: Can I get in trouble for crawling using the Googlebot user agent?Comments−Edmond9yYou won't get in trouble but if the site uses products from the like of Akamai (Bot Manager) or Shape Security then you'll probably be blocked.
Comments
You won't get in trouble but if the site uses products from the like of Akamai (Bot Manager) or Shape Security then you'll probably be blocked.