May violate a site's TOS... but I don't think you'd ever get in any real trouble... most you'd get is a cease and desist letter... have to waste some time with lawyers... But I think it's on them to block you at the IP level if you are violating the TOS / causing them grief. And look... if a crawl causes them grief then they really need to invest more in DevOps. (Please do what you can to encourage more companies to invest in DevOps!)
"I left my door unlocked and told my friends they could use my living room, but then they put their feet up on my coffee table... Not cool, man!" Pretty much the equivalent situation.
Comments
May violate a site's TOS... but I don't think you'd ever get in any real trouble... most you'd get is a cease and desist letter... have to waste some time with lawyers... But I think it's on them to block you at the IP level if you are violating the TOS / causing them grief. And look... if a crawl causes them grief then they really need to invest more in DevOps. (Please do what you can to encourage more companies to invest in DevOps!)
"I left my door unlocked and told my friends they could use my living room, but then they put their feet up on my coffee table... Not cool, man!" Pretty much the equivalent situation.