Comment on Applebot, the web crawler for AppleparentComments−MCRed11yIs there legal backing to enforce obeying robots.txt, or is it a guideline?−WorldWideWayne11yThere are some court cases that involve robots.txt but I'm not sure because there was a lot of reading to do and I was hoping someone who knew would just come along and summarize it for us :)
Comments
Is there legal backing to enforce obeying robots.txt, or is it a guideline?
There are some court cases that involve robots.txt but I'm not sure because there was a lot of reading to do and I was hoping someone who knew would just come along and summarize it for us :)