Comment on Robots.txt Disallow: 20 Years of Mistakes To AvoidparentComments−Istof12y"Some sites try to communicate with Google through comments in robots.txt"I thought that was the whole point of robots.txt−lmm12yNo, the point is to communicate with Google through non-comments in robots.txt.
Comments
"Some sites try to communicate with Google through comments in robots.txt"
I thought that was the whole point of robots.txt
No, the point is to communicate with Google through non-comments in robots.txt.