Comment on Craigslist reportedly blocks search engines to kill competitorsparentComments−danso14y...Do you do any fact checking?I just checked the source:<meta name="robots" content="NOARCHIVE,NOFOLLOW">−rcollamore14yNothing new. That tag has always been in place. It's to prevent using an ad for SEO purposes−sigmadelta14yThose tags aren't as obvious in interpretation as they first appear. A full write-up is a http://noarchive.net/meta/ for completeness, but in short:NOARCHIVE: ok to index this page, but don't cache the result permanentlyNOFOLLOW: don't follow any links on this pageNote that these two things don't prevent a search engine from visiting a page, indexing it, and providing search results for it.
Comments
...Do you do any fact checking?
I just checked the source:
<meta name="robots" content="NOARCHIVE,NOFOLLOW">
Nothing new. That tag has always been in place. It's to prevent using an ad for SEO purposes
Those tags aren't as obvious in interpretation as they first appear. A full write-up is a http://noarchive.net/meta/ for completeness, but in short:
NOARCHIVE: ok to index this page, but don't cache the result permanently
NOFOLLOW: don't follow any links on this page
Note that these two things don't prevent a search engine from visiting a page, indexing it, and providing search results for it.