Comment on Craigslist reportedly blocks search engines to kill competitorsparentComments−sigmadelta14yThose tags aren't as obvious in interpretation as they first appear. A full write-up is a http://noarchive.net/meta/ for completeness, but in short:NOARCHIVE: ok to index this page, but don't cache the result permanentlyNOFOLLOW: don't follow any links on this pageNote that these two things don't prevent a search engine from visiting a page, indexing it, and providing search results for it.
Comments
Those tags aren't as obvious in interpretation as they first appear. A full write-up is a http://noarchive.net/meta/ for completeness, but in short:
NOARCHIVE: ok to index this page, but don't cache the result permanently
NOFOLLOW: don't follow any links on this page
Note that these two things don't prevent a search engine from visiting a page, indexing it, and providing search results for it.