Skip to content

Comment on Robots.txt Disallow: 20 Years of Mistakes To Avoidparent

Comments

If you want to hide admin pages, add the robots meta tag to each one and set noindex, nofollow. Then you don't need to list them all in one place in robots.txt.

That said, obscurity is not really security. Your admin pages should be behind a password, which, if coded properly, will exclude spiders, bots, and bad guys.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.