Skip to content

Comment on Google penalizes original content site because of scrapersparent

Comments

Someone wrote about a honey pot trap for robots. They made a new directory and added it to the robots.txt as do-not-enter. All IPs that hit the directory where banned.

If the sites that scrap your content do it from servers with fixed IPs, then you could go hunting and try to block them.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.