Skip to content

Comment on Ask HN: How do you protect self-hosted websites from LLM scrapers?

Comments

Why do you expect a high level of abuse? Blocking CIDRs of major cloud providers will stop the genuine ChatGPT/Claude traffic. For the other scrapers masquerading as them, just block the agents' user agents.

If you want the LLMs to refuse to scrape your site you could inject lots of encoded trigger words but this would probably affect your SEO score :)

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.