Comment on Ask HN: Any interesting traffic patterns in your llms.txt?Comments−reconnecting5moNot at all, only web crawlers from hetzner network.https://news.ycombinator.com/item?id=46058273−jdauriemmaOP5moI have seen something similar in a site I work on−reconnecting5moThere (1) is more context on this topic, where author of LLMs.txt idea said 'llms.txt files have nothing to do with crawlers or big LLM companies. They are for individual client agents to use.'1. https://news.ycombinator.com/item?id=47058870
Comments
Not at all, only web crawlers from hetzner network.
https://news.ycombinator.com/item?id=46058273
I have seen something similar in a site I work on
There (1) is more context on this topic, where author of LLMs.txt idea said 'llms.txt files have nothing to do with crawlers or big LLM companies. They are for individual client agents to use.'
1. https://news.ycombinator.com/item?id=47058870