Comment on Applebot, the web crawler for AppleComments−beltex11y“It looks like someone at Apple is running a web crawler written in Go.”Likely explains [1] from last year (see [2] for HN thread).[1] http://jan.moesen.nu/2014/11/06/apple-crawler.txt[2] https://news.ycombinator.com/item?id=8567205−_lce011ynice catch!I remember some thread from the Safari's early ages where they hid the user-agent, to avoid raising attention.
Comments
“It looks like someone at Apple is running a web crawler written in Go.”
Likely explains [1] from last year (see [2] for HN thread).
[1] http://jan.moesen.nu/2014/11/06/apple-crawler.txt
[2] https://news.ycombinator.com/item?id=8567205
nice catch!
I remember some thread from the Safari's early ages where they hid the user-agent, to avoid raising attention.