It seems they haven't heard of robots.txt.