There is no legal restriction for crawling any web site. But you should respect robot.txt document for every single web site. Such as: https://www.facebook.com/robots.txt http://web.archive.org/robots.txt