Comment on Ask HN: Open source focused crawler?Comments−sheraz12yI highly recommend Scrapy (http://www.scrapy.org).From their site:Scrapy is a fast high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.
Comments
I highly recommend Scrapy (http://www.scrapy.org).
From their site:
Scrapy is a fast high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.