There are a bunch of comments about rolling your own scraper instead of relying upon a possibly unreliable SaaS app.
That makes me think -- would it be viable to run a service that, instead of running the scraping on their own servers, simply gave you a custom binary to run?
Assuming that you trusted the executable, you would never have to worry about the company failing. It'd just be a one-time fee, and yours to use in perpetuity. Presumably updates would be free.
Comments
There are a bunch of comments about rolling your own scraper instead of relying upon a possibly unreliable SaaS app.
That makes me think -- would it be viable to run a service that, instead of running the scraping on their own servers, simply gave you a custom binary to run?
Assuming that you trusted the executable, you would never have to worry about the company failing. It'd just be a one-time fee, and yours to use in perpetuity. Presumably updates would be free.
That's a really neat idea I'd pay for. Not sure about the sustainability of the model though.
if you use scrapy (which is an awesome python scraping framework) you can plug a different third party solutions such as: http://crawlera.com/
Not really server level hosting, but you get the benefits of their network.