Skip to content

Ask HN: Would you be interested in a course/eBook about scraping and automation?

2 pointsOtmaneBenazzou2 comments
On HN

Hi, I’m a pretty experienced scraper/automator and I was thinking about sharing my knowledge. I originally thought about writing a blog, but I'm not sure that would be the best format to transmit everything I know. Maybe I'll do both.

There would first be a quick introduction to the legacy ways of scraping(i.e GET request+parser), why they’re often not enough today and when can they be still used.

Then I’d delve into more advanced techniques, I usually try to find the cheapest/free solutions to the problems I’m solving but I’ll also compare paid options:

-how to make your scrapers/bots as stealthy as possible

-everything you need to know about fingerprinting

-how to bypass datadome, cloudfare, captchas…

-how to scale to millions of requests

-how to make your own free rotating proxies

-how to do automation on mobile apps and desktops

-how to reverse engineer APIs on the web or mobile

-every tips and tricks I know to scrape better

I may add a section on GPT/LLM, either about how to fasten the development of a scrapers or about how to make self healing scrapers.

Is this a topic that would interest a lot of you?

Would some of you be ready to pay a small amount for an ebook/courses on those subjects?

Thanks

Comments

I would creating a landing page that creates a mailing list to get notified of when the ebook comes out. plus example scripts

Then I would would create blog posts around a hook for each chapter in the book with enough teaser info to want to get the ebook and when new articles are coming out.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.