Skip to content

Comment on Web scraping with your web browser: Why not?parent

Comments

Well, very much yes and no to that claim. Sure, for someone who’s comfortable in the shell, the first step is lighter with curl or wget. But the next step – parsing — is a lot less obvious. And it’s all much more likely to fail on websites that assume a (logged-in?) user in a browser, accepting cookies, executing JS, and so on.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.