Skip to content

PHP Parallel Web Scraper

perplexedlabs.com
11 pointsmatticakes2 comments
On HN

Comments

Does this support HTTP pipelining? I've been looking for a way to efficiently retrieve several pages from the same sever simultaneously, and from what I've read pipelining is the best way to go; I just don't know if it's possible in PHP.

HTTP pipelining support was added in curl and libcurl 7.16.0:

http://freshmeat.net/projects/curl/?branch_id=1612&relea...

So it should work with PHP as well.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.