Comment on Googlebot's recent improvements might revolutionize web developmentparentComments−workhere-ioOP12yBut, have Yahoo or Bing or DuckDuckGo made the transition to be able to crawl the web with a full JS & DOM rendering engine?They can just use PhantomJS (http://phantomjs.org/), which is free and open source.−voltagex_12yThey could, but I wonder what it'd take to scale it to crawling that number of pages.I think only Bing would have the cash and resources to build that.
Comments
But, have Yahoo or Bing or DuckDuckGo made the transition to be able to crawl the web with a full JS & DOM rendering engine?
They can just use PhantomJS (http://phantomjs.org/), which is free and open source.
They could, but I wonder what it'd take to scale it to crawling that number of pages.
I think only Bing would have the cash and resources to build that.