Comment on Use Node.js to Extract Data from the WebComments−zerni13ynice!I did a webcrawler with node.js myself last year. It's only a quick try but you can find the worker class here: https://gist.github.com/zerni/6337067Unfortunately jsdom had a memory leak so the crawler died after a while...−cheeaun13yIf you want to fix the memory leak, I remember you need to do `window.close()` after the job is done.−zerni13ythanks mate!
Comments
nice!
I did a webcrawler with node.js myself last year. It's only a quick try but you can find the worker class here: https://gist.github.com/zerni/6337067
Unfortunately jsdom had a memory leak so the crawler died after a while...
If you want to fix the memory leak, I remember you need to do `window.close()` after the job is done.
thanks mate!