Comment on 102TB of New Crawl Data AvailableparentComments−wpietri12yDid you try this?http://commoncrawl.org/get-started/I haven't tried that one, but I've poked at other of the Amazon Common Datasets collection:http://aws.amazon.com/datasetsIf you're already familiar with using Amazon's virtual servers, it's pretty straightforward.I also note that the Common Crawl project publishes code here:https://github.com/commoncrawl/commoncrawl
Comments
Did you try this?
http://commoncrawl.org/get-started/
I haven't tried that one, but I've poked at other of the Amazon Common Datasets collection:
http://aws.amazon.com/datasets
If you're already familiar with using Amazon's virtual servers, it's pretty straightforward.
I also note that the Common Crawl project publishes code here:
https://github.com/commoncrawl/commoncrawl