I/We are also using Hive to process compressed logs, manipulate them and create sub tables, that we can query on. We use Amazon EMR, and storing the data on s3. But mainly we use it for text processing via HQL.(Right or wrong it creates some structure around our data.)
Comments
I/We are also using Hive to process compressed logs, manipulate them and create sub tables, that we can query on. We use Amazon EMR, and storing the data on s3. But mainly we use it for text processing via HQL.(Right or wrong it creates some structure around our data.)
I use it to replace writing tedious map reduce jobs. Once you have it setup, it is very easy to write new jobs very quickly.