Comment on Ubiq: A Scalable and Fault-Tolerant Log Processing Infrastructure (2016)parentComments−EdwardDiego8ySparkSQL doesn’t support gzYes it does. Source - use Spark SQL routinely. You're right that multiple small Gzipped files are not an ideal input source as it'll create a bunch of small tasks, but Spark definitely does support GZ.
Comments
Yes it does. Source - use Spark SQL routinely. You're right that multiple small Gzipped files are not an ideal input source as it'll create a bunch of small tasks, but Spark definitely does support GZ.