I'm a bit surprised the DuckDB file size is that large (5x Parquet, and nearly as large as sqlite). I haven't used DuckDB, but I thought it had columnar storage and so would have better compression.
DuckDB developer here, compression is still a work in progress and is being incrementally implemented and added to the system. For now the storage is still mostly uncompressed, but expect this to improve in the coming months.
Comments
I'm a bit surprised the DuckDB file size is that large (5x Parquet, and nearly as large as sqlite). I haven't used DuckDB, but I thought it had columnar storage and so would have better compression.
DuckDB developer here, compression is still a work in progress and is being incrementally implemented and added to the system. For now the storage is still mostly uncompressed, but expect this to improve in the coming months.