The Parquet libs I’ve used let you read the Arrow representation straight out of a Parquet file.
There’s also an Arrow “blob” format that can be used, but that’s not meant to be used as a full persistence format (as I understand)
The value proposition is that increasing numbers of query/analysis “engines” support the Arrow in memory format which is fantastic for interoperability.
“I need a data frame library like Pandas but I’m not in Python” is no longer an issue.
Comments
The Parquet libs I’ve used let you read the Arrow representation straight out of a Parquet file.
There’s also an Arrow “blob” format that can be used, but that’s not meant to be used as a full persistence format (as I understand)
The value proposition is that increasing numbers of query/analysis “engines” support the Arrow in memory format which is fantastic for interoperability. “I need a data frame library like Pandas but I’m not in Python” is no longer an issue.