Comment on Databricks open-sources Delta Lake to make data lakes more reliableparentComments−mobileexpert7yYou can see some related but not directly comparable efforts in HuDI see https://eng.uber.com/uber-big-data-platform/Or https://iceberg.apache.org/Which both keep track of data versioning and management of file based datasets on object stores.Most people today have very ad how approaches to handling data versioning and lineage on Hadoop datasets.
Comments
You can see some related but not directly comparable efforts in HuDI see https://eng.uber.com/uber-big-data-platform/
Or https://iceberg.apache.org/
Which both keep track of data versioning and management of file based datasets on object stores.
Most people today have very ad how approaches to handling data versioning and lineage on Hadoop datasets.