DuckDB, for those who don't know it, has a great extension system, for example it can directly use OLTP databases such as PostgreSQL, MySQL, SQLite, SQL Server; cloud data warehouses/data lakes and big data formats (Iceberg, Delta, Snowflake, Hive, ORC, Parquet, AVRO), other data sources (ODBC), storage (S3), and much much more.
It will certainly allow you to automate tons of stuff, such as data movement between those various sources, data transformations (ETL/ELT) and even simple data conversions - such as from CSV to Parquet, or from ORC to JSON etc. etc. Happy quacking!
Comments
DuckDB, for those who don't know it, has a great extension system, for example it can directly use OLTP databases such as PostgreSQL, MySQL, SQLite, SQL Server; cloud data warehouses/data lakes and big data formats (Iceberg, Delta, Snowflake, Hive, ORC, Parquet, AVRO), other data sources (ODBC), storage (S3), and much much more.
https://duckdb.org/docs/current/core_extensions/overview
https://duckdb.org/community_extensions/list_of_extensions
Nice tip, thanks. I may now make Duck CLI the sole gateway to all the databases I have to use at work.
It will certainly allow you to automate tons of stuff, such as data movement between those various sources, data transformations (ETL/ELT) and even simple data conversions - such as from CSV to Parquet, or from ORC to JSON etc. etc. Happy quacking!