The idea is lean towards doing no transformation before loading into your Target Database/Datawarehouse. Do all transformations after the data hits your target database. DEV time is expensive, storage and CPU is cheap now. DBs and DWs are blazing fast now.
You can still only pull the data that you need using Meltano, but you select things on a "Stream" / "Attributes" level. For source DBs this means something like Table and Field level is as granular as you get.
Comments
Meltano is a open source platform for automating ELT pipelines.
Is there any significant difference from the traditional ETL?
The idea is lean towards doing no transformation before loading into your Target Database/Datawarehouse. Do all transformations after the data hits your target database. DEV time is expensive, storage and CPU is cheap now. DBs and DWs are blazing fast now.
You can still only pull the data that you need using Meltano, but you select things on a "Stream" / "Attributes" level. For source DBs this means something like Table and Field level is as granular as you get.
I could imagine a scenario where the target server has more capacity and thereby a better place to do the transforming steps.