Skip to content

Ask HN: Crash course on ETL/data pipeline system design?

3 pointsoopsdiscuss
On HN

Hi HN!

Any recommendations for getting up to speed quickly on data engineering / ETL system design? A course with exercises would be great as would a book/blog with example real-world architectures that has some depth to it, discusses tradeoffs.

My goal is to maintain and eventually improve a system that just fell in my lap. It's using Kinesis/Glue/Parquet/S3/Athena to process telemetry data for hourly reports (non-realtime) and ad-hoc querying. All my experience so far is with rdbms-backed web apps.

Thanks in advance for any links/tips!

Comments

No comments yet.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.