That is, Snowflake follows a traditional data warehouse workflow that requires extenral ETL process to load data into the warehouse. Some of our customers did researching of Snowflake and noticed that their event streaming ingestion can not fit in Snowflake's stage-based loading model - they need real-time insights end-to-end.
Apart from this major downside, about leveraging S3 as a primary storage, Snowflake doesn't have adaptive indexes, and its performance would be significantly degraded as data grows and queries involve a large range of data + multi-condition filters when the simple minmax index can't help.
Comments
We have discussed these in previous blogs:
1. Insight In No Time: https://www.scopedb.io/blog/insight-in-no-time
2. Manage Data in Petabytes for an Observability Platform: https://www.scopedb.io/blog/manage-observability-data-in-pet...
That is, Snowflake follows a traditional data warehouse workflow that requires extenral ETL process to load data into the warehouse. Some of our customers did researching of Snowflake and noticed that their event streaming ingestion can not fit in Snowflake's stage-based loading model - they need real-time insights end-to-end.
Apart from this major downside, about leveraging S3 as a primary storage, Snowflake doesn't have adaptive indexes, and its performance would be significantly degraded as data grows and queries involve a large range of data + multi-condition filters when the simple minmax index can't help.