The bulk of the cost savings comes from the use of object storage rather than attached disks. This eliminates the inter-AZ networking costs associated with Kafka replication mechanism.
So basically Kafka, to provide availability guarantees, requires multi-AZ and the inter-AZ replication gets expensive. And Ursa avoids that by using object storage and probably then just talking inter-AZ?
And while I like Kafka, nobody would claim it likes being scaled up and down dynamically, so probably built-in tolerance for that as well? We ran Kafka on-prem so that wasn't an issue for us, and given the nature of the service, didn't have a lot of usage variance.
Comments
The bulk of the cost savings comes from the use of object storage rather than attached disks. This eliminates the inter-AZ networking costs associated with Kafka replication mechanism.
I break all of the costs down in the following e-book. https://streamnative.io/ebooks/reducing-kafka-costs-with-lea...
So basically Kafka, to provide availability guarantees, requires multi-AZ and the inter-AZ replication gets expensive. And Ursa avoids that by using object storage and probably then just talking inter-AZ?
And while I like Kafka, nobody would claim it likes being scaled up and down dynamically, so probably built-in tolerance for that as well? We ran Kafka on-prem so that wasn't an issue for us, and given the nature of the service, didn't have a lot of usage variance.
This: https://www.youtube.com/watch?v=bb-_4r1N6eg was an interesting watch, btw.
Yes, see WarpStream, Confluent Freight and Diskless Kafka for other examples