One of my favorite papers! This reminds me of Martin Kleppmann's work on Apache Samza and the idea of "turning the database inside out" by hosting the write-ahead log on something like Kafka and then having many different materialized views consume that log.
Seems like a very powerful architecture that is both simple and decouples many concerns.
In their 1992 Transaction Processing book*, Gray and Reuter extrapolate h/w and s/w trends forward and predict that the DBMS of their far future would look like a tape robot for backing store with materialised views in main memory.
Substitute streams for tape i/o, and this description of Samza sounds like it could be very similar to that vision.
* as far as I know, their exposition of the WAL and tradeoffs in its implementation has aged well. Any counter opinions?
Comments
One of my favorite papers! This reminds me of Martin Kleppmann's work on Apache Samza and the idea of "turning the database inside out" by hosting the write-ahead log on something like Kafka and then having many different materialized views consume that log.
Seems like a very powerful architecture that is both simple and decouples many concerns.
In their 1992 Transaction Processing book*, Gray and Reuter extrapolate h/w and s/w trends forward and predict that the DBMS of their far future would look like a tape robot for backing store with materialised views in main memory.
Substitute streams for tape i/o, and this description of Samza sounds like it could be very similar to that vision.
* as far as I know, their exposition of the WAL and tradeoffs in its implementation has aged well. Any counter opinions?
Thanks!