Redis is adding a “Stream” data type that is specifically suited for streams of data and time series storage, at this point the specification is near complete and work to implement it will start in the next weeks.
This sounds like it could be really exciting. Is there anywhere I can find out more?
Specifically, I've been struggling to find an appropriate backend for HTTP Server-Sent Events, could this feature help with that?
Did my enhancement make it into the skip list implementation being used for the STREAM type? I am hoping it would be in place before you publish benchmarks for it.
Hello, very interesting! I missed this, just commented on the issue. The Streams are not based on skiplists, but instead will be implemented using http://github.com/antirez/rax
Thanks for having a look! I only read the early proposals for the data structure behind Streams and haven't had a chance to go over the final implementation. I hope to dive into the source this week and make more contributions down the road!
I'm pretty sure I saw implementations that used the existing publish subscribe mechanism in Redis to handle it and seemed happy with it. I have no personal experience with it though.
Comments
This sounds like it could be really exciting. Is there anywhere I can find out more?
Specifically, I've been struggling to find an appropriate backend for HTTP Server-Sent Events, could this feature help with that?
Hello, please check my two Redis Conf 2017 talks on youtube. There is info about Streams.
Thanks antirez! This looks exactly like the feature I've been searching for. :)
For posterity, here's the referenced videos:
General overview: https://youtu.be/U7J33pd3hLU?t=23m54s
Implementation details: https://youtu.be/Wzy8dIjsY6Y
Did my enhancement make it into the skip list implementation being used for the STREAM type? I am hoping it would be in place before you publish benchmarks for it.
https://github.com/antirez/redis/pull/3889
Hello, very interesting! I missed this, just commented on the issue. The Streams are not based on skiplists, but instead will be implemented using http://github.com/antirez/rax
Thanks for having a look! I only read the early proposals for the data structure behind Streams and haven't had a chance to go over the final implementation. I hope to dive into the source this week and make more contributions down the road!
Here's a discussion on reddit. There's a link to the proposal on github, too.
https://www.reddit.com/r/redis/comments/4mmrgr/stream_data_s...
I'm pretty sure I saw implementations that used the existing publish subscribe mechanism in Redis to handle it and seemed happy with it. I have no personal experience with it though.