In both examples 2 and 3, the author reads the same stream twice independently and assumes that a join is not synchronized between the transformed streams. This seems like a fundamental flaw in their offering.
Pushing in a timestamp along with the max/variance change stream[1]. And then using the timestamp to synchronize the join[2] would naturally produce a consistent output stream.
I quoted flink because they have the best docs around. But it should be possible in most streaming systems.
Disclaimer, I used to work for the fb streaming group and have collaborated with the flink team very briefly.
The aim of the examples is to show what goes wrong in eventually consistent systems where it's possible that two reads of a stream may not be consistent with respect to each other. The examples are not intended to say that such anomalies can't be fixed by providing stronger consistency guarantees by using timestamps.
Comments
In both examples 2 and 3, the author reads the same stream twice independently and assumes that a join is not synchronized between the transformed streams. This seems like a fundamental flaw in their offering.
Pushing in a timestamp along with the max/variance change stream[1]. And then using the timestamp to synchronize the join[2] would naturally produce a consistent output stream.
I quoted flink because they have the best docs around. But it should be possible in most streaming systems. Disclaimer, I used to work for the fb streaming group and have collaborated with the flink team very briefly.
[1] https://ci.apache.org/projects/flink/flink-docs-stable/dev/t...
[2] https://ci.apache.org/projects/flink/flink-docs-release-1.11...
The aim of the examples is to show what goes wrong in eventually consistent systems where it's possible that two reads of a stream may not be consistent with respect to each other. The examples are not intended to say that such anomalies can't be fixed by providing stronger consistency guarantees by using timestamps.