Skip to content

The Dataflow Model: Balancing Correctness, Latency, and Cost in Data Processing [pdf]

static.googleusercontent.com
56 pointstimclark12 comments
On HN

Comments

Eric Schmidt is in the list of authors, I wonder if he is doing CS research again as an executive.

There are multiple Eric Schmidt's at Google. That one is not the Chairman/former CEO :)

Wow. Just imagine all the opportunity for social engineering, with a name like that :)

And his work email is cloude@google.com? That is quirky

Wow ... this is awesome! Quoting from the paper, "live and breathe under the assumption that we will never know if or when we have seen all of our data, only that new data will arrive, old data may be retracted, and the only way to make this problem tractable". That's another amazing mindshift!

Fundamentally still operating in an 'anticipatory' [1] model of computing. A trending shade of pink lipstick for the old pig.

[1]: the margin of this post is too small to contain an elaboration on this ;)

Why is there no related work section in this paper? I'm not sure simply calling it 'The' dataflow model is very friendly either to all the other previous dataflow models for parallelism that have been developed over the last four decades or so. Why can this implementation be the definitive one so much that it doesn't even need a qualified name and why aren't any of the others even worth a mention?

IMHO the same thing happened with the "object" concept in the '90s.

The introduction is essentially the related works section.

This is unfortunate---Colyer's summaries are well worth reading.

They're good summaries, but after thinking about this a fair bit, I don't think they're enough to trump HN's preference for original sources. Anyone who wants to can read both, since when we change a URL we include the previous one in the comments, as above.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.