Skip to content

Comment on Haydex: From Zero to 178.6B rows a second in 30 days

Comments

178 billion? That's nothing. I did trillions just this morning. I went to the grocery store and picked an item off the shelf, effectively filtering out the trillions of other products that I could have picked but didn't.

They did not process 178 billion rows per second. They did a search that found something in a large data set by eliminating the parts of the data set that could not have contained the item. Same way I did by picking one grocery store and going straight to the shelf.

Hm, if I understand their product correctly they are building a DB and their filtering actually returns correct results.

So, the analogy doesn't really hold true unless you actually have these trillions of alternate products stored in your brain and manage to cite the matching subset on demand.

Seemingly, their way of thinking goes roughly like this:

If I have 10 billion rows in an SQL database, with a UNIQUE index, and do SELECT * FROM t1 WHERE pk=<number>, then I have “processed” 10 billion rows.

If I do 10k of these queries per second, I have processed 100T rows per second.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.