I see. So for a large dataset you would still hit a bottleneck, compared to a main mem store which also uses compression. But nonetheless it's a good idea to do certain operations on hot data on the GPU.
Which operations benefit the largest in your experience? I would guess scans with simple predicates and expressions. What about aggregations?
I had not heard of that data structure before, thanks!
Sadly, it appears to be patented and thus of no use to me. http://www.cs.umd.edu/~nick/projects/Dwarf.html I think it's really quite crass and mercantile for UMD to patent basic computer science algorithms and data structures like that.
Comments
I see. So for a large dataset you would still hit a bottleneck, compared to a main mem store which also uses compression. But nonetheless it's a good idea to do certain operations on hot data on the GPU.
Which operations benefit the largest in your experience? I would guess scans with simple predicates and expressions. What about aggregations?
I had investigated aggregates using gpu for building dwarf petacubes (http://www.cs.umd.edu/~nick/projects/Dwarf.pdf), and yes it looks viable.
I had not heard of that data structure before, thanks!
Sadly, it appears to be patented and thus of no use to me. http://www.cs.umd.edu/~nick/projects/Dwarf.html I think it's really quite crass and mercantile for UMD to patent basic computer science algorithms and data structures like that.
Except that hard drives have a much lower transfer rate?
But we're already working on the premise that data is in the main memory, thus the bottleneck being the GPU<->memory transfer rate.