However, I wonder whether the GPU's are a good fit for this to begin with.
I think the GPU could be a great fit for OLAP, but when it comes to the nasty OLTP use cases the CPU will absolutely dominate.
Strictly serialized transaction processing facilities demand extremely low latency compute to achieve meaningful throughput. When the behavior of transaction B depends on transaction A being fully resolved, there are no magic tricks you can play anymore.
Consider that talking to L1 is at least 1,000x faster than talking to the GPU. Unless you can get a shitload of work done with each CPU-GPU message (and it is usually the case that you can), this penalty is horrifyingly crippling.
I think, TrueTime would constitute a "trick," insofar ordering is concerned?
Consider that talking to L1 is at least 1,000x faster than talking to the GPU.
This is largely true for "traditional" architectures, but s/GPU/TPU and s/L1/CMEM and suddenly this is no big deal anymore. I'd like Googlers to correct me here, but it seems well in line with classic MapReduce, and probably something that they're doing a lot outside of LLM inference... ads?
Not necessarily! The setup I'm discussing is explicitly non-GPU, and it's not necessarily a TPU either. Any accelerator card with NoC capability will do: the requests are queued/batched from network, trickle through the adjacent compute/network nodes, and written back to network. This is what "compute-in-network" means; the CPU is never involved, main memory is never involved. You read from network, you write to network, that's it. On-chip memory on these accelerators is orders of magnitude larger than L1 (FPGA's are known for low-latency systolic stuff) and the on-package memory is large HBM stacks similar to those you would find in a GPU.
Comments
I think the GPU could be a great fit for OLAP, but when it comes to the nasty OLTP use cases the CPU will absolutely dominate.
Strictly serialized transaction processing facilities demand extremely low latency compute to achieve meaningful throughput. When the behavior of transaction B depends on transaction A being fully resolved, there are no magic tricks you can play anymore.
Consider that talking to L1 is at least 1,000x faster than talking to the GPU. Unless you can get a shitload of work done with each CPU-GPU message (and it is usually the case that you can), this penalty is horrifyingly crippling.
I think, TrueTime would constitute a "trick," insofar ordering is concerned?
This is largely true for "traditional" architectures, but s/GPU/TPU and s/L1/CMEM and suddenly this is no big deal anymore. I'd like Googlers to correct me here, but it seems well in line with classic MapReduce, and probably something that they're doing a lot outside of LLM inference... ads?
How does the information get to & from the GPU in the first place?
If a client wishes to use your GPU-based RDBMS engine, it needs to make a trip through the CPU first, does it not?
Not necessarily! The setup I'm discussing is explicitly non-GPU, and it's not necessarily a TPU either. Any accelerator card with NoC capability will do: the requests are queued/batched from network, trickle through the adjacent compute/network nodes, and written back to network. This is what "compute-in-network" means; the CPU is never involved, main memory is never involved. You read from network, you write to network, that's it. On-chip memory on these accelerators is orders of magnitude larger than L1 (FPGA's are known for low-latency systolic stuff) and the on-package memory is large HBM stacks similar to those you would find in a GPU.
Could you (assuming no care about efficiency)
Send the query to both GPU and CPU pipelines at the same time and use whichever comes back first
Most database query optimizer engines do a few tests to figure out the most pragmatic approach.
GPUs can incur higher failure risks, and thus one will not normally find them in high-reliability roles. =3