Comment on LeftoverLocals: Listening to LLM responses through leaked GPU local memoryparentComments−dist-epoch2yIt's not so simple.For example Windows takes over GPU memory control, it virtualizes it, and allocates it to various applications, zeros it, etc...−ks6g10OP2yThis is specific to local/scratch memory which is not exposed for allocation in the same way dram is.
Comments
It's not so simple.
For example Windows takes over GPU memory control, it virtualizes it, and allocates it to various applications, zeros it, etc...
This is specific to local/scratch memory which is not exposed for allocation in the same way dram is.