Comment on LeftoverLocals: Listening to LLM responses through leaked GPU local memoryComments−declaredapple2yGPU memory isolation is generally really bad, from between processes and entire virtual machines.Does anyone know if nvidia's virtual gpus improve the isolation at all?−jeroenhd2yThis is one of my main concerns with technologies like WebGPU. Luckily, WebGPU seems to sacrifice some performance to keep attacks like these from working: https://github.com/trailofbits/LeftoverLocalsRelease/tree/ma...−hhh2yIIRC it only improves if you use MIG.−declaredapple2ySo I guess only the A30/A100/H100 then?
Comments
GPU memory isolation is generally really bad, from between processes and entire virtual machines.
Does anyone know if nvidia's virtual gpus improve the isolation at all?
This is one of my main concerns with technologies like WebGPU. Luckily, WebGPU seems to sacrifice some performance to keep attacks like these from working: https://github.com/trailofbits/LeftoverLocalsRelease/tree/ma...
IIRC it only improves if you use MIG.
So I guess only the A30/A100/H100 then?