Skip to content

Comment on LeftoverLocals: Listening to LLM responses through leaked GPU local memoryparent

Comments

And Imagination.

Notably Intel and Nvidia were not impacted. I wonder if the security hardening that Google worked on with Nvidia for Stadia helped prevent this

Jeez, I hope it did not require “security hardening” for Nvidia to do something this basic. If these other vendors missed some tiny corner resulting in state leakage, that would be understandable. But, forgetting to clear local memory is just inexcusable.

Imagine a OS forgetting to replace your general purpose registers across context switches. Only a rank incompetent and useless security process would let something like that get all the way through to deployment.

Vendors have consistently ignored multi tenant issues when coding because gaming doesn’t need it and cloud traditionally hasn’t used GPUs all that much.

You’d be surprised by how many security issues exist in GPU drivers

Could probably be that the shared memory (at least in the past) also was used for cache, so the same mechanism that probably sanitizer the cache is/was in play here.

Google used AMD GPUs for Stadia, not NVIDIA.

Eventually for the product. My memory may be faulty but I talked with engineers working on it during development and I’m pretty sure the initial development was on Nvidia.

Could imagine it being related to CUDA as well. Memory being consistently zero-initialized helps prevent application bugs.

Maybe but multi tenant GPU use cases only really come up for cloud and cloud GPU popularity is only a little more recent.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.