Comment on AMD Unveils Its First Small Language Model AMD-135MComments−craftkiller1yI see multiple mentions of NPU on this page, but its still not clear to me: is this something that can finally use the NPU on my processor?−lhl1yThere's actually seems to be a bunch of stuff now:* https://github.com/amd/RyzenAI-SW - has a list of demos and how to use it directly (including apparently w/ PyTorch and LLMs)* https://github.com/huggingface/optimum-amd - can use RyzenAI to use the NPU for HF transformersThere's now a Linux driver even https://github.com/amd/xdna-driver although it looks like a sufficiently PITA that I haven't even bothered to try it (my 7940HS only has like 10 TOPS anyway, so not much point even if it worked perfectly).
Comments
I see multiple mentions of NPU on this page, but its still not clear to me: is this something that can finally use the NPU on my processor?
There's actually seems to be a bunch of stuff now:
* https://github.com/amd/RyzenAI-SW - has a list of demos and how to use it directly (including apparently w/ PyTorch and LLMs)
* https://github.com/huggingface/optimum-amd - can use RyzenAI to use the NPU for HF transformers
There's now a Linux driver even https://github.com/amd/xdna-driver although it looks like a sufficiently PITA that I haven't even bothered to try it (my 7940HS only has like 10 TOPS anyway, so not much point even if it worked perfectly).