Skip to content

Comment on Local ML inference benchmark: PyTorch vs. llama.cpp vs. the Rust ecosystem

Comments

Author of the benchmark here, as well as Meganeura - one of the tested frameworks. Results are constantly being refined, gaps are filled, but it's hard to get more hardware coverage by myself. Would appreciate someone running it on Intel Arc GPUs, both discreet and integrated. Happy to answer questions about methodology!

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.