Skip to content

Comment on Benchmarking Pocket-Scale Inferenceparent

Comments

There are uses for them but as a sibling comment mentions they're currently extra work to target per vendor. If you squint a bit they're all basically stripped down GPU compute elements with everything except the matrix multiplication hardware removed, a few MB of cache, no RAM of their own, and a nonstandard API.

OpenCL and Vulkan are supposed to gain support eventually but given their extremely constrained instruction sets and the wide variance in capabilities between different hardware models I don't imagine they're the sort of thing you'd want to target manually.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.