Skip to content

Comment on MicroGPT-C in pure C hits 10M TPS on Apple M5

Comments

Check out this port of microgpt to C, posted 5 months ago. It got a 2500x speedup over the python version. https://github.com/moebiusV/cugpt

In the README, the C version gets "452× faster than Python", the 2500x one is using OpenBLAS which uses lower level things than what's in C[1].

[1] see eg https://github.com/OpenMathLib/OpenBLAS/tree/develop/kernel/...

Of course, an interpreted language implementation, versus a compiled language designed to port UNIX.

Even if counting that it is basically calling into C or C++ libraries, there is lots of boilerplate that the CPU has to consume, that just won't happen on the pure C implementation.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.