Hi, fyi I am working on a from-scratch implementation (currently llama2 on linux focused) in Fortran, CPU only but in my initial test about as fast as llama.cpp. Currently has fp16 and 4-bit quantization and I hope this week to finish support for ggml files - I based it off of Karpathy's llama2.c and so it uses that format now which is not great. Llama.cpp is the leader and has more diverse hardware support, for CPU inference and simplicity (complexity of llama.cpp had exploded) there is still room for competition I believe. https://github.com/rbitr/llama2.f90 Once I make it a bit easier to use I want to promote it more.