I spent a while trying to get mesh-llm running, but none of the installable llama.cpp builds worked with my older gpu. It looks like it should be able to be used to proxy an external llama.cpp service, but I had no luck setting that up either. Seems very cool, but definitely some rough edges.