Comment on Running local LLMs offline on a ten-hour flightparentComments−jonaustin4mohttps://github.com/jundot/omlxnote: 27b is going to be slow; use the 35b MoE if you want decent token/sec speed.−dexterlagan4moMany of us tested 27B and 35B side by side, and the dense model is significantly smarter. It indeed is slower, but 35B makes a lot of mistakes 27B doesn't.
Comments
https://github.com/jundot/omlx
note: 27b is going to be slow; use the 35b MoE if you want decent token/sec speed.
Many of us tested 27B and 35B side by side, and the dense model is significantly smarter. It indeed is slower, but 35B makes a lot of mistakes 27B doesn't.