I use to think small models needed more handholding but I think it’s more a question of age. Newer models do fine with less handholding regardless of size.
Yesterday for fun I tried Ornith 9G on my 16GB M1 with OpenCode. It took 3.5 hours do so what Kimi does in a minute (prompt: “analyze this codebase front end and back end and explain the auth mechanism”), but the subtlety and nuance it showed in its response was very surprising to me - I would have lost the bet if someone asked me if a 9G model could do this.
it is indeed surprising how much a small model can do! As you said, for specific tasks like coding, the training data quality and new architecture of newer models probably beats model size
Comments
I use to think small models needed more handholding but I think it’s more a question of age. Newer models do fine with less handholding regardless of size.
Yesterday for fun I tried Ornith 9G on my 16GB M1 with OpenCode. It took 3.5 hours do so what Kimi does in a minute (prompt: “analyze this codebase front end and back end and explain the auth mechanism”), but the subtlety and nuance it showed in its response was very surprising to me - I would have lost the bet if someone asked me if a 9G model could do this.
it is indeed surprising how much a small model can do! As you said, for specific tasks like coding, the training data quality and new architecture of newer models probably beats model size