I feel like what happens is first they release a giant model. Then they start optimizing the model to increase inference speed and reduce costs.

But then they introduce bugs leading to lots of complaints. Usually the complaints are about models being dumbed down but almost always the labs say they are doing no such thing.

So I am thinking if we believe the labs then they have a very error prone optimization effort going on.