Comment on Smaller, faster, safer: running Kimi and GLM at scaleparentComments−adoltech1moWorth knowing a cache hit rate can be structurally zero and look identical to a broken one.−mordae1moOf course. On the other hand if you see 8 % gap for similar workloads, averaged across tens of sessions, with the same underlying model, it becomes a pretty clear signal. And I do exclude first request per provider per session from the statistic.
Comments
Worth knowing a cache hit rate can be structurally zero and look identical to a broken one.
Of course. On the other hand if you see 8 % gap for similar workloads, averaged across tens of sessions, with the same underlying model, it becomes a pretty clear signal. And I do exclude first request per provider per session from the statistic.