Skip to content

Is there an hour by hour model quality tracker?

1 pointsshadmand1 comment
On HN

I'd love to put some actual science behind a feeling I think a lot of us share: certain hosted models seem to dip in quality at certain times of day. In my group, it’s almost treated as a “yeah, of course” kind of thing. Having something near-real-time that shows which models are performing best at any moment would be… honestly, pretty awesome.

I asked Perplexity and got nothing: https://www.perplexity.ai/search/is-there-an-hour-by-hour-model-8.03Z_BlSomAY7o9F5HwmQ#0

Comments

I have noticed this 100%. My theory is that they are serving quantized versions of the models randomly to save costs.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.