This is like a major highway. They get overloaded at rush hour, so they build lanes to accommodate peak traffic. But travel any highway at 2AM (outside of LA/NYC sized metropolises which can have rush hour at any time) and you get an entire highway to yourself. The problem is rush hour. If more AI requests could be load balanced to run overnight, we’d solve a large part of the current pressure. But everyone demands search engine-like response times, even if overnight wouldn’t be a problem.
Comments
This is like a major highway. They get overloaded at rush hour, so they build lanes to accommodate peak traffic. But travel any highway at 2AM (outside of LA/NYC sized metropolises which can have rush hour at any time) and you get an entire highway to yourself. The problem is rush hour. If more AI requests could be load balanced to run overnight, we’d solve a large part of the current pressure. But everyone demands search engine-like response times, even if overnight wouldn’t be a problem.