If some logic re-orders a job-related queue in order to group similar things, enhancing caches' hit-ratio, then a quite large queue can be quite effective (especially if there is no priority nor barrier, however in such a case the high throughput cost is a high maximal latency).
Exactly. In fact I'd go so far as to say this is an equally Important Thing to the Important Thing in the article. If your queue is just a simple FIFO / LIFO / whatever to buffer surges in demand until you get to them, then you absolutely get the "queue suddenly goes to infinity as you hit a critical point near maximum throughput" thing happening.
If, however, you use the queue as a way to process queries more efficiently then that's a different story.
Comments
If some logic re-orders a job-related queue in order to group similar things, enhancing caches' hit-ratio, then a quite large queue can be quite effective (especially if there is no priority nor barrier, however in such a case the high throughput cost is a high maximal latency).
Exactly. In fact I'd go so far as to say this is an equally Important Thing to the Important Thing in the article. If your queue is just a simple FIFO / LIFO / whatever to buffer surges in demand until you get to them, then you absolutely get the "queue suddenly goes to infinity as you hit a critical point near maximum throughput" thing happening.
If, however, you use the queue as a way to process queries more efficiently then that's a different story.