Backpressure is not a contract. A protocol is a contract and there are plenty of examples of algorithms deployed that account for the existence of backpressure. But you do not need the consumer to do anything special for a system to have backpressure.
I still don't understand your point about queues not creating backpressure. If we're getting abstract about it, everything has a queue size of 1. In the article and in this thread we're talking about infinite vs bounded queues. An ideal infinite queue has no backpressure because anything can be immediately queued. However, it comes at the unfortunate expense of infinite processing time. The difference between an infinite queue and a bounded queue is that a bounded queue, when it fills up, propagates backpressure. If you choose a queue size of 1 then whatever, it doesn't change anything conseptually. You're just trying to say a queue size of 1 is not a queue and you add a queue when you go from size 1 to > 1. And now we're just arguing semantics.
Making a bounded queue bigger adds slack and relieves pressure caused by unpredictable producers. But it does not change the maximum throughput of your system. The only thing that can do that is adding more consumers or making the existing ones faster (either by improving their speed or allowing more sophistication in the handling of requests in the queue, e.g. batch processing). If you system is already at peak load, adding a queue does not relieve pressure.
I agree with everything you just said, it's just that in my (maybe twisted) mind the principle of back-pressure can be implemented in a number of ways and adding a blocking queue in front of a system that is falling over is just one of those ways. But yeah, I guess at some level everything is a queue (or has a queue in it) so you win.
Comments
Backpressure is not a contract. A protocol is a contract and there are plenty of examples of algorithms deployed that account for the existence of backpressure. But you do not need the consumer to do anything special for a system to have backpressure.
I still don't understand your point about queues not creating backpressure. If we're getting abstract about it, everything has a queue size of 1. In the article and in this thread we're talking about infinite vs bounded queues. An ideal infinite queue has no backpressure because anything can be immediately queued. However, it comes at the unfortunate expense of infinite processing time. The difference between an infinite queue and a bounded queue is that a bounded queue, when it fills up, propagates backpressure. If you choose a queue size of 1 then whatever, it doesn't change anything conseptually. You're just trying to say a queue size of 1 is not a queue and you add a queue when you go from size 1 to > 1. And now we're just arguing semantics.
Making a bounded queue bigger adds slack and relieves pressure caused by unpredictable producers. But it does not change the maximum throughput of your system. The only thing that can do that is adding more consumers or making the existing ones faster (either by improving their speed or allowing more sophistication in the handling of requests in the queue, e.g. batch processing). If you system is already at peak load, adding a queue does not relieve pressure.
I agree with everything you just said, it's just that in my (maybe twisted) mind the principle of back-pressure can be implemented in a number of ways and adding a blocking queue in front of a system that is falling over is just one of those ways. But yeah, I guess at some level everything is a queue (or has a queue in it) so you win.
I mean it’s not about winning I think we’re mostly just saying the same thing differently at this point XD