I didn't mean to imply that this is a problem with arbitrary machines that have GPUs. There are specific kinds of applications that use GPUs and because of how they work have hard limits on the number of tasks they can concurrently process on those GPUs.

Either way, I feel like these details are orthogonal to my original question. Do you think it matters?