Maybe I need to update my mental model of how many threads is too many on Linux (and benchmark the impact on the GIL here, which should at least be released for I/O network waits).
On my Linux desktop the default cap seems[1] to be ~32k.
See eg https://www.baeldung.com/linux/max-threads-per-process for some runtime sysctl knobs if you want to go higher than the default limits. (Though the section 6 there seems out of date and written with the 32-bit OS in mind)
Comments
Maybe I need to update my mental model of how many threads is too many on Linux (and benchmark the impact on the GIL here, which should at least be released for I/O network waits).
On my Linux desktop the default cap seems[1] to be ~32k.
See eg https://www.baeldung.com/linux/max-threads-per-process for some runtime sysctl knobs if you want to go higher than the default limits. (Though the section 6 there seems out of date and written with the 32-bit OS in mind)
[1] as reported by the Python snippet from https://stackoverflow.com/a/64406494