Skip to content

Comment on Practical threaded programming with Pythonparent

Comments

To say "will still use only one core at a time" is a common misconception. It depends. CPython's GIL is not held during many file or network operations and may be released by C code. It's true that multiprocessing is generally a better solution for distributing CPU-bound tasks across cores, but let's not oversimplify.

And as some people may not know, some implementations like Jython do not have a GIL.

This should at the top. A CPython extension author need do no more than Py_BEGIN_ALLOW_THREADS before some native work, and Py_END_ALLOW_THREADS on completion, literally just 2 macros. In between the interpreter can run freely on another thread.

Consider a network server running a Twisted main loop, serving static files from cold mechanical disk. In this case, you could have 80 or more disk IO worker threads running without feeling any contention, assuming they're asleep for a minimum of 12ms/request, which would be an ideal seek time assuming the disks weren't under any load.

Assuming one disk, those 80 sleeping threads do something particularly useful that's difficult to accomplish without threads: they let the kernel IO scheduler reorder the requests to minimize seeks, and in a much more general way than the large variety of crappy AIO APIs available on UNIX.

There's a trillion uses like this for threads in Python.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.