Skip to content

Comment on Faster Parallel Python Without Python Multiprocessingparent

Comments

One problem is that because of the GIL, many python libraries don't use locking on their data, etc., because they don't need to. That makes them thread-unsafe without the GIL.

If you look more closely, it often makes them thread-unsafe even with the GIL. Take a look some time at what operations the GIL can pre-empt a python thread in the middle of.

You mean libraries with c extensions

No, even libraries without. Picking something random in the python 3.7 standard library: collections.OrderedDict.__setitem__ doesn't look thread-safe when it updates its linked list. EDIT: well, in fact, the data race in that one is there whether there is a GIL or not...

Just FYI, there are few remaining use cases for OrderedDict, now that built-in dict keeps insertion order.

Yes, getting rid of the GIL breaks most c extensions, many of which are very important.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.