Skip to content

Comment on Passing the torch of NumPy and moving on to Blazeparent

Comments

Julia is a wonderful and elegantly designed language. Fast and a great type system. Intuitive. For several days, I was very excited about the prospect of moving my research to Julia.

And then I discovered that it too has no reasonable shared memory parallelism story, just the same manual distribution of arrays plus multiprocessing that exists in Python.

I will speculate that Julia's authors have the same attitude as many in the Python community -- namely, that there are small jobs, which can be run in one process, and large jobs, which need to be massively parallelized, and nothing in between. But in reality there are many scientific tasks that are medium-sized, for which OpenMP-style solution is the best fit. Tasks which might take days can be reduced to hours. With new developments like Xeon Phi, that ratio might further improve.

Also many problems require a lot of heterogeneous shared state, and it is tedious to manually distribute each element in this shared state. Finally there are many problems, such as natural language processing, that are only partially numerical. For these problems, distributing arrays is only part of the solution.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.