Have you tried the recently released v.1.6.0 of Julia? Precompilation is dramatically faster. Also, for serious numerical work, startup time is eclipsed by run time, so it tends not to be relevant. I did most of my physics simulations in Fortran, which is a great language for the task, but if I were writing a new simulation code today I would use Julia. It is the only language in the petaflop club that is actually better to program in than Fortran. (+1 for gnuplot.)
Have you tried the recently released v.1.6.0 of Julia? Precompilation is dramatically faster.
Yes, I tried it upon its release and, while much faster, the startup time is still dramatically sub-par, especially when loading packages. This means that when I call Julia scripts from makefiles, etc., a considerable amount of running time is spent in re-compiling over and over the same packages. I agree that my usage pattern is not at all representative, but still it seems that Julia is not a good "unix citizen", since it tries to force you to do everything inside its own REPL, instead of the native one. This is indeed my main point of friction with Julia. If it started instantaneously it would be essentially perfect. This is not a matter of dividing the startup time by 2, but at least by 200.
The type system and dispatch paradigm provide a powerful method of code and project organization, and facilitate a type of code reuse that you don’t get with Fortran: the ability to pull other people packages into my project and use their types and methods. That’s one thing.
EDIT: Also, having a REPL, and a nice one, is a big deal.
Can you expand a bit on the need for "types" in numerical computation? They sort of seem like overkill to me. The only types I use are float and double, but I could really do with just doubles.
OK, admittedly I also need complex numbers (two floats) and ad numbers (however they are represented, typically by two floats also). But that's it. They are numbers after all, associative and commutative. The "dispatch paradigm" that these types require seems really simple, as it is implemented by e.g. generics in C as in tgmath.h. I'm wholly unconvinced--or more honestly, wholly ignorant--of the interest of a really complicated and powerful type system with multiple dispatch, broadcasting and whatnot that is offered by Julia. Fortran really seems enough for me. [And I do not care at all for the REPL, but that is a separate issue.]
Some other nice things that come from having a type system for numerical work:
- Forward mode automatic differentiation. Having a type system that allows `Dual` numbers to pass through your algorithm, simulation, or whatever means you can calculate derivatives, gradients, jacobians, and hessians efficiently (for small problems) and accurately without having to change any of your code. There are so many times where I say “hey, I wonder what the sensitivity of my simulation output to this input parameter is” and it’s really nice to be able to answer that question with one line of code.
- Unitful numbers. It’s really nice to be able to pass numbers with units through a simulation (little to no performance penalty!) to make sure everything checks out in that respect.
- Uncertainty. Both Measurements.jl and MonteCarloMeasurements.jl provide numbers that propagate (linear and nonlinear, respectively) uncertainty as the pass through calculations. Want to see how uncertainty in a parameter propagates through a calculation? Just change that one parameter to an uncertain number and let it run through your algorithm as-is and it will spit out an answer with uncertainty bounds on the other side.
These are just a few examples of the stuff I use it for in my everyday work. Having a full type system for numerical work is one of those things that seems silly before you use it, but once you do, you wonder how you got by without it before.
EDIT: BTW, these are just examples of numerical types. Sending specialized array types through a your code is also a thing. For example, if you have a `Diagonal` type matrix and you send it to an eigenvalue solve, it just pulls the elements from the diagonal without wasting any time trying to calculate anything. Or there are things like ComponentArrays.jl (full disclosure, I wrote this library), that let you pass arbitrarily deep structured information through a differential equation or optimization solver for much cleaner and more readable code than just indexing into a plain vector like you would usually have to do. And you can even put your weird numerical types inside of the weird array types and just send it on through.
It’s not that other types are “needed”, but that they let you do some pretty powerful things with surprising ease. And the nice thing is that in Julia, you can ignore them. You can just compute with floats or doubles as if the type system doesn’t exist. But it’s there in case you would like, for example, to apply the differential equation solver that you just wrote to quaternion-valued functions with no extra work.
This is how I found out it worked in the differential equation solver: users were using it. The issue was unrelated (they didn't define enough boundary conditions), so it's quite cool that it was useful to someone. It turns out the quaternions have use cases in 3D rotations:
Thanks for this article, it's really well written and engaging.
But here, I cannot resist:
You can just compute with floats or doubles as if the type system doesn’t exist.
Except when you can't! I want to plot a stupid array of floats. Yet it takes 10 seconds because it is juggling useless types around. A complex type system may be a nice thing to have, if you really want it, but it is definitely not "free", and it always involves serious compromises that make other things impossible or very cumbersome. I would like an option like --type=float in the interpreter that assumed that all numbers are of that type and ran extremely fast.
I’m glad you liked the article. Obviously the 10 second or so of precompilation time is a big issue for you. I guess I have no further suggestions about that other than to suggest again to look at creating sysimages (learning how to do this is on my to-do list).
I was particularly "seduced" by the idea of using a standard ODE solver directly on quaternions. Having worked in the smoothing of 3D camera trajectories the last year, I would have definitely loved to know that at the time!
Comments
Have you tried the recently released v.1.6.0 of Julia? Precompilation is dramatically faster. Also, for serious numerical work, startup time is eclipsed by run time, so it tends not to be relevant. I did most of my physics simulations in Fortran, which is a great language for the task, but if I were writing a new simulation code today I would use Julia. It is the only language in the petaflop club that is actually better to program in than Fortran. (+1 for gnuplot.)
Yes, I tried it upon its release and, while much faster, the startup time is still dramatically sub-par, especially when loading packages. This means that when I call Julia scripts from makefiles, etc., a considerable amount of running time is spent in re-compiling over and over the same packages. I agree that my usage pattern is not at all representative, but still it seems that Julia is not a good "unix citizen", since it tries to force you to do everything inside its own REPL, instead of the native one. This is indeed my main point of friction with Julia. If it started instantaneously it would be essentially perfect. This is not a matter of dividing the startup time by 2, but at least by 200.
I understand. I haven’t tried this myself, but others have had luck, for this use case, in using sysimages: https://julialang.github.io/PackageCompiler.jl/dev/sysimages...
Thanks! I was told about this on a previous HN thread, but I have still to try it. Will do as soon as I have some time. It looks really promising.
Can you elaborate on why it "is actually better to program in than Fortran"?
The type system and dispatch paradigm provide a powerful method of code and project organization, and facilitate a type of code reuse that you don’t get with Fortran: the ability to pull other people packages into my project and use their types and methods. That’s one thing.
EDIT: Also, having a REPL, and a nice one, is a big deal.
Can you expand a bit on the need for "types" in numerical computation? They sort of seem like overkill to me. The only types I use are float and double, but I could really do with just doubles.
OK, admittedly I also need complex numbers (two floats) and ad numbers (however they are represented, typically by two floats also). But that's it. They are numbers after all, associative and commutative. The "dispatch paradigm" that these types require seems really simple, as it is implemented by e.g. generics in C as in tgmath.h. I'm wholly unconvinced--or more honestly, wholly ignorant--of the interest of a really complicated and powerful type system with multiple dispatch, broadcasting and whatnot that is offered by Julia. Fortran really seems enough for me. [And I do not care at all for the REPL, but that is a separate issue.]
Some other nice things that come from having a type system for numerical work:
- Forward mode automatic differentiation. Having a type system that allows `Dual` numbers to pass through your algorithm, simulation, or whatever means you can calculate derivatives, gradients, jacobians, and hessians efficiently (for small problems) and accurately without having to change any of your code. There are so many times where I say “hey, I wonder what the sensitivity of my simulation output to this input parameter is” and it’s really nice to be able to answer that question with one line of code.
- Unitful numbers. It’s really nice to be able to pass numbers with units through a simulation (little to no performance penalty!) to make sure everything checks out in that respect.
- Uncertainty. Both Measurements.jl and MonteCarloMeasurements.jl provide numbers that propagate (linear and nonlinear, respectively) uncertainty as the pass through calculations. Want to see how uncertainty in a parameter propagates through a calculation? Just change that one parameter to an uncertain number and let it run through your algorithm as-is and it will spit out an answer with uncertainty bounds on the other side.
These are just a few examples of the stuff I use it for in my everyday work. Having a full type system for numerical work is one of those things that seems silly before you use it, but once you do, you wonder how you got by without it before.
EDIT: BTW, these are just examples of numerical types. Sending specialized array types through a your code is also a thing. For example, if you have a `Diagonal` type matrix and you send it to an eigenvalue solve, it just pulls the elements from the diagonal without wasting any time trying to calculate anything. Or there are things like ComponentArrays.jl (full disclosure, I wrote this library), that let you pass arbitrarily deep structured information through a differential equation or optimization solver for much cleaner and more readable code than just indexing into a plain vector like you would usually have to do. And you can even put your weird numerical types inside of the weird array types and just send it on through.
I think I have a pretty good answer to your question in this article: https://lwn.net/Articles/834571/
It’s not that other types are “needed”, but that they let you do some pretty powerful things with surprising ease. And the nice thing is that in Julia, you can ignore them. You can just compute with floats or doubles as if the type system doesn’t exist. But it’s there in case you would like, for example, to apply the differential equation solver that you just wrote to quaternion-valued functions with no extra work.
This isn't theoretical too, here's an actual user who opened an issue where their MWE was using quaternions:
https://github.com/SciML/BoundaryValueDiffEq.jl/issues/52
This is how I found out it worked in the differential equation solver: users were using it. The issue was unrelated (they didn't define enough boundary conditions), so it's quite cool that it was useful to someone. It turns out the quaternions have use cases in 3D rotations:
https://en.wikipedia.org/wiki/Gimbal_lock
which is where this all comes in. Anyways, it's always cool to learn from users what your own library supports! That's really a Julia treat.
Thanks for this article, it's really well written and engaging.
But here, I cannot resist:
Except when you can't! I want to plot a stupid array of floats. Yet it takes 10 seconds because it is juggling useless types around. A complex type system may be a nice thing to have, if you really want it, but it is definitely not "free", and it always involves serious compromises that make other things impossible or very cumbersome. I would like an option like --type=float in the interpreter that assumed that all numbers are of that type and ran extremely fast.
I’m glad you liked the article. Obviously the 10 second or so of precompilation time is a big issue for you. I guess I have no further suggestions about that other than to suggest again to look at creating sysimages (learning how to do this is on my to-do list).
I was particularly "seduced" by the idea of using a standard ODE solver directly on quaternions. Having worked in the smoothing of 3D camera trajectories the last year, I would have definitely loved to know that at the time!