I don't know why everyone is so quick to dismiss the simple explanation, "That's just the current culture of the industry."
Personally, I find this idea very compelling. The quantitive finance profession isn't exactly open and taking a constant influx of new ideas from the rest of the industry. They're a much more secretive bunch. Is it any surprise, then, that their tools stagnate somewhat because of their reluctance to engage openly with the rest of the industry?
I know that some people are engaged in high-speed trading applications where they require a language with a close analogue to machine instructions for the purposes of performance. I haven't seen any evidence that this is all quants everywhere, or that these suites of quantlibs actually provide all their functionality to that segment of the community.
Meanwhile, the CLR and JVM actually generate remarkably fast code from remarkably high-level specifications and LLVM is a real thing. Haskell and Ocaml take functional definitions and often generate better code than longer C++ definitions. I suspect that there is an under-served market here that is reluctant to adopt new tools for social reasons rather than for technical reasons.
I find it hard to use the term "stagnate" to describe an industry that is developing its own specialized languages/systems (see Q/KDB) and is currently in the process of exploring FPGA technology to improve performance. The secretive nature of the industry means that people outside it don't know what's happening on the inside, not that participants lack awareness of developments in more open industries.
Just to answer your point about CLR/JVM code performance: one early comment in the original article that stood out for me was "Pure computational performance (ignoring memory allocation/deallocation) under .NET runtime (ignoring vectorization) is pretty close to the performance of raw C++", which is all well and good except for the fact that you are going to end up with worse code than someone who doesn't ignore memory allocation and vectorization.
> I find it hard to use the term "stagnate" to describe an industry that is developing its own specialized languages/systems (see Q/KDB) and is currently in the process of exploring FPGA technology to improve performance.
That's fine. We disagree on what's the right direction for software engineering. I welcome debate on the subject. I don't consider migrating to customized hardware a cutting edge technique.
It seems like every second justification I hear for the toolchains I see seems to revolve around performance complaints that are only reasonable in a hard-realtime situations or 2001.
> which is all well and good except for the fact that you are going to end up with worse code than someone who doesn't ignore memory allocation and vectorization.
Actually, that's exactly not what happens. Modern GC is good, man, really good. The vectorization scene is even better for the FP world.
I don't mean it in the sense that they're somehow not developing new techniques. I mean it in the very limited sense that they're not taking advantage of the technology that's bolstering the rest of the industry.
Comments
I don't know why everyone is so quick to dismiss the simple explanation, "That's just the current culture of the industry."
Personally, I find this idea very compelling. The quantitive finance profession isn't exactly open and taking a constant influx of new ideas from the rest of the industry. They're a much more secretive bunch. Is it any surprise, then, that their tools stagnate somewhat because of their reluctance to engage openly with the rest of the industry?
I know that some people are engaged in high-speed trading applications where they require a language with a close analogue to machine instructions for the purposes of performance. I haven't seen any evidence that this is all quants everywhere, or that these suites of quantlibs actually provide all their functionality to that segment of the community.
Meanwhile, the CLR and JVM actually generate remarkably fast code from remarkably high-level specifications and LLVM is a real thing. Haskell and Ocaml take functional definitions and often generate better code than longer C++ definitions. I suspect that there is an under-served market here that is reluctant to adopt new tools for social reasons rather than for technical reasons.
I find it hard to use the term "stagnate" to describe an industry that is developing its own specialized languages/systems (see Q/KDB) and is currently in the process of exploring FPGA technology to improve performance. The secretive nature of the industry means that people outside it don't know what's happening on the inside, not that participants lack awareness of developments in more open industries.
Just to answer your point about CLR/JVM code performance: one early comment in the original article that stood out for me was "Pure computational performance (ignoring memory allocation/deallocation) under .NET runtime (ignoring vectorization) is pretty close to the performance of raw C++", which is all well and good except for the fact that you are going to end up with worse code than someone who doesn't ignore memory allocation and vectorization.
> I find it hard to use the term "stagnate" to describe an industry that is developing its own specialized languages/systems (see Q/KDB) and is currently in the process of exploring FPGA technology to improve performance.
That's fine. We disagree on what's the right direction for software engineering. I welcome debate on the subject. I don't consider migrating to customized hardware a cutting edge technique.
It seems like every second justification I hear for the toolchains I see seems to revolve around performance complaints that are only reasonable in a hard-realtime situations or 2001.
> which is all well and good except for the fact that you are going to end up with worse code than someone who doesn't ignore memory allocation and vectorization.
Actually, that's exactly not what happens. Modern GC is good, man, really good. The vectorization scene is even better for the FP world.
I don't think "stagnate" is fair.
I don't mean it in the sense that they're somehow not developing new techniques. I mean it in the very limited sense that they're not taking advantage of the technology that's bolstering the rest of the industry.