I'm not a quant, and quite removed from the field. The transaction systems I work with are much slower, and most likely deal with significantly less information.
But, is the end all be all the execution speed? Does flexibility and robust recovery not matter?
Everything I'm hearing is that, it's better to gut the car down to the frame with no safety measures then even a little bit of coverage. If that program crashes or cross wires some data, how much damage can it do to your financial position?
From what I understand, billions of dollars are on the line everyday, but everyone is racing towards the bottom of that nanosecond mark, generally at the expense of risking instability etc.
Things that would never fly in other high-risk environments seem to fly in the stock system, and I'm going out on a limb, but it seems that's because if there was there is an adult watching the kids play in the stock market sand box and a fuse will blow if an HFT feedback loop triggers some real idiocy. (http://www.zerohedge.com/article/hft-fat-digital-finger-brea... )
There's a dramatic difference between high-frequency trading and other quantitative trading. In HFT, speed is ALMOST everything. It's not entirely, but many of these firms are running VERY similar algorithms (most of the quants have worked at many other firms), and are primarily competing on speed or gaming each other.
Now, there is still a market for advanced quantitative modeling in which reliability is important, and the ease of maintenance, troubleshooting and extensibility make up for microseconds or nanoseconds. A lot of statistical arbitrage can be categorized in this way, although some of the ideas in stat arb are making their way to HFT.
Basically, different applications have different performance requirements. And even though performance is now measured in low microseconds, the battle for nanosecond performance is underway, and extremely lucrative.
Now as to the argument of whether any of this actually provides any value to society, I think the answer is a definitive "no" (wrt HFT). But that's just my opinion - I used to be in the industry but left because of that. The majority of arguments used to justify HFT are really just self-serving rationalizations by people who are only interested in getting rich. It's another example of a screwed up incentive system that is increasing risk in the market rather than decreasing it.
I work in this environment, and execution speed is the "end all be all" for certain programs in the distributed system that makes up the average modern trading environment in most, if not all, firms.
Your analogy to a car is exactly the one that I've used for years: a race car doesn't generally have air bags or electronic stability control; it's stripped down to 4 wheels, an engine, and a steering wheel. I don't really trust "robust recovery" as a idea in this sort of environment anyway because most error conditions are extremely rare and usually involve errors in external systems such as the exchange so the recovery code is impossible to test in a normal sense and is thus very likely to be incorrect. My philosophy (developed after a fairly long career in this field) is: "when in doubt, print a descriptive error message, call abort(), and let the support team sort it out." I am extremely risk averse when it comes to the systems I design, so it's not the case of lack of adult supervision but more an understanding of the limits of what you can do in an extremely complicated environment and that writing lots of error-handling code gives only the illusion of safety.
I was only peripherally involved in trade systems. My understanding is that the amounts of money to be made, and lost if you're second, completely drown the arguable maintenance and personnel costs of C++.
That is true for a small subset of the stuff quants do. For a lot of other stuff an hour this way or that makes hardly any difference at all. A normal day for one of my quant friends basically looks like, get into the office and spend the morning putting together some calculations, run them and got to lunch, get back from lunch and discuss the results with some colleagues, if you decided to trade based on the numbers, call a broker. Seconds and minutes are totally irrelevant to her.
Comments
I'm not a quant, and quite removed from the field. The transaction systems I work with are much slower, and most likely deal with significantly less information.
But, is the end all be all the execution speed? Does flexibility and robust recovery not matter?
Everything I'm hearing is that, it's better to gut the car down to the frame with no safety measures then even a little bit of coverage. If that program crashes or cross wires some data, how much damage can it do to your financial position?
From what I understand, billions of dollars are on the line everyday, but everyone is racing towards the bottom of that nanosecond mark, generally at the expense of risking instability etc.
Things that would never fly in other high-risk environments seem to fly in the stock system, and I'm going out on a limb, but it seems that's because if there was there is an adult watching the kids play in the stock market sand box and a fuse will blow if an HFT feedback loop triggers some real idiocy. (http://www.zerohedge.com/article/hft-fat-digital-finger-brea... )
There's a dramatic difference between high-frequency trading and other quantitative trading. In HFT, speed is ALMOST everything. It's not entirely, but many of these firms are running VERY similar algorithms (most of the quants have worked at many other firms), and are primarily competing on speed or gaming each other.
Now, there is still a market for advanced quantitative modeling in which reliability is important, and the ease of maintenance, troubleshooting and extensibility make up for microseconds or nanoseconds. A lot of statistical arbitrage can be categorized in this way, although some of the ideas in stat arb are making their way to HFT.
Basically, different applications have different performance requirements. And even though performance is now measured in low microseconds, the battle for nanosecond performance is underway, and extremely lucrative.
Now as to the argument of whether any of this actually provides any value to society, I think the answer is a definitive "no" (wrt HFT). But that's just my opinion - I used to be in the industry but left because of that. The majority of arguments used to justify HFT are really just self-serving rationalizations by people who are only interested in getting rich. It's another example of a screwed up incentive system that is increasing risk in the market rather than decreasing it.
I work in this environment, and execution speed is the "end all be all" for certain programs in the distributed system that makes up the average modern trading environment in most, if not all, firms.
Your analogy to a car is exactly the one that I've used for years: a race car doesn't generally have air bags or electronic stability control; it's stripped down to 4 wheels, an engine, and a steering wheel. I don't really trust "robust recovery" as a idea in this sort of environment anyway because most error conditions are extremely rare and usually involve errors in external systems such as the exchange so the recovery code is impossible to test in a normal sense and is thus very likely to be incorrect. My philosophy (developed after a fairly long career in this field) is: "when in doubt, print a descriptive error message, call abort(), and let the support team sort it out." I am extremely risk averse when it comes to the systems I design, so it's not the case of lack of adult supervision but more an understanding of the limits of what you can do in an extremely complicated environment and that writing lots of error-handling code gives only the illusion of safety.
I was only peripherally involved in trade systems. My understanding is that the amounts of money to be made, and lost if you're second, completely drown the arguable maintenance and personnel costs of C++.
That is true for a small subset of the stuff quants do. For a lot of other stuff an hour this way or that makes hardly any difference at all. A normal day for one of my quant friends basically looks like, get into the office and spend the morning putting together some calculations, run them and got to lunch, get back from lunch and discuss the results with some colleagues, if you decided to trade based on the numbers, call a broker. Seconds and minutes are totally irrelevant to her.