Thanks for posting a very top level view of automated trading for engineers.
In case discussion goes cynical here, I can attest to the fact that this sort of thing can be done consistently and successfully for thousands of trades over months' time. These days though with HFT and "intelligent routing" your broker is your opponent in the equities world at least. When conditions become unfavorable to an algorithm things tend to fizzle out rather than wipe you out as long as you're not leveraged to the hilt. The point about optimizing is key: the best algorithm has the minimum number of parameters to tweak otherwise you're just curve-fitting. Whether you add value to the world is anyone's opinion.
I can attest to the fact that this sort of thing can be done consistently and successfully for thousands of trades over months' time.
Then you know that forex is a bank driven market; these little "statistical arbitrages" (which should be a 4 letter word) are subject to scale. When they break, they drain you dry. The "street" is littered with failed FX traders. It is the crack rock of the consumer level game because you can muster such phenomenal leverage (it isn't regulated nearly as much as equities).
At the bank level, they can see who the counter party is and they track their interactions with you. If you milk them too much, they will block you and your trade is dead. I can attest to that.
What is worse is the consumer-level brokers are essentially running a game against you.
My only experience is with equities, but yes forex is both highly leveraged and poorly regulated.
Even in equities, your brokerage is not your friend: if you place a limit order, there's nothing to stop them from placing the same order just a bit closer to the market and using you as their free stop loss.
I'm not sure I understand what you are saying. How does the brokerage make money off of that. If they are improving the price you haven't lowered their risk profile at all.
If they are taking the other side of your trade and the market doesn't reflect that, they've actually given you a free roll.
Suppose you want to buy XYZ for $1.00. You place your bid and are the highest price against a $1.03 offer.
Evil broker places a $1.01 bid and gets a fill.
He immediately offers $1.02.
Case 1: The market runs up, he makes his tick on essentially zero commissions. On some markets, you earn commissions for providing liquidity.
Case 2: The market falls off, he smacks your bid and only loses a penny.
This is edge.
EDIT: When I say "the market", he could base his actions off an index. As for "free roll", sure... but there is still an impact on liquidity and you're in a situation where you are more likely to get a fill whenever the market is going against you.
OK, but to my other reply. If they are going to blatantly front run, why would they even bother with the markets. Just spoof market behavior to you and take your money. Much easier.
If this behavior is impacting your execution why are you still using "Evil Broker"?
As someone who's experience is way more on the commodities futures side of the house, I'm shocked that blatant front running is an expected cost of business in equities trading. Even in the highly monopolized CME world front running is considered beyond the pale.
I will say (in my experience in the commodities markets), 9 times out of 10 folks who claim that they are being front run are actually just unaware of how sophisticated their competition is when it comes to micro-book structure optimization.
FX is a great market, but not for the reasons people typically think. The problem lies in brokers for the mass market (the little guy). If you're at a billion dollar hedge fund, you have access to an entirely different class of brokers (and those markets are full of intrigue...)
I second your claim ("9 times out of 10") for CME. Human nature is to assign a complex answer ("they're reading the tape") to something very simple ("my model is shit").
That said, front running happens on CME, too. Again, it is the same construction: consumer sending orders through a nefarious broker.
Imagine your algorithm says that a certain non-volatile instrument isn't going to go much below 5.00 and often goes above 5.15. You place a limit order in advance to buy at 5.00 (market orders don't guarantee price) and if you get filled you'll be hoping to short at 5.15 or so later. "Evil Brokerage" comes in and offers to buy at 5.01. Remarkably, you observe that any time you pull your 5.00 order, their 5.01 order disappears.
The instrument heads toward your 5.00 target. If their order turns out to provide all the necessary liquidity at that moment, the sellers never reach you, and the instrument turns around and "Evil Brokerage" soon sell for higher. Alternatively, if the instrument is on a move and it going to go against you too, brokerage sell to you and they only lose 0.01/share. That's why I call the scenario a free stop loss.
I get that, but again, doesn't that fall under illegal trading behavior? If we are going to assume that the brokerages are rampantly front running, why assume they are even telling the truth about execution.
At that point they can just tell you that your trades executed that way without the bother of actually going into the market.
Further, why would you stay on an execution platform that is so obviously bad for you?
I don't think it's illegal, not even really front-running. That'd be them turning a price profit on filling your order, not just using it to avoid a potential loss.
Also, you're right, at a minimum, via a subsidiary it's possible for the brokerage to "fill" your order without it ever reaching the exchanges, I think as long as it's inside the national bid-ask price spread at that moment. They pitch this as a feature - it also lets them avoid exchange fees. Then they can manage their overall risk separately.
Yes, I stopped trying to make those trades years ago. At one time though it was a remarkably consistent way to succeed on a small scale.
I'll admit I haven't looked at every broker, but every exchange that I know of, you have to opt in for orders to not go to the other exchanges.
This frequently is a feature for both sides of the equation due to lower fees, better execution rules, rebates, priority, etc.
I certainly know that I would never opt in to a platform that consistently provided worse execution (especially in the equities space where there are so many choices).
You place a limit order in advance to buy at 5.00 (market orders don't guarantee price) and if you get filled you'll be hoping to short at 5.15 or so later.
Minor nitpick.
Since you've bought the stock first, your second trade will be a sell and not a short, unless you overfill on the second leg, in that case you'll have a partial sell and partial short.
Correct. In this simple algo scenario the position is assumed to be long 1X or short 1X at any given time. After the first trade, all reversals trades are then either buy 2X or sell 2X until the very last trade.
Interesting, so, you are saying that the right approach (plus a religious avoidance of leveraged positions) is generally a low risk investment? What about trading fees, won't those progressively eat into you as your model continues to fizzle out at thousands of trades per month?
Trading costs have to part of your model, but at Interactive Brokers for example you only pay $0.005/share (with a $1 minimum) to trade. At that rate you can try to trade for an average of pennies per share (including winners and losers). When an algorithm fizzles, my experience is that it stops making small amounts of money per trade on average and starts losing small amounts of money per trade on average, with more volatility.
As you see then, you may automate trading itself, but then your discretion is shifted from deciding what stock to buy or sell short to when to turn on or off an algorithm. The advantage I suppose is that an algorithm isn't necessarily directly tied to market direction.
Comments
Thanks for posting a very top level view of automated trading for engineers.
In case discussion goes cynical here, I can attest to the fact that this sort of thing can be done consistently and successfully for thousands of trades over months' time. These days though with HFT and "intelligent routing" your broker is your opponent in the equities world at least. When conditions become unfavorable to an algorithm things tend to fizzle out rather than wipe you out as long as you're not leveraged to the hilt. The point about optimizing is key: the best algorithm has the minimum number of parameters to tweak otherwise you're just curve-fitting. Whether you add value to the world is anyone's opinion.
Proceed at your own risk of time and money.
I can attest to the fact that this sort of thing can be done consistently and successfully for thousands of trades over months' time.
Then you know that forex is a bank driven market; these little "statistical arbitrages" (which should be a 4 letter word) are subject to scale. When they break, they drain you dry. The "street" is littered with failed FX traders. It is the crack rock of the consumer level game because you can muster such phenomenal leverage (it isn't regulated nearly as much as equities).
At the bank level, they can see who the counter party is and they track their interactions with you. If you milk them too much, they will block you and your trade is dead. I can attest to that.
What is worse is the consumer-level brokers are essentially running a game against you.
Caveat Trado
My only experience is with equities, but yes forex is both highly leveraged and poorly regulated.
Even in equities, your brokerage is not your friend: if you place a limit order, there's nothing to stop them from placing the same order just a bit closer to the market and using you as their free stop loss.
I'm not sure I understand what you are saying. How does the brokerage make money off of that. If they are improving the price you haven't lowered their risk profile at all.
If they are taking the other side of your trade and the market doesn't reflect that, they've actually given you a free roll.
Suppose you want to buy XYZ for $1.00. You place your bid and are the highest price against a $1.03 offer.
Evil broker places a $1.01 bid and gets a fill.
He immediately offers $1.02.
Case 1: The market runs up, he makes his tick on essentially zero commissions. On some markets, you earn commissions for providing liquidity.
Case 2: The market falls off, he smacks your bid and only loses a penny.
This is edge.
EDIT: When I say "the market", he could base his actions off an index. As for "free roll", sure... but there is still an impact on liquidity and you're in a situation where you are more likely to get a fill whenever the market is going against you.
Is that legal trading anywhere? That seems like it would fall into front running definitions in all the jurisdictions I know of.
Legal? No. Does it happen: yes.
OK, but to my other reply. If they are going to blatantly front run, why would they even bother with the markets. Just spoof market behavior to you and take your money. Much easier.
If this behavior is impacting your execution why are you still using "Evil Broker"?
You mean, provide a false market?
They do that in FX (you're trading their market; it is up to you to know what is going on beyond their walls).
As for spoofing other markets, you can get away with front running.
I'll grant trading FX is for crazy people.
As someone who's experience is way more on the commodities futures side of the house, I'm shocked that blatant front running is an expected cost of business in equities trading. Even in the highly monopolized CME world front running is considered beyond the pale.
I will say (in my experience in the commodities markets), 9 times out of 10 folks who claim that they are being front run are actually just unaware of how sophisticated their competition is when it comes to micro-book structure optimization.
FX is a great market, but not for the reasons people typically think. The problem lies in brokers for the mass market (the little guy). If you're at a billion dollar hedge fund, you have access to an entirely different class of brokers (and those markets are full of intrigue...)
I second your claim ("9 times out of 10") for CME. Human nature is to assign a complex answer ("they're reading the tape") to something very simple ("my model is shit").
That said, front running happens on CME, too. Again, it is the same construction: consumer sending orders through a nefarious broker.
EDIT: Adding this here since I can't reply to you kasey.
Why trade with Evil Broker? Don't. I suppose you can argue the difference between trading and investing.
Now, if you want to trade, well, just know they're going to fuck you and lubricate appropriately.
Imagine your algorithm says that a certain non-volatile instrument isn't going to go much below 5.00 and often goes above 5.15. You place a limit order in advance to buy at 5.00 (market orders don't guarantee price) and if you get filled you'll be hoping to short at 5.15 or so later. "Evil Brokerage" comes in and offers to buy at 5.01. Remarkably, you observe that any time you pull your 5.00 order, their 5.01 order disappears.
The instrument heads toward your 5.00 target. If their order turns out to provide all the necessary liquidity at that moment, the sellers never reach you, and the instrument turns around and "Evil Brokerage" soon sell for higher. Alternatively, if the instrument is on a move and it going to go against you too, brokerage sell to you and they only lose 0.01/share. That's why I call the scenario a free stop loss.
I get that, but again, doesn't that fall under illegal trading behavior? If we are going to assume that the brokerages are rampantly front running, why assume they are even telling the truth about execution.
At that point they can just tell you that your trades executed that way without the bother of actually going into the market.
Further, why would you stay on an execution platform that is so obviously bad for you?
I don't think it's illegal, not even really front-running. That'd be them turning a price profit on filling your order, not just using it to avoid a potential loss.
Also, you're right, at a minimum, via a subsidiary it's possible for the brokerage to "fill" your order without it ever reaching the exchanges, I think as long as it's inside the national bid-ask price spread at that moment. They pitch this as a feature - it also lets them avoid exchange fees. Then they can manage their overall risk separately.
Yes, I stopped trying to make those trades years ago. At one time though it was a remarkably consistent way to succeed on a small scale.
I'll admit I haven't looked at every broker, but every exchange that I know of, you have to opt in for orders to not go to the other exchanges.
This frequently is a feature for both sides of the equation due to lower fees, better execution rules, rebates, priority, etc.
I certainly know that I would never opt in to a platform that consistently provided worse execution (especially in the equities space where there are so many choices).
Minor nitpick.
Since you've bought the stock first, your second trade will be a sell and not a short, unless you overfill on the second leg, in that case you'll have a partial sell and partial short.
Correct. In this simple algo scenario the position is assumed to be long 1X or short 1X at any given time. After the first trade, all reversals trades are then either buy 2X or sell 2X until the very last trade.
Very true. Case in point, Ameritrade's odd marriage with Citadel... think they might be reading the tape?
https://www.tdameritrade.com/forms/AMTD2055.pdf
Interesting, so, you are saying that the right approach (plus a religious avoidance of leveraged positions) is generally a low risk investment? What about trading fees, won't those progressively eat into you as your model continues to fizzle out at thousands of trades per month?
Trading costs have to part of your model, but at Interactive Brokers for example you only pay $0.005/share (with a $1 minimum) to trade. At that rate you can try to trade for an average of pennies per share (including winners and losers). When an algorithm fizzles, my experience is that it stops making small amounts of money per trade on average and starts losing small amounts of money per trade on average, with more volatility.
As you see then, you may automate trading itself, but then your discretion is shifted from deciding what stock to buy or sell short to when to turn on or off an algorithm. The advantage I suppose is that an algorithm isn't necessarily directly tied to market direction.
Ah, thanks for the explanation! I've stayed completely away from forex, so I don't know much about it.
Look into interactive brokers. Their fees are pennies per trade.