What is your SLA for an expressive bid? I'm guessing it its less than 1ms?
The optimization procedure (which includes bid evaluation) is ~30ms. We cycle bound (under a formal model of computation via function application and graph reduction) computation of bidders to ensure that everyone shares an identical and deterministic resource cap.
Do you use a database of some sort?
Not as part of the real-time trading system, which operates as a CP fail-stop distributed system model checked for safety and liveness by TLA+ and system tested by Jepsen.
How do you to handle settlement?
Regular way (T+2 settlement with a 3rd party clearing BD)
How do you handle ingest?
We use a constellation of GPS synchronized Stratum 1 clocks and proprietary network timestamping software + hardware to ensure that we process orders entered by the auction call time regardless of what physical host we receive the order on. We do the same for market data broadcast from other trading venues across data centers and geographies. We stream both market data and orders to a central point for processing. Every node in our distributed system that processes orders or “away venue” market data broadcasts a “Gateway Call Announcement (GCA)” message at auction call time to downstream compute nodes that run the auction. Auction solver nodes get to work after receiving GCA messages from the hosts they expect to hear from.
We're following the project with interest, but for now, we're focused on directing engineering resources to other portions of the stack. FWIW our approach is in the low nanos of precision and accuracy, which puts us within spitting distance of the Nyquist criterion for never aliasing two packets to the same timestamp (thus losing a total ordering) at line rate 10G. That's massive overkill for our purposes (auctions 100ms apart), but it's a satisfying property nonetheless :)
Since the GPS signal is just disciplining a local oscillator, it would have to be a sustained outage before drift starts to really matter. But yeah there is a point where it would make a difference.
Same day settlement
This one is outside our control for the moment - we partner with a 3rd party for clearing and settlement, and would depend on our subscribers also making the switch to same-day.
Once we get into other asset classes, fast settlement is definitely of interest. Some cool stuff we could do with incorporating settlement instructions and/or counterparty risk constraints as part of the expressive bidding language.
Comments
The optimization procedure (which includes bid evaluation) is ~30ms. We cycle bound (under a formal model of computation via function application and graph reduction) computation of bidders to ensure that everyone shares an identical and deterministic resource cap.
Not as part of the real-time trading system, which operates as a CP fail-stop distributed system model checked for safety and liveness by TLA+ and system tested by Jepsen.
Regular way (T+2 settlement with a 3rd party clearing BD)
We use a constellation of GPS synchronized Stratum 1 clocks and proprietary network timestamping software + hardware to ensure that we process orders entered by the auction call time regardless of what physical host we receive the order on. We do the same for market data broadcast from other trading venues across data centers and geographies. We stream both market data and orders to a central point for processing. Every node in our distributed system that processes orders or “away venue” market data broadcasts a “Gateway Call Announcement (GCA)” message at auction call time to downstream compute nodes that run the auction. Auction solver nodes get to work after receiving GCA messages from the hosts they expect to hear from.
Could you use something like this to not rely on Stratum 1 clocks or have this as a backup? https://www.datacenterdynamics.com/en/news/facebook-creates-...
We're following the project with interest, but for now, we're focused on directing engineering resources to other portions of the stack. FWIW our approach is in the low nanos of precision and accuracy, which puts us within spitting distance of the Nyquist criterion for never aliasing two packets to the same timestamp (thus losing a total ordering) at line rate 10G. That's massive overkill for our purposes (auctions 100ms apart), but it's a satisfying property nonetheless :)
Well other than GPS being unavailable right ?
Another question are your looking into having same day settlement?
Since the GPS signal is just disciplining a local oscillator, it would have to be a sustained outage before drift starts to really matter. But yeah there is a point where it would make a difference.
This one is outside our control for the moment - we partner with a 3rd party for clearing and settlement, and would depend on our subscribers also making the switch to same-day.
Once we get into other asset classes, fast settlement is definitely of interest. Some cool stuff we could do with incorporating settlement instructions and/or counterparty risk constraints as part of the expressive bidding language.