Not an expert on this, but here are a few open source projects that came up from a quick search:

http://www.marketcetera.com/site/ (HFT platform)

http://zocalo.sourceforge.net/ (prediction market)

http://www.grism.org/ (stock price tracker/analyzer)

Searches

trading: http://sourceforge.net/search/?type_of_search=soft&words...

market: http://sourceforge.net/search/?type_of_search=soft&words...