Definitely influenced by a few existing libraries (libsvm, svmlight etc.) but ultimately custom built to really ramp up performance and scaling. Our customers throw the Twitter firehose at us, so we need to be able to analyze data quickly.
Yeah, in terms of performance "improved" is the wrong term. We've tailored things more to our use cases, especially since we need feature weighting and multi classification, rather than simply binary classification.
Comments
Definitely influenced by a few existing libraries (libsvm, svmlight etc.) but ultimately custom built to really ramp up performance and scaling. Our customers throw the Twitter firehose at us, so we need to be able to analyze data quickly.
Wow, I assumed libsvm was already really fast. That is awesome you could improve on that! Any chance you might opensource it in the future?
Sibling comment is impressed, but I'm skeptical that you've improved libsvm. Sorry, can't help it. :)
Can you speak a bit about how you optimized it? I took libsvm to be fairly optimized as far as SVM libraries go...
Yeah, in terms of performance "improved" is the wrong term. We've tailored things more to our use cases, especially since we need feature weighting and multi classification, rather than simply binary classification.
Very cool! I geek out over SVM optimizations and tweaks--have you collected your work anywhere (post? HN comment?) by any chance?
Not really, sorry, but we're adding a new API to be able to setup and configure your own svms. Stay tuned or follow us on twitter @repustate
its possible he is using a specialized kernel or something, or some other edge use case that would make optimizations more feasible