I think you would use MLComp as a baseline. You have a set of data that you can run generic algorithms on to show you what the best results are without any further considerations.
It is also useful for researchers who develop new algorithms for a specific dataset but then make it a generic algorithm. No research has the time to sit there manually testing every dataset that they can find to see what their algorithm works well on.
So for data set providers it gives them a quick look at what machine learning can offer without a lot of development.
For researchers it gives them a chance to see a surprising result, maybe their algorithm works well on a dataset they never considered.
I would argue the people that MLComp is trying to attract probably already know about common machine learning techniques. And if that is the case, they probably already know where to get good data (ie. Machine Learning Repository). I'm one of the machine learning researcher's you speak of, and I can tell you this is something I would not use--only because I apply the algorithms I create on problems that I'm interested in and can find data for. I don't ever think "gee, I've got this sweet algorithm, I wonder what it could be used for." Instead I think, "this algorithm does clustering really well, I bet it would work in this case". Then I go find that data. And data providers don't care what happens to their data--that's why they provide it.
Comments
I think you would use MLComp as a baseline. You have a set of data that you can run generic algorithms on to show you what the best results are without any further considerations.
It is also useful for researchers who develop new algorithms for a specific dataset but then make it a generic algorithm. No research has the time to sit there manually testing every dataset that they can find to see what their algorithm works well on.
So for data set providers it gives them a quick look at what machine learning can offer without a lot of development.
For researchers it gives them a chance to see a surprising result, maybe their algorithm works well on a dataset they never considered.
I would argue the people that MLComp is trying to attract probably already know about common machine learning techniques. And if that is the case, they probably already know where to get good data (ie. Machine Learning Repository). I'm one of the machine learning researcher's you speak of, and I can tell you this is something I would not use--only because I apply the algorithms I create on problems that I'm interested in and can find data for. I don't ever think "gee, I've got this sweet algorithm, I wonder what it could be used for." Instead I think, "this algorithm does clustering really well, I bet it would work in this case". Then I go find that data. And data providers don't care what happens to their data--that's why they provide it.