There are very standard ways of validating a predictor. Using test & training sets, and tuning the output to be optimized even on unseen data.
If they did this, they should be open about the numbers. Hell, I'd be open about the data too, and make a challenge to make a better predictor. If the algorithms still run on their machines, they can only gain by being open.
But in order to be a reliable predictor, I think it would be necessary to predict the economy as well. The same startup that would be successful during the bubble would not be so successful after it popped. In other words, I don't think they have enough inputs to be reliable predictors.
On the other hand, perhaps they don't need to be reliable absolute predictors. The VC's question is basically which should be funded. This might be able to give a rough ordering, which is plenty valuable.
The life of a company from start to big exit or large acquisition is pretty long. I'm not sure the economy matters so much -- just delays things sometimes.
I guess I was making the assumption that the 7 or 8 startups mentioned there were all of them. If they had some sort of large data set I'd be more impressed.
Still not so much as if their algorithm holds up to scrutiny over the next 3.
Comments
There are very standard ways of validating a predictor. Using test & training sets, and tuning the output to be optimized even on unseen data.
If they did this, they should be open about the numbers. Hell, I'd be open about the data too, and make a challenge to make a better predictor. If the algorithms still run on their machines, they can only gain by being open.
Sure, standard machine learning stuff.
But in order to be a reliable predictor, I think it would be necessary to predict the economy as well. The same startup that would be successful during the bubble would not be so successful after it popped. In other words, I don't think they have enough inputs to be reliable predictors.
On the other hand, perhaps they don't need to be reliable absolute predictors. The VC's question is basically which should be funded. This might be able to give a rough ordering, which is plenty valuable.
The life of a company from start to big exit or large acquisition is pretty long. I'm not sure the economy matters so much -- just delays things sometimes.
I guess I was making the assumption that the 7 or 8 startups mentioned there were all of them. If they had some sort of large data set I'd be more impressed.
Still not so much as if their algorithm holds up to scrutiny over the next 3.
Yeah, I'm agreeing with you. What they show is really weak, and they is a well known way to make it more acceptable.