This is a super interesting paper, but I wouldn't put much trust in the author's suggestion that their machine learning model knows how to make better investment decisions than VC's do.
Of course their model soundly beat the returns of real investors. Its training dataset contained hard data about the future of the tech industry. Real investors don't have hard data on the future, they have to guess.
The dataset includes descriptions of each of the firms. The model gets to learn from the training dataset whether sectors like VR, 3D printing, business collaboration, clean tech, or etc. were good or bad areas to invest in in 2014, and then gets to apply that knowledge to the test dataset which covers the same epoch. (EDIT ADDED: to quote another commenter here, "[The paper] says the most important features are text descriptions of the company and CEO, which are featurized as unigram and ngram TF-IDF counts.")
Real investors making real investment decisions don't know in advance whether sectors like VR or 3D printing or clean tech or low-code/no-code or etc. are going to be hot or duds, they have to guess. Equally importantly, this model doesn't know which of the sectors being touted as hot investment opportunities today will, in hindsight, prove to have been hot today. It only knows which ones were hot in 2014.
Real investors making real investment decisions don't know in advance whether sectors like VR or 3D printing or clean tech or low-code/no-code or etc. are going to be hot or duds, they have to guess
Did you just baseline against a random walk?
If so, then at least 60 years of quant investing will beat the model control sample.
Comments
This is a super interesting paper, but I wouldn't put much trust in the author's suggestion that their machine learning model knows how to make better investment decisions than VC's do.
Of course their model soundly beat the returns of real investors. Its training dataset contained hard data about the future of the tech industry. Real investors don't have hard data on the future, they have to guess.
The dataset includes descriptions of each of the firms. The model gets to learn from the training dataset whether sectors like VR, 3D printing, business collaboration, clean tech, or etc. were good or bad areas to invest in in 2014, and then gets to apply that knowledge to the test dataset which covers the same epoch. (EDIT ADDED: to quote another commenter here, "[The paper] says the most important features are text descriptions of the company and CEO, which are featurized as unigram and ngram TF-IDF counts.")
Real investors making real investment decisions don't know in advance whether sectors like VR or 3D printing or clean tech or low-code/no-code or etc. are going to be hot or duds, they have to guess. Equally importantly, this model doesn't know which of the sectors being touted as hot investment opportunities today will, in hindsight, prove to have been hot today. It only knows which ones were hot in 2014.
Did you just baseline against a random walk?
If so, then at least 60 years of quant investing will beat the model control sample.