Skip to content

Comment on Stupid Data Miner Tricks: Overfitting the S&P 500

Comments

Does interpolation ever work in forecasting?

My gut instinct would be that markets and human systems are chaotic in nature. Even in the most chaotic systems, if you look at a suitably small sample, you can see some correlations and patterns between different factors which really don't exist. These are mirage correlations.

Take the lorenz attractor as an example. At some points, it will cycle on the same "wing" of the butterfly many times. But betting that it will do it again is a really lousy bet.

Polynomial approximation and curve fitting in general works when we're trying to explicate relationships between variables in a problem space in which we understand causal linkages very well (and they're constant) - it can be really useful in engineering.

AboutSource Built by g1lg1l

Hackerly is an independent reader for Hacker News, built on the public HN API. Not affiliated with Y Combinator.