Terrible generalization of polynomials is useful for demonstrating overfitting (I've done it myself in tutorials). However, responsible tutorials should mention that the other obvious lesson is that the polynomials (1, x, x², x³, etc) are a terrible set of basis functions for regression. Don't just watch for overfitting, but use a sensible regression model! For complicated fits some methods to consider are: local regression, splines, various artificial neural nets, or Gaussian processes.
I tried not to be that guy and already gave some alternatives for regression.
“Polynomial regression” implies to me that the basis functions are polynomials. I‘ll assume you meant “good basis for a simple fit, maybe by least squares”. More local functions like “radial basis functions” can work well. Or use splines or sigmoidal functions, which saturate to a flat line or linear trend. In some applications Fourier or wavelet bases might be appropriate.
Gaussian process regression is a Bayesian treatment of some basis function models, potentially with an infinite number of basis functions. Artificial neural nets usually use local or sigmoidal basis functions, potentially in a more complicated way.
Comments
Terrible generalization of polynomials is useful for demonstrating overfitting (I've done it myself in tutorials). However, responsible tutorials should mention that the other obvious lesson is that the polynomials (1, x, x², x³, etc) are a terrible set of basis functions for regression. Don't just watch for overfitting, but use a sensible regression model! For complicated fits some methods to consider are: local regression, splines, various artificial neural nets, or Gaussian processes.
So what is a good basis for polynomial regression? I have heard this statement a few times, but I never heard of a good alternative.
I tried not to be that guy and already gave some alternatives for regression.
“Polynomial regression” implies to me that the basis functions are polynomials. I‘ll assume you meant “good basis for a simple fit, maybe by least squares”. More local functions like “radial basis functions” can work well. Or use splines or sigmoidal functions, which saturate to a flat line or linear trend. In some applications Fourier or wavelet bases might be appropriate.
Gaussian process regression is a Bayesian treatment of some basis function models, potentially with an infinite number of basis functions. Artificial neural nets usually use local or sigmoidal basis functions, potentially in a more complicated way.