You can use a large set of sample articles that are well written, the machine learning is done on these, then the search engine would compare against these for each web page if finds when searching. You could easily have a search go a few hundred pages, just have it take 20+ pages of google results. I worked on a project that used NLP (for a startup), I wasn't the one doing most NLP related stuff but it was definitely an interesting experience and the guy who was in charge of the NLP was able to get some pretty interesting results, even from fairly small data-sets.
Comments
You can use a large set of sample articles that are well written, the machine learning is done on these, then the search engine would compare against these for each web page if finds when searching. You could easily have a search go a few hundred pages, just have it take 20+ pages of google results. I worked on a project that used NLP (for a startup), I wasn't the one doing most NLP related stuff but it was definitely an interesting experience and the guy who was in charge of the NLP was able to get some pretty interesting results, even from fairly small data-sets.