I actually built this, and shipped it, in 1996, with no knowledge of page rank, citation analysis, or bibliometrics, for an internal/external search engine for the Envirolink web site. Envirolink was a directory of environmental web sites, so they already had a list of URLs to crawl. The reason it was feasible to build was that it was a fairly constrained list of URLs, it wasn't the entire web.
I didn't really know what I was doing (I was 17), but it was an awesome unpaid summer internship. There were two parts of the search engine - a crawler and the search engine. Both were written in Perl.
Comments
I actually built this, and shipped it, in 1996, with no knowledge of page rank, citation analysis, or bibliometrics, for an internal/external search engine for the Envirolink web site. Envirolink was a directory of environmental web sites, so they already had a list of URLs to crawl. The reason it was feasible to build was that it was a fairly constrained list of URLs, it wasn't the entire web.
I didn't really know what I was doing (I was 17), but it was an awesome unpaid summer internship. There were two parts of the search engine - a crawler and the search engine. Both were written in Perl.