I was just wondering about how Google Scholar works the other day. It does seem like they can't just have a manual list of sources, because in many cases, it picks up papers from the websites of their authors, not just from large sources like the ACM or the NIPS proceedings or whatever. But even if you are able to pick out research papers when you crawl the whole web, and get author/title/citation information out of them, how do you rank them? In particular, they have citation links between papers -- but presumably any paper can only cite to older papers, so papers form a DAG, where it seems like PageRank shouldn't really work (because the most 'authoritative' papers would end up being old ones).
PageRank already has some regularization (essentially epsilon of outgoing links to everything). Also, papers don't form a DAG because people frequently forward-cite or cross-cite (because authors and their friends often have relevant work in various states of preparation/review).
Comments
I was just wondering about how Google Scholar works the other day. It does seem like they can't just have a manual list of sources, because in many cases, it picks up papers from the websites of their authors, not just from large sources like the ACM or the NIPS proceedings or whatever. But even if you are able to pick out research papers when you crawl the whole web, and get author/title/citation information out of them, how do you rank them? In particular, they have citation links between papers -- but presumably any paper can only cite to older papers, so papers form a DAG, where it seems like PageRank shouldn't really work (because the most 'authoritative' papers would end up being old ones).
PageRank already has some regularization (essentially epsilon of outgoing links to everything). Also, papers don't form a DAG because people frequently forward-cite or cross-cite (because authors and their friends often have relevant work in various states of preparation/review).