Google's power comes not from the crawling, but from the retrieval and ranking. They use many more signals than the hyperlinks and anchor text (which is all you'd have if you crawled yourself). Indexing crawled content would have been OK in the year 2000; but today, the users demand more. Relevance is the top priority, and no one does it better than El Goog.
Sorry but Google's ranking algorithm for me is far from brilliant.
To give you an example, search for "webhcat primary key" (without quotes) and note how the top three search results do not actually contain the term webhcat. Google constantly does this. It randomly ignores search terms unless you explicitly quote them.
I believe that there is still a market for a technical/advanced search engine.
Isn't google doing that because it detected the semantic information was on the page, even if the exact term wasn't? Is your issue with the fact that they're doing more than just a keyword retrieval, or is your issue with the fact that they're doing it poorly?
Isn't my issue obvious ? I wanted search results that contained the search terms. Otherwise I wouldn't have entered them in the first place.
I understand Google is trying to be clever here and appealing to novices who don't really understand what they want.
But my point is that for those of us that do it is an incredibly annoying "feature". Feature is in quotes because in the specific case above they didn't find semantic equivalents. They just dropped the "webhcat" term entirely.
Not touching that with a 10 foot pole... (just teasing, just teasing...)
But seriously, it sounds like what you want is a keyword matching engine. Google, for better or worse, has decided they know enough about their user's searches that they don't mind modifying the query parameter in an attempt to retrieve what people want rather than what they literally say they want.
I understand that you don't feel they're serving your needs any longer, and that can be frustrating. I think, however, that you can preceded mandatory terms with a + sign to require it to be present on page.
After Google Plus was released, Google changed its search syntax slightly so that you now need to "quote" mandatory terms instead of prefixing them with a + symbol.
Comments
Google's power comes not from the crawling, but from the retrieval and ranking. They use many more signals than the hyperlinks and anchor text (which is all you'd have if you crawled yourself). Indexing crawled content would have been OK in the year 2000; but today, the users demand more. Relevance is the top priority, and no one does it better than El Goog.
Sorry but Google's ranking algorithm for me is far from brilliant.
To give you an example, search for "webhcat primary key" (without quotes) and note how the top three search results do not actually contain the term webhcat. Google constantly does this. It randomly ignores search terms unless you explicitly quote them.
I believe that there is still a market for a technical/advanced search engine.
Isn't google doing that because it detected the semantic information was on the page, even if the exact term wasn't? Is your issue with the fact that they're doing more than just a keyword retrieval, or is your issue with the fact that they're doing it poorly?
Isn't my issue obvious ? I wanted search results that contained the search terms. Otherwise I wouldn't have entered them in the first place.
I understand Google is trying to be clever here and appealing to novices who don't really understand what they want.
But my point is that for those of us that do it is an incredibly annoying "feature". Feature is in quotes because in the specific case above they didn't find semantic equivalents. They just dropped the "webhcat" term entirely.
Not touching that with a 10 foot pole... (just teasing, just teasing...)
But seriously, it sounds like what you want is a keyword matching engine. Google, for better or worse, has decided they know enough about their user's searches that they don't mind modifying the query parameter in an attempt to retrieve what people want rather than what they literally say they want.
I understand that you don't feel they're serving your needs any longer, and that can be frustrating. I think, however, that you can preceded mandatory terms with a + sign to require it to be present on page.
After Google Plus was released, Google changed its search syntax slightly so that you now need to "quote" mandatory terms instead of prefixing them with a + symbol.
http://waxy.org/2011/10/google_kills_its_other_plus/ https://productforums.google.com/forum/#!topic/websearch/3oI...
This is really weird, because now I have to enter this:
"singer" "song" -inurl:(htm|html|php) intitle:"index of" "last modified" "parent directory" "description" "size" "m4a"|"mp3"|"ogg"|"flac"
to search for songs. Too many quotation marks!
original, unmodified: http://lifehacker.com/207672/turn-google-into-your-own-perso...
I think, however, that you can preceded mandatory terms with a + sign to require it to be present on page.
That was my first reaction too, but I tried it out and it still doesn't work. In fact, the result is kinda funny:
Googling "webhcat primary key" got me 3 results with "webhcat" on the first page.
Googling "+webhcat primary key" got me 5 results in total, none of which contains the word "webhcat".
There's a feature for that: Verbatim mode.
https://www.google.com/search?q=webhcat+primary+key&tbs=li:1
Wow, great to know. I'm sick up putting quotes around half of my keywords.
Unfortunately, now when I search <webhcat primary key> in verbatim mode, the top three results are some dude complaining about Google on HN. :D
Also on their dominance of the advertising market from which the value of crawling comes.