Robert Scoble has a post entitled Search for Toshiba music player demonstrates search engine weakness where he complains about relevance of search results returned by popular web search engines. He writes

Think search is done? OK, try this one. Search for:

Toshiba Gigabeat

Did you find a Toshiba site? All I see is a lot of intermediaries.

I interviewed a few members of the MSN Search team last week and I gave them hell about this. When I'm writing I want to link directly to the manufacturer's main Web site about their product. Why? Because that's the most authoritative place to go.

But I keep having trouble finding manufacturer's sites on Google, MSN, and Yahoo.

Relevancy ratings on search engines still suck. Let's just be honest about it as an industry.

Can the search researchers find a better algorithm? I sure hope so.

Here, compare for yourself. If you're looking for the Toshiba site, did you find what you're looking for when you do searches Google ? On Yahoo ? On MSN ?

Here's the right answer: http://www.gigabeat.toshiba.com . Did you find it with any of the above searches? I didn't.

The [incorrect] assumption in Robert Scoble's post is that the most relevant website for a person searching for information on a piece of electronic equipment is the manufacturer's website. Personally, if I'm considering buying an MP3 player or other electronic equipment I'm interested in (i) reviews of the product and (ii) places where I can buy it. In both cases, I'd be surpised if the manufacturer's website would be the best place to get either.

Relevancy of search results often depends on context. This is one of the reasons why the talk on Vertical Search and A9.com at ETech 2005 resonated so strongly with me. The relevancy of search results sometimes depends on what I want to do with the results. A9.com tries to solve this by allowing users to customize the search engines they use when they come to the site. Google has attempted to solve this by mixing in both traditional web search results with vertical results inline. For example, searching for MSFT on Google returns traditional search results and a stock quote. Also searching for "Seattle, WA" on Google returns traditional web search results and a map. And finally, searching for "Toshiba Gigabeat" on Google returns traditional web search reults and a list of places where you can buy one. 

Even with these efforts, it is unlikely any of them would solve the problem Scoble had as well as if he just used less ambiguous searches. For example, a better test of relevance is which search engine gives the manufacturer's website for the search for "toshiba gigabeat website".

I found the results interesting and somewhat surprising. There definitely is a ways to go in web search.


 

Thursday, 29 September 2005 22:57:56 (GMT Daylight Time, UTC+01:00)
I think all three of your less ambiguous searches could benefit from a judicious quoting of the phrase "Toshiba Gigabeat". Also, when I'm looking for information but not reviews about a piece of hardware I usually add -review. The following yields almost the same first page.

http://www.google.com/search?q=%22Toshiba+Gigabeat%22+-review
http://search.msn.com/results.aspx?q=%22Toshiba+Gigabeat%22++-%28review%29&FORM=QBRE
http://search.yahoo.com/search?p=%22Toshiba+Gigabeat%22+-review

One other interesting option... both Google and Yahoo allow a Directory search (that I can't find in MSN). The Yahoo search with "Toshiba Gigabeat" gives ONE match (and a good one). The Google search yeilds tons, none of the first page being relevant.

Monday, 18 December 2006 21:07:49 (GMT Standard Time, UTC+00:00)
Just-Posted.com [http://www.Just-Posted.com] .... leading the discussion and advancing search engine 'relevance' as a necessity, not a luxury.
Comments are closed.