New French Relevancy Study Puts Yahoo & Google Tied For First But Search Overall As Disappointing
Jean Véronis has published the results of a relevancy study he conducted with
his students at the Université de Provence showing that, at least for French
searches, Google and Yahoo are tied on the relevancy front but that search
engines overall disappoint. However, he also takes a different spin on the stats
that give the search engines higher marks and puts Yahoo slightly ahead of
Google.
Jean summarizes the study
here,
with a more detailed PDF rundown
here.
Google, MSN and Yahoo along with Exalead, Voila and Dir.com were evaluated. In
summary, here’s what was done:
Scores were:
As you can see, no service gained above the 2.5 middle ground on the 0 to 5
scale. But at the suggestion of Jakob Nielsen, Jean did a
new
comparison of scores, to take into account that users may be happy if they
get at least one good result per search, rather than 10 of them. So — if I
understand right — he calculated the average highest score for URLs on search
results pages.
In other words (and again, if I understand right), he looked at all the search
results pages from Yahoo, then looked at the highest scoring URL on each page,
then figured the average. Relevancy shoots up with this much tighter
examination, and Yahoo pushes slightly ahead of Google, while MSN and Exalead
also close the gap: