Search Engine Watch
SEO News

Go Back   Search Engine Watch Forums > General Search Issues > Search Technology & Relevancy
FAQ Members List Calendar Forum Search Today's Posts Mark Forums Read

Reply
 
Thread Tools
Old 01-04-2005   #1
AussieWebmaster
Forums Editor, SearchEngineWatch
 
AussieWebmaster's Avatar
 
Join Date: Jun 2004
Location: NYC
Posts: 8,153
AussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant future
What is relevancy?

The word gets bandied about so much, but realistically you can have 10 people searching using the same words and yet not be looking for the same things...
How do you measure relevancy? There is a need for some set of criteria - give perspective and also something to measure results against.
AussieWebmaster is offline   Reply With Quote
Old 01-04-2005   #2
glengara
Member
 
Join Date: Nov 2004
Location: Done Leery
Posts: 1,118
glengara has much to be proud ofglengara has much to be proud ofglengara has much to be proud ofglengara has much to be proud ofglengara has much to be proud ofglengara has much to be proud ofglengara has much to be proud ofglengara has much to be proud of
I don't believe in relevancy, much to subjective IMO.

Topicality on the other hand.... ;-)
glengara is offline   Reply With Quote
Old 01-04-2005   #3
orion
 
orion's Avatar
 
Join Date: Jun 2004
Posts: 1,044
orion is a splendid one to beholdorion is a splendid one to beholdorion is a splendid one to beholdorion is a splendid one to beholdorion is a splendid one to beholdorion is a splendid one to behold
Exclamation It's a moving target

If you look for a technical definition, here is

"A judgment which relies heavily on semantics.", taken from Employing Semantical Issues in
Syntactical Navigation


If you are not into IR technical formulas, the simplest way I define relevancy is by asking, what is not relevant to the reader, but again this is a bit subjective. One needs to make a hypothesis and then try to disprove it, like this,

Hypothesis: "If a document and a query have a (key)word in common,
the document is likely to be relevant."

Then you need to disprove this. Some IR systems precisely are designed for the sole purpose of disproving this hypothesis. Others, are not so, but for monetary gain. Here is where IR folks get in trouble.

Smarter systems work around this hypothesis using high level semantic technology such as LSI, topic analysis, etc., which do not depend heavily on literal keyword-document matching but on concept matching. So, either you're on-topic or you're not.

Orion

Last edited by orion : 01-04-2005 at 06:39 PM. Reason: fixing typos
orion is offline   Reply With Quote
Old 01-04-2005   #4
Mel
Just the facts ma'm
 
Join Date: Jun 2004
Location: Malaysia
Posts: 793
Mel is just really niceMel is just really niceMel is just really niceMel is just really nice
Relevancy to the words searched for in a search engine?

Relevancy to the topic searched for in a search engine?

Relevancy regarding links?

Relvancy for the searcher?

IMO all of these may need a different definition.

Search engines may use onpage relevancy factors, off page relevancy factors and link relevancy factors, but none seem to be able to develop an algorithm that simply provides relevancy for the searcher.

While relevancy to the search term may be the goal of many search engines (and by and large they achieve that goal fairly well) trying to define the relevancy of a link between two pages is a much harder process, since ultimately it involves trying to detemine the intent of the person placing the link.

As an example if I happen to like a particular pizza so much that I place a link (perhaps accompanied by a brief note) to that on my SEO site is that link relevant?

If I provide a link to a site that sold me a new Jaguar auto, is that link relevant if I place it because I was happy with the car or the price? What if I placed the link because I got a bettter discount for doing so?

What if the Jaguar distributor places a link to the same site. Is that link more relevant than mine simply because they are in the same business? Or is it less relevant because thay have a financial interest in selling Jaguars?

Relevancy a great topic.
__________________
Mel Nelson
Expert SEO Dont settle for average SEO
Singapore Search Engine Optimization and web design
Mel is offline   Reply With Quote
Old 01-04-2005   #5
AussieWebmaster
Forums Editor, SearchEngineWatch
 
AussieWebmaster's Avatar
 
Join Date: Jun 2004
Location: NYC
Posts: 8,153
AussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant future
I agree the definitions cross the measurement, semantic, financial and numerous other areas. That is why it becomes so subjective and to claim an engine is not being relevant can elict such strong responses.
The SEO industry is motivated by financial demands - the fact that ODP has had such problems shows there may not be as many altruists in this industry.
The idea that hobbyists may donate time to build a global project obviously has not held up.

Some engines are relevant for different aspects of search. By recognizing what the factors are and which engines address each area we are better able to deal with them.
AussieWebmaster is offline   Reply With Quote
Old 01-05-2005   #6
projectphp
What The World, Needs Now, Is Love, Sweet Love
 
Join Date: Jun 2004
Location: Sydney, Australia
Posts: 449
projectphp is a splendid one to beholdprojectphp is a splendid one to beholdprojectphp is a splendid one to beholdprojectphp is a splendid one to beholdprojectphp is a splendid one to beholdprojectphp is a splendid one to beholdprojectphp is a splendid one to behold
Relevance has two meanings IMHO:
1. Relevance of a specific result.
2. Relevance of SERP.

These are very different. For result to be relevant, it needs to relate to the query, tht is have some relationship to the words typed in.

Then there is the relevance of a SERP. What makes a whole page relevant is, in a word, variety. What is a user serching for "dog", as you cn see a popular serch, really looking for?

IMHO, as it could be anything, the Search engine, on a single SERP needs to provide variety in order to be relevant.

So relevance, IMHO, is variety in the majority of serches in which semantic intent is not understood.
projectphp is offline   Reply With Quote
Old 01-05-2005   #7
Dave Hawley
Please remove heart from sleeve before replying
 
Join Date: Nov 2004
Location: Australia
Posts: 573
Dave Hawley will become famous soon enoughDave Hawley will become famous soon enough
I don't believe it matters how we measure relevancy, but rather how SE's do.
Dave Hawley is offline   Reply With Quote
Old 01-05-2005   #8
dannysullivan
Editor, SearchEngineLand.com (Info, Great Columns & Daily Recap Of Search News!)
 
Join Date: May 2004
Location: Search Engine Land
Posts: 2,085
dannysullivan has much to be proud ofdannysullivan has much to be proud ofdannysullivan has much to be proud ofdannysullivan has much to be proud ofdannysullivan has much to be proud ofdannysullivan has much to be proud ofdannysullivan has much to be proud ofdannysullivan has much to be proud of
Ah, relevancy. As Mel said, there are a ton of different factors that come into play. And if you want to try and measure it, even the same test might yield different results.

I did a big piece on this back in Dec. 2002: In Search Of The Relevancy Figure. It talks about how people measure relevancy, such as anedotal evidence (I hear Google is the best), ego searching (I found myself; Google rules!), mega searching (I got X number of results, which was more than on this other search engine, so this one must be great), binary rating (is this result good or bad. All good, fantastic -- but doesn't measure the degree of good).

You might also check out this sidebar to the above article, AltaVista, Overture Speak Up About Perfect Page Test and one example of a very specific type of relevancy test we did, The Search Engine "Perfect Page" Test.

All those are from the end of 2002. Inktomi, Google Win In Recent Relevancy Test from 2003 looks at testing VeriTest did, and from the middle of last year, this write-up looks at testing Vividence did.

The Vividence findings, the more recent relevancy test we have out there, are also touched on in Delving Deep Inside the Searcher's Mind, Inside The Searcher's Mind - Live from SES San Jose and Others Close the Tech Gap with Google.
dannysullivan is offline   Reply With Quote
Old 01-05-2005   #9
AussieWebmaster
Forums Editor, SearchEngineWatch
 
AussieWebmaster's Avatar
 
Join Date: Jun 2004
Location: NYC
Posts: 8,153
AussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant futureAussieWebmaster has a brilliant future
Quote:
Originally Posted by dannysullivan
Ah, relevancy. As Mel said, there are a ton of different factors that come into play. And if you want to try and measure it, even the same test might yield different results.

I did a big piece on this back in Dec. 2002: In Search Of The Relevancy Figure. It talks about how people measure relevancy, such as anedotal evidence (I hear Google is the best), ego searching (I found myself; Google rules!), mega searching (I got X number of results, which was more than on this other search engine, so this one must be great), binary rating (is this result good or bad. All good, fantastic -- but doesn't measure the degree of good).

You might also check out this sidebar to the above article, AltaVista, Overture Speak Up About Perfect Page Test and one example of a very specific type of relevancy test we did, The Search Engine "Perfect Page" Test.

All those are from the end of 2002. Inktomi, Google Win In Recent Relevancy Test from 2003 looks at testing VeriTest did, and from the middle of last year, this write-up looks at testing Vividence did.

The Vividence findings, the more recent relevancy test we have out there, are also touched on in Delving Deep Inside the Searcher's Mind, Inside The Searcher's Mind - Live from SES San Jose and Others Close the Tech Gap with Google.
And that post, folks, shows you why people quote Danny Sullivan!

Impressive reply there mate.
AussieWebmaster is offline   Reply With Quote
Old 01-05-2005   #10
Orange Hues
 
Posts: n/a
Its me me me

Relevancy is simply how quickly the results work for me. It's deeply personal.

Wrote about it in a recent post on the Google/ Microsoft/ Yahoo search wars.
Quote:
Web searches as they are done today, can be divided into three main components. Conducting search - translating your need into a query and entering it in the search interface. Result presentation - the results UI, the way they are presented for interpretation. And result accuracy - relevance of the results or search precision.

Any quantum leap in search user experience in future would have to occur in search relevancy as it has in the past. Improvements in conducting search and result UI, Although important, can only enhance the user experience incrementally. They are irrelevant if results aren't accurate. What's most important is to lead the user to what he's looking for. If a search engine can predict that even with failing accuracy, it will be the beginning of the perfect search.

Such precision in result accuracy however, is held back by what can be called the Java paradox. As Yahoo's Jerry Yang recently noted in the Yahoo! Search blog, "people expect to find precisely what they're looking for exactly as it relates to them. It's the old example of the "Java" search query. Are you looking for coffee or for the programming language?"

When Sergey talks about the perfect search engine being like "the mind of God," he's perhaps talking about this very mind-reading quality of a search engine.
Google and the Great Mousetrap Fallacy
  Reply With Quote
Old 01-07-2005   #11
orion
 
orion's Avatar
 
Join Date: Jun 2004
Posts: 1,044
orion is a splendid one to beholdorion is a splendid one to beholdorion is a splendid one to beholdorion is a splendid one to beholdorion is a splendid one to beholdorion is a splendid one to behold
Exclamation Relevance Conference

For those of you interested, back in September 04, a KES Conference session on relevance was held in New Zealand. The Call for Papers, which I didn't attend is given below.


8th International Conference on Knowledge-Based Intelligent Information and Engineering Systems (KES2004)

Wellington, New Zealand
20-24 September 2004

Aim and scope

The aim of this session is to provide a forum for researchers to exchange ideas on the meaning and measurement of "similarity" in content-based multimedia retrieval. In information retrieval, the terms "similar" and "relevant" are often used interchangeably, usually without any supporting justification. In many content-based multimedia retrieval systems, the (usually tacit) assumption is made that similarity is captured by some metric on a feature space—often Euclidean. We solicit papers that address these issues, and particularly those that investigate the implications of the choice of a given similarity measure on retrieval performance, in terms of both accuracy and efficiency.

Topics of interest include:

1. the nature of similarity
2. the relationship between similarity and relevance
3. psychological and psychophysical data on visual similarity judgements, as 4. applied to multimedia retrieval
5. mathematical approaches to the measurement of similarity
6. performance evaluation of differing similarity measures
7. Publication

I look forward to Springer-Verlag's Proceedings

Some comments.

Note the distinction between relevance and similarity. When we talk about relevance and ranking we are often discussing similarity scores and sorting of those scores (ranking) but this is not necessarily makes the results relevant. Here is where one need to make a hypothesis and try to disprove it. Now if you add a commercial component to the mix, forget about the experiment.

One problem about relevance is how similarity functions are used in connection with relevancy. One of many components that affects this is the local and relative nature of some similarity functions S(q, d) that are applied over a text stream, (most notorioulsy Dice, cosine, Salton indices and other sim functions).

For instance, one can selectively remove one letter at a time from the word "black", create a new word, and then mesure the similarity between the two with Dice sim. Keep doing this and soon you end with the word "white". Globally, there is no similarity, but a the local level one can measure a similarity value.

I can do this evaluation with the words "luck", "buck", "suck"... you get the idea. Or I can use a pairwise Dice sim measure for different words such as

"fr-en-ch"
"wr-en-ch"
"br-an-ch"

Still, these sim scores lack of a semantic component.

The point is that IR systems that rely heavily on these type of measures often lack of conceptual and semantics, still some progress has been made recently. This is one of the things that makes me stay away from classic models and brute force search engines as any ranking measure or relevance measure could be faulty.


Orion

Last edited by orion : 01-07-2005 at 12:44 PM. Reason: typos, refining lines
orion is offline   Reply With Quote
Old 01-20-2005   #12
Black_Knight
Ancient SEO
 
Join Date: Jun 2004
Location: UK
Posts: 152
Black_Knight is just really niceBlack_Knight is just really niceBlack_Knight is just really niceBlack_Knight is just really niceBlack_Knight is just really nice
For me, relevancy is a combination value based on three other attributes:
Topicality, Context and Timeliness.

Many folks put way too much attention on the first part, topicality, then just slightly miss the boat on the second, context. The personal taste or subjective part of relevancy is largely a matter of context. The context of what I already know. The context of where I am. The context of 'why' I have searched, not just 'what' I have searched.

This is why a record of search history can be so valuable to search engines. When someone searches for 'thunderbird' there are so many possible options. Do they mean the car, the TV puppet show, the drink, the email client, the mythical beast, or something else entirely? There isn't enough contextual info in the word alone. Knowing that their previous search was for 'Ford' would naturally supply a good clue to context.

Some engines offer refinement suggestions or sub-phrases to help increase the contextual information provided. This has proven to be a strong help in providing a better overall search experience. User experience is also a great help. More experienced searchers are better at providing more contextual information in their query, and of making more specific queries. Providing the 'advanced search' options helps accelerate this learning process for searchers, showing them ways to easily create more specific queries. To provide more context.

The phrases used may themselves provide a form of context. A simple phrase may provide information that this is not an advanced searcher, and may even be a person looking for a general overview, not yet knowing themselves which aspects they are particularly interested in.

Any skilled PPC advertisers are well aware of this factor. That generic phrases are earlier in the shopping cycle. People looking to see what is available and not yet giving any serious consideration to buying anything. As they get more serious, they add more personal context, like a favourite/trusted brand they'd prefer. Or a price factor (e.g. cheap).

Then there's timeliness. This too can be contextual at times. A search for 'tsunami' now is searching for something different to what would have been sought by that phrase before the disaster. But even when not in context of big events, many searches will want accurate information. Later documents are often (but certainly not always) likely to be more advanced than earlier documents. This is where link popularity becomes a handicap, because it favours older, more established documents over newer, fresher ones.

I do think that search engnes need to provide more 'prompts' for contextual clues. It is the only real method for being able to serve more contextually relevant results.
Black_Knight is offline   Reply With Quote
Old 01-24-2005   #13
ericgoldman
Newbie
 
Join Date: Jan 2005
Location: Milwuakee, WI
Posts: 2
ericgoldman is on a distinguished road
A lengthy discussion of search engines and relevancy

I have summarized and analyzed some of the information science literature about "relevancy" in this lengthy article:

http://papers.ssrn.com/sol3/papers.c...ract_id=635803

I also discuss how advertisers, search engines and consumers all want to improve the flow of relevant content. I welcome your comments. Eric.
ericgoldman is offline   Reply With Quote
Old 10-21-2005   #14
Dave Andrews
Member
 
Join Date: Sep 2004
Location: Levin New Zealand
Posts: 17
Dave Andrews is on a distinguished road
Relevancy results.

Relevancy needs to be a measure of the relationship between the meaning of the words searched for, so that the content that is displayed in the search results is accurate.
The word "Love" could be searched and it would give say 30,000 results because the word is converted into a binary sequence and then found in the database.
It doesn't search for ways that the word love can be meant or used, it doesn't find words of simlar meaning or related word meanings.
I have just developed a new method of page ranking which is now in use in my search engine. Links are important and it is an easy method of forming some order in the chaos of the web pages and websites being indexed to give a way of ranking pages.
The person doing the search is looking for content about what they are trying to find, to them is doesn't matter where it is, just as long as they can find what they want?
So a websites ranking doesn't count that much, page ranking should be based on the content of the page and also the relevancy of that page to the meaning of the word or words that were searched for.
Some search engines can give billions or results to a keyword search but heaps of the pages are useless links, because they simply contain the searched for word which doesn't relate in any form to the meaning of the word that was searched for in the first instance.
We need to look at the way a search engine gives the results, it's not the sheer amount or results it's in the quality of the content found.

When a person searches the web they are in most cases, not looking for an accurate cold result which is based on computer logic but for things that relate to what they are searching for in loose way.
I am interested in finding something about say "farming" so I do a search and start surfing, where I end up has very little to do with what I searched for in the first place but it will be related and have some relevancy to what I started out looking for!

Webpages should be indexed and ranked by content and the related meanings of the content should form part of the results to enable the meaning of the words and the relationships between the meanings to count towards it's importance.
So I will go back to my coding, of what I call the "Why Factor" which is a page content, word meaning and word relationship calculation that interacts with the meaning of what the searcher is searching for and gives the engine the scores to make the search results pages.
Is that Relevant my friend??

heaps of regards
Dave Andrews
Dave Andrews is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off