Does MSN Ignore Too Much?

[Chris Boggs]

I received an interesting email from a colleague, Andy the Analyst, who pointed out some results in MSN Live that matched up exactly. He showed me that a search for "better," "for better or for," and "for better or for what" all yielded the exact same SERPs. Obviously, MSN is ignoring the words "for," "or," and "what" in these examples, but it seems strange that they would rank the results set all the same - geared towards the word "better."

We are curious as to what others think about this. Does MSN ignore too many words, especially if taken in the context of a phrase like "for better or for," which one would think should at least deliver some results for pages using the phrase "for better or for worse?" Is it a bug that will be fixed?

[Marcia]

It isn't a bug, because it's ignoring very commonly ignored stop words.

Does MSN ignore too many words, especially if taken in the context of a phrase like "for better or for"

Nope, FINDALL wouldn't look for context, using quotes would find exact match and sequence.

Question: would the expression high water be in context for high water level of a lake or river, or high water as in "come hell or high water?"

[Chris Boggs]

Yeah Marcia I realize that they are ignoring stop words, but the way that they present the remaining results is different from G, Y!, and Ask. It seems as if there must be a "missing step" in the MSN algorithm, which could make your question:

would the expression high water be in context for high water level of a lake or river, or high water as in "come hell or high water?"

actually more difficult to answer if you are using MSN:

You may have to actually do a second search, to find examples of both. In Google, both contexts appear on page one. Yahoo and MSN are both devoid of a mention of "hell or/and high water." In relation to my example, though, you should try the search for "(or)(and) high water" on each engine, and you will see that MSN doesn't change the results. In this case though, G doesn’t either.

In either case, it just seems as if the MSN algo is lacking a je-ne-sais-quoi and I hope we can attempt to figure it out in this conversation.

[Silver]

of course, if you put quotes around the phrase, it will show different ranking/results when compared with "better".

[fulton savage]

I think the most interesting question about these types of issues is this:

How many users know of/use search query operators?

[mcanerin]

That's really the problem, fulton savage.

The vast majority of users don't use quotes or operators. Therefore the default behaviour of a search engine is a very important issue. It doesn't matter how they act when operators are used (they should all work the same), it matters how they act when explicit operators are not used.

At this point, the search engine needs to go into "handholding" mode, basically trying to guess at the best methodology for the words typed in. Generally up until recently, this meant assuming the AND operator.

But this assumption can be too simplistic. What if there are only a few results? Perhaps in this case a search engine would place and rank the AND list first, then perhaps throw in a bunch of OR results to fill in the blanks.

Without an explicit operation requested, it's really up to the search engine how to interpret the request, and there is evidence to support that the search engines are tinkering with how to interpret queries without operators in order to give the most helpful results.

For example, if you type in a search containing a locale that Google recognizes, (let's say "Las Vegas hotel", for example) it may decide that rather than just assuming that it's been asked to look for pages that contain "Las" AND "Vegas" AND "hotel", it will look for sites that it knows contain the term "hotel" and are in Las Vegas, even if the owners don't have the word "Las Vegas" on the home page that is returned. It may also return sites that just contain "Vegas" because that's a common short form for Las Vegas.

Is that a better or worse result? Well, if you were an expert researcher expecting an explicit AND operator, then it would clearly be a worse result, since it's giving you listings that don't comply with your expectations. That darn search engine!

But if you were Joe Searcher, you are probably just looking for a hotel in Las Vegas. You don't care about keyword density, or search operators, or any of that stuff. In this case, you are given what you want. They even go a step further and provide local search results, for hotels that might not even have a website.Yay search engine!

I'll give you a prediction - I can see a time in the very near future where a key differentiation between search engines is the type of handholding they do with searches that don't contain explicit operators.

As you can see from the "Las Vegas hotel" example, assuming that the searcher is intending an explicit AND is too simplistic. The future of personalized search is based on the assumption that users are not expert searchers and that they expect the search engine to try it's best to help them, not to cop out and choose the AND operator just because it's easier to do.

Chris, I'm wondering if the lack of the expected functionality you are seeing with MSN is related to this. Perhaps they are doing the wrong type of handholding (or none at all).

Just thinking out loud,

Ian

[Chris Boggs]

As usual, Ian, you add great insight to a thread. I think your handholding idea is "spot on."

(Channeling Andy the Analyst)

That is an insightful message

I think that sounds right – that it is a smaller degree of handholding. While our example is not very common in a search, I think it identifies a way that the MSN search algorithm may be misinterpreting the information need of a user. Perhaps their users are more oriented to search results of this nature, but I think it is hard to know without a study of how users are pleased/unpleased with MSN results. I also consider that if other mainstream search engines handle queries differently based on additional words (even stop words) in the query, there may have been studies that gave them a reason to implement this.

For this specific issue on handling queries, I feel confident that there is some proof out there that a user performing a query expects refined results when adding additional terms to the query. To throw out terms other than the common ‘and’, ‘or’, etc. without notifying the user seems to me a poor ‘handholding’ – but again, maybe MSN users search differently than Google or Yahoo users, in this regard?

I have to concur with Andy that there is a difference in audience that must be accounted for. However, in my experience, I would wonder if in fact MSN should be better off by asking further questions. Granted, in some of the examples above, Live does provide the "missing" elements within the suggested similar categories/searches displayed on the upper right, which is good, IMO. But people are still so focused on the organic area that I fee this could actually be almost wasted on the MSN audience because of the location. I think that is more of a style that Ask.com users have come to expect and are more proficient in.

Marcia, do you not feel that MSN "owes it to its users" to try and handhold a little?