Relevancy's Importance in Microsoft's Goal for MSN Search

[rustybrick]

I'll prefix this post by telling you I wrote something very similar at the search engine roundtable (http://www.seroundtable.com/archives/000622.html) on this topic, but I really wanted to get your feedback on my thoughts on MSN and what it holds for search as a whole. So here it is:

Google is the number one search engine because of its search results. Google's search results (to the most part) are more relevant then all the other search engines. Every competing engine talks about how and why their search algorithms are better at bringing back more relevant search results then the next. Most say, that is why Google is the leader in the industry, they have more relevant results then the rest.

My question is, will relevancy make a difference when Microsoft enters the search race?

I am not questioning whether or not Microsoft can build a more relevant engine then Google or Teoma. What I am questioning is if Microsoft needs to build a more relevant or equally relevant engine.

Let's take two areas where Microsoft monopolizes on. The first is the operating system market. Is Windows a better and more user friendly operating system then Apple's OS X or Linux? I don't think so. How often do I hear people complaining about how stupid Windows is. All the complaints on computer crashes and issues with viruses or spyware make me sick. Why do you Windows people accept these issues? Why don't you switch to an Apple computer? Second example, Microsoft's Internet Explorer. In reality it is a poor example of a Web browser. All the IE exploits where a hacker can take over your computer and destroy all your files, simply because of a single exploit in IE. IE doesn't have tabbed browsing or other basic functions one can find in Firefox and other popular, non Microsoft, browsers. Why do you use IE?

The answers to these questions is that you use Microsoft products because you always had. Computers came out, it was easier and cheaper to buy a PC running Windows on it then to buy a PC with Linux or an Apple computer. Now that you have Windows pre-installed on your PC, hey, Internet Explorer is on it, so its easy and cheaper to use IE then an other browser.

I can see the same logic being used for MSN Search. There is no doubt in my mind that MSN Search one day will be built into the Windows OS. It will be easier and cheaper to use the built in and pre-installed MSN Search then to use Google, Ask or Yahoo. You'll "live with" less relevant results, because its easier and cheaper to use something that is pre-installed. And as MSN Search gains market share, the results will get worse and worse. Just like IE keeps getting worse and worse, as compared to the newer browsers available today. Why? Because Microsoft has captured the market and the searcher has no where else to go. Because its too hard to type in google.com into my browser, I have MSN Search built into my OS.

I am afraid for the future of search, a future where relevancy does not matter as much.

[Ron Carnell]

In at least one sense, Barry, I think you're comparing apples and oranges.

One of the great weaknesses recognized in the SE business model is that they have no way (so far) to trap their users. Google's popularity, for example, is only as good as last month's relevancy. If another demonstrably better engine surfaced tomorrow, relatively few users would have any reason to maintain their loyalty to Google.

It's a bit different, however, with operating systems and even, to a lesser extent, with browsers. I can only speak for myself, but I suspect my sentiments will be echoed by many other experienced computer users. As much as I like the Mac and Linux, I stick with Windows because no other OS has as much software written for it as does Windows. Just as a new forum struggles to attract users so it can become popular enough to attract users, the same vicious circle faces an operating system. If Mac had a larger user base, more developers would port software to it. But if more developers ported software to it, it would quickly gain a larger user base. Of course, the flip side to that is that if Mac had a larger user base, it would attract more virus attacks, too. For an operating system, dominance is both a strength and a weakness, and I think that is epitomized by Windows. It is and will remain ubiquitous in large part because it's ubiquitous.

While this is less true of browsers, the same nonetheless still applies. Do you want to run the Google toolbar? Guess which browser you're going to use. While Firefox is quickly becoming my browser of choice, I will never abandon IE as long as the tools I want and need are only available in IE. Its dominance, again, is both its greatest strength and greatest weakness.

Google's current dominance, on the other hand, is much more transitory. They have no way to lock me into their SE, and frankly, if Teoma could scale their algorithm to a substantially larger index, I suspect I would be doing most of my searches over there already. My loyalty to Google is defined entirely by their superiority, not by their current dominance.

It remains to be seen if MSN can use Longhorn to "lock in" users, but I honestly think it's going to take more than a link or desktop shortcut to do the trick. To capture a dominating user base, MSN is going to have to offer something that can't be offered by a non-integrated search engine.

Lacking that USP, I think search relevance will still remain the most important criteria.

[DianeV]

Good points, Ron.

I bought the first Macintosh computer in 1984. Needless to say, it came without a hard drive, which we only learned after we paid $3,500 and carried it home, or even a second "floppy" drive (which wasn't available for some time) . To save a file, one had to swap the blank diskette with the OS diskette, back and forth, sometimes over 20 times. That was for one file save. $3,500 in 1984 wasn't nothing. When the first Mac hard drive finally came out, they wanted another $1,000 for it. On principle, I will never buy a Mac product again. It's poor business judgment to dump on your early supporters and expect anything more out of them.

Ron's right about the software, too. Recently I watched a diehard Mac fan discover that several programs she wanted to use simply wouldn't run on a Mac. As to viruses, mostly it takes some common sense, and good security (and the first Mac virus was discovered, to my knowledge, this year; it apparently wasn't created to be damaging, but there it is).

Perhaps it needs to be repeated that other search engines have held the dominant spot in years past, only to lose it. As to Longhorn, the one thing I will note is that I was amazed some time back to hear non-Web people say they "loved" Google. These were non-Web folk, for whom AOL or MSN was the ISP (with AOL or MSN as their browsers' home page), so they clearly had to jump through a hoop or two even to find Google.

[David Wallace]

I would like to think that relevancy will win but Microsoft has dominated the OS industry even though they do not have the best operating system. They have dominated the browser industry even though they do not have the best browser. I could go on and on.

It is like Ron said - people use Windows because many of the software products they use won't run on Linux or other OS's. People use IE so they can run the Google toolbar. So it is more of convenience that these two products dominate rather than them being the best.

If they are going to dominate the search engine industry, I don't believe they will be able to get by with a mediocre product, at least I hope they will not.

[rustybrick]

I am really not sure. I have a really bad feeling that in 5 years we will see MSN at the top. I feel if that happens, the relevancy at MSN will be less then at other search engines.

Oh, in regards to not buying an other Apple since 1984, well I think that is kind of extreme. Now Apple computers come with everything (except a floppy). Built in CD/DVD burners, firewire, wireless g, bluetooth, dual processors standard in all G5s and much more. Today they go way beyond Dell, IBM, and HP when it comes to hardware. Its more expensive of course. Ok, sorry for going of tangent.

Point is, I feel that everything that Microsoft has monopolized has made for an inferior product in the long run (except for Excel). I expect Microsoft to be a major threat to Google and possibly take over search (I am no prophet). When/If that happens, I feel that in the long run, search will suffer in terms of relevancy. Also, keep in mind that relevancy will ultimately improve over the years but the question is, at what lengths?

[wiseMouse]

Point is, I feel that everything that Microsoft has monopolized has made for an inferior product in the long run (except for Excel). I expect Microsoft to be a major threat to Google and possibly take over search (I am no prophet). When/If that happens, I feel that in the long run, search will suffer in terms of relevancy. Also, keep in mind that relevancy will ultimately improve over the years but the question is, at what lengths?

You're not alone with these feelings. One thing is that relevancy is subjective. The other thing is that MSN is one of the most popular web sites anyway... these people will use MSN search unless (a) it's utter rubbish, and (b) they want something really fancy (e.g. serious researchers). I'm sure MSN will be big, because like *all* other Microsoft products, they're popular because it's just there.

[detlev]

Hello everyone,

Well, at least Yusef Mehdi has said they are not going to prevent access to other search engines :-)

MSFT has the tendency to build user "sand traps" that make newbies default to a Microsoft product offering or service. This is nothing unique to MSFT, AOL does similar things as does a huge list of top companies.

There are a lot of factors that go into relevancy. Freshness and comprehensiveness is the start. How many times have we been disappointed after following a relevant looking result to a 404? How many times have we been disappointed by not seeing the relevant result(s) in the appropriate query set of SERPs?

Search users are fickle, and MSN can offer a compeling product, claim it is the best (like eveyrbody else does) and harbour a boat load of search users to gain their loyalty away from Google - Yahoo! - Teoma etc.

The thing most likely to immediately gain users is that in Longhorn MSFT will offer a single search box to search across docs on the machine as well as Web. My thought about this is that few will install the alternative brand products (in development to do the same thing) and people will use the MSFT search in Longhorn as default without even realizing they are searching the Web. I simply hope the search interface will make the difference entirely clear even for newbies.

*cheers*
-detlev

[Nacho]

The thing most likely to immediately gain users is that in Longhorn MSFT will offer a single search box to search across docs on the machine as well as Web.
I agree with this and what most of you have said about the idea of users preferring Microsoft products, not because they are better, but because of the "convenience" factor of working well with their Windows operating systems.

I have talked about this before in the past based on what I've heard from experts and authorities on search, some call it "Internal Search: Unlocking the Search Within (http://www.webproworld.com/viewtopic.php?t=15219)" and others call it different. Whatever it's called it does not matter. What's important is the concept behind it.

http://www.mexgrocer.com/lib/mex-grocer/internal-search.gif

In the present time, when you search the web you get results in seconds or a fraction of that from many current search engines. That is because technology and human intelligence have made web crawlers be able to index documents and find relations among them via hypertext links and return a fast result. Among many other factors that determine a search engine's algorithm, link relationships (relevant link popularity) is among one of the most important to find out which document is the most significant to the search query.

Today, this efficiency is not the same when you search documents within your computer or network environments with software developed from current technology. Sometimes it takes seconds to find on most popular Microsoft versions, but at times it may take minutes to find the right document. Therefore, if all documents (doc, worksheets, email, presentations, etc.) on your computer and your network environment (ie. work, home network, etc.) could be searched in a similar ability as the web (for example, as if they were transformed into html/xml documents), then you would get results from the most popular or relevant documents showing in a ranking format and hopefully time of response would be just as fast. This would then make the user more efficient by receiving results faster and making better decisions when going to the next task or decision. When this efficiency is multiplied by many times, then we can see very significant value and increase in productivity.

This could, of course, violate many privacy matters when searching outside your own personal computer and shared network that should not have access to your documents. This could also turn into other potential lawsuits for those creative minds out there.

I spoke with Danny Sullivan about this (back in March 2004) and he said, "Keep in mind that MSN Search and Microsoft operating systems are two completely different things. Right now the more important question is 'what is MSN really thinking to develop in new search technology?' and they are probably still thinking about it without any actual direction." He then mentioned that this has been studied in the past by one of his contacts, but the concept was then dropped.

The entire concept came to mind because what if you could take email and each message is transformed into an html document, where the mailto and the mailfrom can be seen as outbound and inbound link structures. You could then easily determine communities and authorities by analyzing communications and keywords from the messages. All email documents (in html) would be spidered for building the index and assigning algorithms to determine how a document would rank high.

Then, if that same concept could be replicated into just about any working document, then it could break new grounds on the future in search technology for our more internal universe, rather than the existing external universe to achieve a very powerful global productivity tool.

To me this is just a concept or theory of what could be part of the future in search. Hopefully there will be a company that can allow me to search my computer, my work environment or the web in a fraction of a second, just as Google or Yahoo! Search have accomplished today for the web.
So what if Microsoft could build this into their operating systems and allow you to easily search: My PC; My Network; The WWW with one single box carefully placed where you can efficiently use it? Then add a big dose of "relevancy results for www" (as good or better than the current engines) . . . YEAH! You might find that to be the new leader thanks to GREAT CONVENIENCE, IMO.

[Mel]

First off the comparasion of search engines and operating systems seems to me like comparing trucks and factories, there may be some similarities, but there are many more differences.

Is it really any more convenient to search directly from your operating system than from a browser configured to start up automatically with a Google search window?

The idea of one great big box that can search not only the web, but you personal computer and the companies server may sound great but to me it sounds more like a nightmare.

Who is going to have access to all this information?

Certainly not the Government or we will have to write a new constitution.

Certainly not the public or Ford would set up a department of Cadillac research.

Certainly not everyone on a company intranet, or the shipping clerks would be reading the Personnel and Finance Departments darkest secrets, not to mention the new ad campaign getting wrong footed.

Certainly not any individual who wants to read my emails.

Yes it is technically possible, but is it desireable?

I am not a prophet, but I would still bet that in five years time I still will not have access to my senators emails, or be able to find out the forumla Shell uses for its antiknocking additive. In other words it will be more like it is today than like some Orwellian novel.

[rustybrick]

I can't believe that some of you guys aren't with me on my prediction. :) Just kidding. I'm sticking with it. We will see how it plays out in 5 years.

But I also predicted the end of the world in 4.5 years, so not sure if we will get that far. (Trying to be funny, not sure if it worked) :)

[Mel]

While this is less true of browsers, the same nonetheless still applies. Do you want to run the Google toolbar? Guess which browser you're going to use. While Firefox is quickly becoming my browser of choice, I will never abandon IE as long as the tools I want and need are only available in IE. Its dominance, again, is both its greatest strength and greatest weakness

Firefox has a plugin which gives you all the features of the Google toolbar as implemented on IE. I am running Firefox with this installed and the only time I see an IE browser is when I click on a link in an email and Outlook insists on opening an IE browser.

http://www.googlebar.mozdev.org/