PDA

View Full Version : Which Search Engines are Playing Copy Cat?


rustybrick
03-14-2005, 09:28 AM
Over the past several months I have been thinking about this topic on and off. Which search engines are the true inventors of the future of search and which are the followers?

Of course, I think we all would agree that MSN is, at this point, a follower.

How about Yahoo!? What about Google?

A point driven into my head at the Search Convergence (http://forums.searchenginewatch.com/showthread.php?t=4470) session at SES NYC, was that Ask Jeeves has been providing "smart search" results for TWO years now. Only recently (within the year, I believe) has the other engines been doing so.

Think of the Teoma search engine technology...It is pretty unique compared to the others. Think of the Butler, they had a whacky Butler before Google had their whacky Google logo (right?), before MSN had that whacky butterfly man, before Yahoo! had those Yoodeling commercials.

So who is the true leader of this industry? Which engines really invent? Who is the copy-cat?

andrewgoodman
03-14-2005, 09:58 AM
Jeeves, to me, is a lot like Google or Yahoo in the sense that it is a brand that has been able to stay around and continue to test new features that consumers might like, mainly because that brand awareness has kept people coming to the site.

But on the whole, I still think Google is a bit ahead in the sense of really practical search features. Some things that come to mind are (1) Google Maps - quick scroll instead of reload; (2) GMail searchability in an instant; (3) desktop search that works very fast; etc.

Also, do we forget that Google invented AdWords, which was on numerous fronts an advance on the PPC auction popularized by Overture. That auction has allowed Google to keep a clean page - it's even helped partners like Jeeves monetize their traffic and take steps towards reducing clutter.

On pure algorithmic search, Teoma may definitely have a few advantages, but these kinds of advances can be subsumed/copied by other brands. In that sense, you make a good point. Others may wish (or try) to copy some of what Teoma does... just as AltaVista, Inktomi, FAST, et al. rushed to incorporate link analysis into their algos after Google proved its value to searchers.

Google has made some good moves -- not all of the "invented here" variety, either. Acquiring Applied Semantics and Deja (Usenet archives) were just a couple of great moves that helped Google quickly consolidate its position.

Jeeves' stock is valued decently in the marketplace. While "invented here" may be doing OK for them, I think there is still an opportunity or two for them to acquire additional "cool" technology without it stretching their finances too much. If you look at the history of the past few years, key acquisitions, not just in-house innovation, have helped the leaders in this business consolidate their lead, and reach whole new emerging market segments. There are natural language search companies out there that could be worth going after in a year or two, for example. But as we have seen with Google, counterintuitive acquisitions like Blogger, while not "core," have no doubt been profitable. How much did Blogger cost them? Like $15 million?

I guess what I'm saying is sticking to your knitting doesn't always cut it. Microsoft is saying "we can hire a bunch of people to copy what the others are doing." Short term they may be right, but perhaps Godin is right, that "very good" is in fact very bad. It's those small, nimble startups out there who may be coming up with the innovations that will stun the marketplace.

MUSCLE13
03-14-2005, 11:24 AM
NY Times Excerpt 3/10 -

Ask Jeeves will introduce technology this spring that will further the question-and-answer abilities of its engine. The new feature, Direct Answers From Search, will search across the entire Web, rather than simply from its own database, to find answers to natural-language queries (that is, those phrased as questions rather than mere search terms).

"This allows us to answer far more questions than would be possible using editors or structured databases," said Jim Lanzone, the company's senior vice president for search properties. "When you're diving into structured databases, you're limited in your coverage. We want to harvest the power of the 2.5 billion English-language documents in our index, to more broadly answer people's keywords and questions."

http://www.nytimes.com/2005/03/10/technology/circuits/10sear.html

MUSCLE13
03-14-2005, 11:29 AM
My opinion - "Direct Answers from Search" is probably Jeeves response to Gurunet's Answers.com exploding popularity. Should be interesting.

AussieWebmaster
03-14-2005, 11:37 AM
I remember Ask Jeeves having a fair better system than what exists now with Google Answers years ago.

mcanerin
03-14-2005, 12:08 PM
I see the future of search as returning information, not web pages. If the information happens to be in a web page, or if the request was for a web page, then that's what wil come back, but the focus will be on trustworthy information, IMO.

None of them are really at this point yet, but they all seem to be headed in that direction. Things such as desktop search, local search, stock quotes, etc show this thinking already. Want to be a multi-millionaire? Develop an information resource that can be used by a search engine to give useful answers to questions under the conditions that they have to deal with that no one else has and sell it to one.

I love Ask Jeeves (with the exception of some complaints I've made earlier regarding a small index and high spam/virus ratio compared to it.) Some of their innovations have been (and continue to be) very cool.

But it's not all about innovation - it's also about marketing. I've used the analogy in the past of a musician. There are musicians that have tons of talent but are stuck singing cover songs in bars or writing music no one will play because no one ever heard of them. There are also musicians that everyone has heard of but don't necessarily have any talent other than self promotion.

Then there are the musicians that you were listening to 20 years ago and will still be listening to 20 years from now - they have talent, and they were marketed well. You need both. Being commercially successful isn't selling out your talent, it's gaining the resources to be able to expand on it and fully explore it.

Right now, Google has talent and commercial success. Yahoo has talent and commercial success, but leaning more towards a focus on commercial success. MSN currently has a little of both, but is too new to judge accurately.

Ask has talent, but no real commercial success to date, at least not when compared to G and Y. They are the struggling songwriter of the search engine world.

Someone needs to see that work. Someone other than a small group of search experts and a few die-hards using older portals. They have taken the right step and got financing (I suppose the equivalent of finally signing a record deal). Now they have to convince others it's worth their time to try them out.

Why am I focused on this rather than the original question of who is leading the pack in innovation? Because having money and resources lead to yet more innovation, given the right people and challenges. Creativity when you are hungry tends to take the form of looking for new ways to find food, rather than dealing with more abstract but ultimately more valuable research.

One example of this "looking for food" mentaility is the horrible way they are using PPC results on their site. No one blames them for trying to keep afloat, but it's being penny-wise and pound foolish in the long run. As long as they are starving (or acting like they are), it would be natural to expect this type of behavior. It's ultimately self destructive.

I think ASK has been a leader in the past, and may be in the future. The question is, how are they going to deal with their new-found resources? Invest in more technology and research, or market? What they do, and how, I think will decide their fate.

Ian

PhilC
03-14-2005, 12:14 PM
To my way of thinking, Google are the innovators, and Yahoo! and MSN followed. Even before they launched, they showed the rest how to do 3 things:-

(1) Make more use of links to produce more relevant results - "link analysis", as Andrew put it. Before they did that, one or two engines were simply counting links - linkpop.

(2) Create a very large search engine. Just before Google arrived, big engines were talking in terms of a really massive 1 billion pages, and that was a giant leap from the 100 to 300 million that they'd been boasting about.

(3) Make money from a search engine. The others had been scratching around with portals and all sorts in an effort to make it pay.

As soon as Google showed people #3, those who were able to copy the whole thing jumped into it - Yahoo! and MSN. And, for that, we are grateful because there is more than one basket again in which we can put our eggs.

Both Yahoo! and MSN had web search engines long before Google came along. The results were provided 3rd parties (AV and Inktomi), but neither of them made any moves to have their own engine/data until Google showed how to make money from it.

mcanerin
03-14-2005, 12:20 PM
You know what I'd like to see? Instant Vortals.

Imagine performing a search and being taken to a page (or two or three!) that are basically a vertical information portal on the subject. There would be the requisite links ranked on relevance and authority, of course, but also relevant facts, advertising, authority sites in the area, forum discussions, etc.

It's moving towrds this now, but I'd like to see a more comprehensive treatment and design.

Ian

Mikkel deMib Svendsen
03-14-2005, 12:23 PM
I think it's important to note a few things, that I personally find to be facts of the search industry:

1) The technology race is FAR from over yet! I believe there are still HUGE inventions to be made - completely new ways of doing search technology. Something that will not just improve search, monetization and usability a little bit but something that will make all search as we know it look like a Ford T from the 30s. I have yet to see one single valid argument that what we have now is near the end of search evolution - in fact, I think we are all still newborn babies to this!

2) All current engines have "borrowed" many ideas from each other. FAST could do incremental updates of their index before Google did. TEOMO have pioneered a lot on the linking arena. On the other had they have done nothing on most markets outside English - for example in Scandinavia (where I operate mostly) - Google covers it all and does so very well (Yahoo pulled completely out of here last year!). Overture invented PPC as we know it and Google reinvented it for the better. I don't think you can say anyone of the
major search engines are far ahead or behind any of the others. They have all contributed to what we have now. They are all copy-cats to some degree.

3) Technology is not all! USEFULL technology is what we want. You may be able to build a search engine that theoretically serve better results than others but if I can't find what I am looking for, in the way and in the speed I want to, then it's not good. End of storie! Academic arguments just dosn't work. The average user of the Internet do not care much about technology - they care about usefullness. It's not enough to be able to argue that your engine is better - you have to proof it to the users.

PhilC
03-14-2005, 12:28 PM
>I have yet to see one single valid argument that what we have now is near the end of search evolution.

I think that's because nobody has suggested that the end is near ;)

To the best of my knowledge, FAST have been way ahead of everybody in semantic engines for quite a long time. I understand they've had niche ones running for a while.

rustybrick
03-14-2005, 04:34 PM
Based on all the great feedback on this thread, I have moved it to the Search Industry Growth & Trends forum. Seems like it belongs here.

Mike Grehan
03-14-2005, 05:44 PM
It's a difficult thing to assess in an emerging sector who is leading who, as we really are still at the "tip of the iceberg" in where search is going. On the technology side, information retrieval on the web is still very much in its infancy.

I think it's safe to say that, technological breakthroughs take place fairly frequently in the industry. Some being more easy to implement than others.

As for who influences who? Personally I think Jon Kleinberg's work was an influence on PageRank and Google. I don't have any evidence to support that, other than the fact that his work is cited in Larry Page and Sergey Brin's original paper: The anatomy of a large scale hypertextual web search engine.

There again, Kleinberg's work had an influence on most people in the web search research field.

Google promotes the fact that its server infrastructure is one of its major assets. Yet, I think I'm right in saying that, this type of distributed system approach was first developed and used by Michael Palmer at Inktomi.

Not that the most important person to a search engine, the end user, could give a fig about any of this. My guess is that, the average clueless end user (nitwit :-) buys into the brand first. And as long as that brand serves up relevant results, they'll continue to support it with their clicks.

Certainly each of the brands has its own audience (although there is obviously some brand switching going on). Each has been built on the audience profile. Yahoo! became the brand gateway to the web long before Google was around. They really were THE place to search on the web all those years ago. Then they became the place to go an "do stuff" like news, reviews and email etc.

They relied very much on other partners for crawler based search technology, such as Inktomi, Alta Vista and Google, until they invented their own. Of course, with Inktomi and a gazillion Alta Vista patents that they now own, as well as a search technology pioneer such as Gary Flake on board, they have a lot to shout about.

As Jon Glick from Yahoo! explained to me a while ago, after Yahoo! had finished their shopping spree (buying Inktomi and Overture and acquiring Alta Vista and AllTheWeb in the same deal), they retired to Foster City for a huge geek-out. I certainly expect to see further innovations coming out of the yahoo! stable.

Jeeves, unfortunately had a curious mix of technology and some very mixed branding messages in the early days. They managed to capture an audience of people who wanted to ask questions. It's a shame they were never really able to answer them properly!

However, when Teoma came along, they ended up with, in my opinion, the most superior technology out there. But they still had this very odd brand image to deal with. The Butler's been on a diet and there's a good attempt being made at getting away from being a "quaint" question answering machine and more into a hip search pad.

Google won the hearts and minds of the propeller heads from the very beginning. And I think they still have that techno audience. However, in our part of the industry there's certainly some disillusionment with them over the whole PageRank myth.

We'll come to MSN in a second, but much like Microsoft has a lot of mistrust surrounding it, the same could apply to Google if they don't start fessing up a bit as to what's really going on over there.

Certainly there are some great things coming out of Google labs. Having said that, Andrew's nod to them inventing AdWords is a little tarnished, in that, they based it on GoTo's patented technology which ended up with them being sued!

So - what about MSN? Well the first thing is the branding issue. When personalisation does come to the forefront, then it may be difficult for them to get away from the trust issue that always hangs over the house of Microsoft.

Insofar as the technology goes, it's not as if they were short on this kind of research about information retrieval on the web. They've had people researching in the field for years! They just never moved into it commercially.

Will they invent new technology that will beat the existing methods? Doubt it. As I've already mentioned, this is an emerging field and ideas abound. However, implementing them and making them work depends heavily on the infrastructure. It'll be interesting to see how they grow their distribution network to get a better idea on how they may (or may not) change current methods.

One thing is for sure, and that's that, any new technology will be based around the end user with little or no consideration to the search marketing industry.

I had an interesting discovery conversation today with the panel for a session I'm moderating at AD:TECH, San Francisco. Safa Rashtchy, Harrison Magun and Kevin Ryan are among the most clued up people in the industry. We discussed the upside of the industry and its future - but also the downside. And one of the downsides is the lack of protection for the advertiser i.e. click fraud in PPC and the whole spam issue on the machine-natural listings side.

For those who are involved in cross channel marketing (which Harrison will cover when we discuss integrated marketing) they simply don't come across the problems we have in search marketing, specifically.

The search companies MUST establish better avoidance technologies. Relevancy algorithms are almost there in theory. But the honest search marketer is still always the injured party in all cases of spam and click fraud.

And the relationship between the search companies and the search marketers needs to be radically improved if we're to have any kind of major developments both technology wise and in the commercial sense.

I'll go back to Jeeves for a second. It was a personal and passing comment, but Jim Lanzone did mention to me in New York, that he thinks they should be a bit more open and reveal a little more about what *really* happens to make this stuff work.

Having said that, with due respect to Jim, if the whole industry fell off a cliff tomorrow and advertisers were forced to go direct - you'd less than likely find a single search company executive reaching for the Kleenex.

To conclude... Everybody's inventing stuff in this arena. Who's copying who? All of them probably!

andrewgoodman
03-14-2005, 08:58 PM
Seems like a very exciting development, if Jeeves is going back to focusing on Q&A.

NY Times Excerpt 3/10 -

Ask Jeeves will introduce technology this spring that will further the question-and-answer abilities of its engine. The new feature, Direct Answers From Search, will search across the entire Web, rather than simply from its own database, to find answers to natural-language queries (that is, those phrased as questions rather than mere search terms).

"This allows us to answer far more questions than would be possible using editors or structured databases," said Jim Lanzone, the company's senior vice president for search properties. "When you're diving into structured databases, you're limited in your coverage. We want to harvest the power of the 2.5 billion English-language documents in our index, to more broadly answer people's keywords and questions."

http://www.nytimes.com/2005/03/10/technology/circuits/10sear.html

shor
03-14-2005, 11:33 PM
You know what I'd like to see? Instant Vortals.

Imagine performing a search and being taken to a page (or two or three!) that are basically a vertical information portal on the subject. There would be the requisite links ranked on relevance and authority, of course, but also relevant facts, advertising, authority sites in the area, forum discussions, etc.

It's moving towrds this now, but I'd like to see a more comprehensive treatment and design.

Ian

Just a note about this - The Korean search engine market has been using the 'Instant Vortal' concept for many years now. Their search industry is almost alien compared to what Western (and even Japanese/Chinese) search users are accustomed to.

Using one of the big players as example, Naver.com (http://www.naver.com) uses SERPs which are gigantic extrapolations of traditional Western SERPs. We have our paid listings and our organic natural results right? Naver shows a bucketload more than that...well it's kinda hard to explain. Best way to illustrate would be do take you there - try a search query for 'Houston Rockets (http://search.naver.com/search.naver?where=nexearch&query=houston+rockets&frm=t1&x=0&y=0)'

Naver SERP:

1. Naver Wikipedia listing link (single) - essentially an internal Naver directory listing Wikipedia-style on the Houston Rockets, a succinct and concise summary
2. Naver Directory listing link (single) - Yahoo/ODP style directory listing, clicking takes you to the Naver directory/category for Rockets, with further subdirs. for players, related dirs to the the NBA etc.
3. Sponsored paid listing (single) - not sure how this works, think it's a partnered company to Naver
4. Q&A listings (multiple) - Google/Ask Experts type Q&A listing for the query, click on it and you get taken to Naver's Q&A section, a place where you can ask questions and get answers at the drop of a hat. Who are the Houston Rockets? What year did the Rockets last win a NBA title? etc..
5. Cafe listings (multiple) - Media listings on the query - video, audio, transcripts, music, etc
6. Book listings (multiple) - Amazon-style listings on the query
7. News listings (multiple) - Google/Yahoo style latest news listings on the query
8. Organic websearch listings (multiple) - hello, what have we here! About 40+ listings down the page, right at the bottom, come the natural results :)

Wow, now you can find all the related information on the query that you could ever want from a single page! This is a different type of 'relevance' that this vortal is giving its users. Its users are comfortably reassured that they will find what they want, no matter what it is, on this portal. Google websearch will give you the most relevant webpage, Naver will give you access to the most relevant information - Rockets' tickets? Rockets' memorabilia? Rockets' books? Rockets' news? Rockets' multimedia? Rockets whatever?

Also note the further possibilities on the tabs stuck at the top/bottom nav bar - dictionary search, image search, local search etc, there are at least 10 different tabs related to different search types. You can search deeper within in any of their database types if you want to get more specific.

The Korean search industry took the internet long jump while we Westerners are doing a triple-jump - hop, skip and jump.

I don't know the full history of the Korean search industry but I do have a few ideas on the differing search evolution:

The Koreans are an 'always-on' Internet society that is connected to the 'Net at all demographics.
Computer literacy is extremely high - little difference in the computer literacy from a 10 y.o and 80 y.o. users.
Broadband/Internet accessibility and penetration is near 100%.
Vast supply (and demand) of IT intellectual capital for a very niche audience. South Koreans are highly educated, specifically with regard to the IT sector and their only audience is their own population

So to get the topic more on-track :) I think we should also look beyond our borders when we mention search innovators, pioneers and trailblazers ;)

mcanerin
03-15-2005, 12:54 AM
I couldn't agree more, shor!

I've done a bunch of work for Samsung and LG in both Seoul and Suwon, and was very pleasantly surprised with South Korea. It doesn't get the recognition it deserves from the west, IMO.

As a matter of fact, it's my favorite Far East country - I'd go back in a second. Very modern, excellent levels of English usage, and reasonable prices, not to mention a very high level of technical sophistication.

That Naver.com is pretty much exactly the type of thing I was thinking of, and honestly I do think it's a good start on the future of search. I would probably organize things slightly differently on the page, but that's just me. The concept is pretty much exactly the type of thing I was thinking about.

Good find!

A genuinely useful information results page. Will wonders never cease? I guess I can't patent the idea now ;)

Ian

NFFC
03-15-2005, 09:10 PM
I think the second tier engines, maybe only ask jeeves left in that marketplace, have to decide if they want to lead or follow. I don't think they will die more just wither away, Google didn't get where they are today [reggie perrin reference in there for the brits] by accepting the status quo.

Yahoo is a whole different animal, not leaders.

MSN, I don't know, maybe.

I think the next big SE [there is always a next big one] will come from Asia. They have the right cost base, the desire, the people and in almost the blink of an eye the tech, the infrastructure is in the works too.

I think the current search players are a little too settled to be truely leaders, the new wave of inovation will have to come from elsewhere. Evolution rather than revolution is the key imho, I think the next leader will just simply use the screen space better than the current mob.

After all thats all Google, which must be the greatest sucess ever in search, did. Fast results, a good adult filter and a killer spell check is all it has taken to turn the heads of the www. Add a "twist"on top of that and we may have a new "leader", myself I like the vivisimo style cats as the next step but the current crop don't have the screen real estate to work that in.

I don't see in reality anybody taking the lead in the short term, its just too easy to run on Google's shoulder at the moment, they are the daddy.

FeldBum
03-16-2005, 01:57 PM
To my way of thinking, Google are the innovators, and Yahoo! and MSN followed. Even before they launched, they showed the rest how to do 3 things:-

(3) Make money from a search engine. The others had been scratching around with portals and all sorts in an effort to make it pay.

As soon as Google showed people #3, those who were able to copy the whole thing jumped into it - Yahoo! and MSN. And, for that, we are grateful because there is more than one basket again in which we can put our eggs.


Altavista was the first to introduce PPC, though they quickly took it down after an uproar in the market. In my opnion, AV has always been an innovator. It had 16 million pages indexed when it debuted, was the first to offer a translation service Babel Fish (Eurgh!) and I still use it now for multimedia searches.

Back to the money: Overture (then GoTo.com, soon Yahoo Search Solutions) is really the father of a profitable engine based on PPC. In that sense, every engine using PPC is really a copycat.

Brad
03-16-2005, 03:08 PM
I think the next big SE [there is always a next big one] will come from Asia. They have the right cost base, the desire, the people and in almost the blink of an eye the tech, the infrastructure is in the works too.

I kind of think so too. This is why I am keeping my eye on Lycos.com - the new owners are S. Korean and they have been refocusing the portal around search. They have been hinting that more features will be rolled out and that some major thing will be introduced in the second half of 2005. Frankly they have little reason to play it safe - nothing to loose.

rustybrick
03-16-2005, 07:09 PM
Answers.com does not answer questions, like Ask Jeeves. Type in, “when was Ben Franklin born? (http://www.answers.com/when%20was%20Ben%20Franklin%20born%3F)”. They give you no direct results (try it on Ask (http://web.ask.com/web?q=when+was+Ben+Franklin+born%3F&qsrc=0&o=0) and see the smart answer). They make you type in “Ben Franklin (http://www.answers.com/Ben%20Franklin)” and scroll through an encyclopedic result to find the answer. In short, they are an encyclopedia, not a search engine, and CERTAINLY not an answer engine. And even when it comes to being an encyclopedia, they are nothing new. Mostly they rely on Wikipedia. For definitions, Ask has been giving dictionary reponses in SmartAnswers since long before Answers.com. But to truly answer BILLIONS of questions or keyword queries, you need billions of documents and hundreds of databases, not 1 million encyclopedia pages. That's what Ask realized when it purchased and then built out Teoma. And that's the source of Ask Jeeve's innovation, like the NYTimes gives a taste of. Lastly, when it comes to traffic/growth for answers.com, it's only because of Google sending them traffic. Look what happened to Dictionary.com's traffic after Google dropped them.

Webvisitor
03-20-2005, 06:05 PM
Rustybrick you write,

"Lastly, when it comes to traffic/growth for answers.com, it's only because of Google sending them traffic. Look what happened to Dictionary.com's traffic after Google dropped them."

Bob Roshenshein of Gurunet says more than half of his traffic is from outside Google.
He announced an agreement with Accoona last week and claims to have many more deals in the "pipeline". They intend to begin SEO for organic placement internet wide.
I have the Answer bar pullout on my desktop as does most of my friends and family.
MSN is talking about a beefed up Encarta coming soon and Ask Jeeves promises an "Answers" type product very soon. I would guess this one page 'answers' thing is catching on. For my purposes it comes down to this. Search 'baseball' on Google or any SE and then search the term again on Answers. Unless you are doing research you need go no further than Answers.

shor
03-21-2005, 02:40 AM
I have the Answer bar pullout on my desktop as does most of my friends and family.
MSN is talking about a beefed up Encarta coming soon and Ask Jeeves promises an "Answers" type product very soon. I would guess this one page 'answers' thing is catching on.

Yes indeed. And why shouldn't it? If you look at the Naver.com example I posted above, you can see the power of a search 'vortal'.

Entering 'Baseball' on answers.com gave me a dictionary listing, an encyclopaedic listing, wordnet/american history listing, a wikipedia listing and related internal and external links. The listings intermingle and while exhaustive, barely scratch the surface of what a single page 'Answer' could provide.

Entering the query 'Baseball' on a true vortal would give you all of the above listings as well as news, richmedia, directory, Q+A Google Experts-style, sponsored, organic and local/regional listings + the ability to refine your search in any of these search categories.

As we all know Google have been the innovators (and copycats) of the SE world for years. If you look at what Google Labs have been churning out, you'll notice that they are well set up for a future vortal opportunity. Google Maps, Google Video, Google News, Google Suggest, Google Scholar, Google Groups, Froogle and Google Experts are just some of the search properties that could be integrated into a vortal.

A successful integration of these properties into a single vortal that provides relevant SERPs is going to be a dilemma but one that in the long term would be incredibly beneficial for end users.

I can envision a long SERP with multiple categorical listings, which can be rapidly refined using a GoogleX-type tab system. For example, after entering the keywords for an item you are interested in buying, you'd be able to quick-switch between the Wikipedia summary, official specs retrieved from the manufacturer's homepage, read the Google Group/Blogger reviews and then compare the Froogle listings, all by following 1-click links within the G vortal.

I guess the question is, why would G do it? Does the benefit outweight the costs involved? In the meantime I guess they'll just keep inventing new tools (and toys) to play with.

xan
03-21-2005, 07:32 AM
Hey guys,

Firstly, forgive me for such a long post. I read the thread and as I read, I realised I have stuff to say, and I quoted from all of your posts to write my answer, but soon realised I hadn't written your names down, so all my apologies!

"Jeeves, to me, is a lot like Google or Yahoo in the sense that it is a brand that has been able to stay around and continue to test new features that consumers might like, mainly because that brand awareness has kept people coming to the site."

Yes, I agree. brand is massivly important, afterall, it encourages user confidence in the search engine.

"On pure algorithmic search, Teoma may definitely have a few advantages, but these kinds of advances can be subsumed/copied by other brands."

Teoma has quite a few disadvantages, as it tries to use to use semantic search but doesn't go all the way. The technology looks very young to me, and needs to be further developped.

"Microsoft is saying "we can hire a bunch of people to copy what the others are doing." Short term they may be right, but perhaps Godin is right, that "very good" is in fact very bad. It's those small, nimble startups out there who may be coming up with the innovations that will stun the marketplace."

I don't agree that "very good is very bad" unless I interpret it in the wrong way. At the end of the day, if a search engine doesn't deliver, it can be as pretty as can be, but I wouldn't use it. I want to use a search engine because I'm looking for something. I consistently run into trouble, I'll get frustrated with it and go to an SE which I trust.
"Small nimble start-ups" worth their salt, in my opinion should accept offers to be brought out by the bigger guys. They have good ideas, but they don't usually have the best engineers (although that may be biased of me), and in my experience have 1 good method to to surf until the wave hits the beach.

Google have great insight, and they tend to make the right purchases. I think they were buying blog stuff before the others (but I might be wrong here). They also churn out betas boldly at quite an impressive speed. This engineering company has the knowledge, education, experience and people to do this. Microsoft clearly have brilliant engineers as well, but maybe don't use them to their full potential, for whatever reason they may have.

"The new feature, Direct Answers From Search, will search across the entire Web, rather than simply from its own database, to find answers to natural-language queries (that is, those phrased as questions rather than mere search terms)."

This is not by any means a new idea. It a very old one actually. Natural language search uses algorithms that rely on phonetics, morphology, syntax, semantics, discourse, pragmatic context, amongst other things. These are all used in natural language processing (speech generation for example). Here are some systems that have been made/used in the past:
MARGIE (Meaning Analysis, Response Generation, and Inference on English system),Stanford-1973
PAM (Plan Applier Mechanism),Wilensky-1978
HARPY (Speech understanding for document retrieval), CMU-1980
HWIM (Hear what I mean), BBN-1976
ELIZA ( ) MIT-1966
ARPA (Speech Understanding Research Program), DoD-1970's
"The systems to be designed were to accept normally spoken sentences in a constrained domain with a 1,000 word vocabulary, and were to respond in reasonable time with less than 10% error. (This was one of the few times that AI programs had any design objectives specified before development.) "

...there are many of them, and the reasearch goes back to the 60's. Improving on it is no easy task, and we are a long way off at the moment. Have a look at lovely ALICE, she won the loebner prize 3 times aferall, and there's still bit of work to do, although she doesn't work on grammatical, semantic, etc... text generation.
Bottom line, no one is quite sure how to proceed as yet anyway.

"This allows us to answer far more questions than would be possible using editors or structured databases," said Jim Lanzone ..."When you're diving into structured databases, you're limited in your coverage. We want to harvest the power of the 2.5 billion English-language documents in our index, to more broadly answer people's keywords and questions."
No S**t sherlock! If its DB driven, its no better than templating really.

AussieWebmaster - yes, all sensible stuff!

"Imagine performing a search and being taken to a page (or two or three!) that are basically a vertical information portal on the subject. There would be the requisite links ranked on relevance and authority, of course, but also relevant facts, advertising, authority sites in the area, forum discussions, etc.

It's moving towrds this now, but I'd like to see a more comprehensive treatment and design."

You're right there, but result visualization is really very very hard. Bringing all of this together is not easy either. I know that users expectations are always getting higher and higher, but none of us are not miracle makers!

"Academic arguments just dosn't work. The average user of the Internet do not care much about technology - they care about usefullness. It's not enough to be able to argue that your engine is better - you have to proof it to the users."

I have to disagree with you there - pants results and a nice delivery of results won't work. Kartoo had a lovely way of delivery results but it wasn't scalable. The average user wants relavant results, and that's why they use search really. This can only be achieved by technology. I mean you don't know what software makes an elevator work do you (luckily, its a bit scary), and your reason for using it is to get to the right floor. If it looks pretty and foesn't get you anywhere, well you wouldn't be happy. You don't prove anything to the user - you deliver.

"I have yet to see one single valid argument that what we have now is near the end of search evolution."

Good call, the internet and search are in their infancy.

" technological breakthroughs take place fairly frequently in the industry"

Not really, its improvements rather than breakthoughs.

"Jon Kleinberg's work was an influence on PageRank and Google."

HITS and PR were both produced at the same time (1998). There's always a race on in research, lots of people have the same idea due to the progress in the research area. HITS just wasn't as good.

"Kleinberg's work had an influence on most people in the web search research field."

It is always introduced to students in academia, but that's it really. There were many other influences, and that stuff isn't at all the biggest challenge in search, so his work was great, but that's it.

"Google promotes the fact that its server infrastructure is one of its major assets."

I know a lot of people who have been running this infrastucture but on much smaller systems, or medium sized ones. Distributed systems and parallel computing are necessary for any computing task requiring a lot of power.

"a gazillion Alta Vista patents that they now own"

I think they tried to patent meta-tags as well, they patent everything! But the biggest patent people are IBM, if I'm not wrong and my info is out of date.

"Google won the hearts and minds of the propeller heads from the very beginning. And I think they still have that techno audience. However, in our part of the industry there's certainly some disillusionment with them over the whole PageRank myth".

I don't think they ever had anything much to do with the PageRank myth, it was pushed by SEO work I think, not that there's anything wrong with that. When you build something, you don't tell the world "this is how it works guys!".

"(MSN) Insofar as the technology goes, it's not as if they were short on this kind of research about information retrieval on the web. They've had people researching in the field for years! They just never moved into it commercially."

Yes, they have and information retrieval research has been going on in academic circles, research circles, labs,... since a very very long time, even before MSN, and it first applied to digital libraries, and the topic of digital libraries is still very important in IR, actually internet IR is just a bit of the IR reasearch arena:

"In 1876 Dewey published (anonymously) a work titled: A classification and subject index for cataloguing and arranging the books and pamphlets of a library that was to have far reaching effects. Although his classification scheme still is in wide use all over the world, the real value of Dewey's work was the idea of:

* introducing relative as opposed to absolute location.
* the assignment of (decimal) numbers to books rather than shelves, thereby making the specification of detailed subjects feasible,
* the provision of a relative index on subjects. "

A history of information retrieval (http://pi0959.kub.nl/Paai/Onderw/V-I/Content/history.html)

There is a lot going on in IR, as there always has been. The research labs and the post-doc, doc research are always a great hunting ground for what's going on and who to hire. Quite a few conferences are a bit like shopping for SE's, DL's, the medical and pharmaceutical industry and so on. I think many people always focus on SE's and forget or take for granted all of the other IR application and A.I applications they use day to day. There's an A.I module in your washing machine! IR systems enjoy massive interest in visul document IR (pictures basically), check out Computer and Vision Research Center (CVRC). IR is used in Telecommunications, ...

Piquant by IBM is quite intersting:

PIQUANT by IBM research labs (http://www.research.ibm.com/compsci/spotlight/nlp/)

Artyom
03-23-2005, 05:24 AM
Recently I've read about there are 137,000 home robots at this world already. My vision of a true innovation is combining SE with robot. Let's say we have wireless connection for Asimo (Honda robot) which may reply to your request 'here is the asnwer on your question' (pulling results from Google SERPs.) I wouldn't mind some funny comments to hear from him as well ;)

rustybrick
05-20-2005, 06:03 PM
Back on the topic of Playing Copy Cat...

I found this blog entry by Jeremy Zawodny from Yahoo! to be funny.

http://jeremy.zawodny.com/blog/archives/004684.html

See my comment at http://jeremy.zawodny.com/blog/archives/004684.html#comment-17463