Microsoft Answer Engine Research (Ask MSR/Answerbot)

[garyp]

From factoids to facts
The Economist
http://www.economist.com/printedition/displayStory.cfm?Story_ID=3127462

The Economist takes a look at the work of Eric Brill (http://research.microsoft.com/users/brill/) and his Ask MSR research at Microsoft. Very interesting and dare I say, exciting, research.

From the article,
Ask MSR" research at Microsoft from the current issue of The Economist. From the article, "What is the next stage in the evolution of internet search engines? AltaVista demonstrated that indexing the entire world wide web was feasible. Google's success stems from its uncanny ability to sort useful web pages from dross. But the real prize will surely go to whoever can use the web to deliver a straight answer to a straight question. And Eric Brill, a researcher at Microsoft, intends that his firm will be the first to do that."

Ask MSR is still a prototype, although Microsoft is trying to improve it and it may be launched commercially under the name AnswerBot.


Here are some related links that might be of interest:

+ Automatic Question Answering: Beyond the Factoid
http://www.isi.edu/~radu/Papers/HLT_NAACL04/camera-ready-QA.pdf
A 2004 paper by Eric Brill and Radu Soricut. This paper is mentioned in the article.
+ "Web Question Answering: Is More Always Better?"
A 2002 paper by Microsoft researchers.
http://www.ai.mit.edu/people/jimmylin/papers/Dumais02.pdf
+ "An Analysis of the AskMSR Question-Answering System"
Another 2002 paper by Microsoft researchers.
http://research.microsoft.com/~sdumais/EMNLP_Final.pdf

+ LearnitAll
Answer engine/knowledge extraction research at the University of Washington.
http://www.newscientist.com/news/news.jsp?id=ns99994961
http://www.cs.washington.edu/research/knowitall/

[cariboo]

A very interesting subject, garyp...

This article was published some time ago now, and AskMSR is in the air for more than a year now...

Brill's team made a demonstration of AskMSR almost a year ago, in front of journalists that came to learn about future features of Longhorn. Most of them were amazed, but it's difficult to know, from this demonstration, how smart the system really is...

We don't know much about the real performances of Ask/MSR... And we can't be sure it isn't just a lab curiosity ! I mean I doubt Microsoft can use it as a commercial software at this moment.

But AskMSR is actually a good step towards a new kind of search engine...

Microsoft Research works hard in this area :

I've found out that several Microsoft labs are working on this subjects, or very closely related systems. For instance, the chinese lab studied a new search engine based on FAQs, called Brilliant. It seems this is a major research theme for them...

[garyp]

Cariboo (and others):

You might be interested in compilation of recent Microsoft search "writings" and patents that I've posted on my ResourceShelfPLUS site. Several of the papers were written by the MS team in China.

http://www.resourceshelf.com/archives/2004_07_25_resourceshelfextra_archive.html/#109114760046163042

[orion]

In reference to Table 3 in

http://research.microsoft.com/~sdumais/EMNLP_Final.pdf they write and quote

"A correlation coefficient
of 1 indicates strong positive association,
whereas a correlation of –1 indicates strong negative
association. We see that there is indeed a correlation
between the scores output by the system
and the answer accuracy, with the correlation being
tighter when just considering the score of the first
answer.

Correlation Coefficient

Score #1 .363
Score #1 – Score #2 .270"

End of the quote.

These correlation coefficient values indeed indicate a weak correlation between the scores output and the answer accuracy. In this paper, they haven't show a test for linearity, either.

Susan Dumais is a well recognized authority on this type of research. I'm sure the group will improve this model.

Orion