Search Engine Watch
SEO News

Go Back   Search Engine Watch Forums > Search Engines & Directories > MSN Search
FAQ Members List Calendar Forum Search Today's Posts Mark Forums Read

Reply
 
Thread Tools
Old 06-13-2004   #1
bethabernathy
Looking forward to Summer
 
Join Date: Jun 2004
Location: Lake Tahoe, California
Posts: 419
bethabernathy has a spectacular aura aboutbethabernathy has a spectacular aura aboutbethabernathy has a spectacular aura about
MSN Powered by Inktomi?

Hi There: I have been reading articles and posts in the forums and I am confused, is MSN powered by Inktomi or Yahoo! search? I realize that Yahoo purchased Inktomi, so there may some semantic issue related to the way people are stating MSN is powered by Yahoo, but I did read on one thread in this forum where Danny Sullivan stated:

"shades of what used to happen back in the old days when we still had an Inktomi and it powered other people."

Posted at:

http://forums.searchenginewatch.com/...hp?threadid=62

I several clients who paid for Inclusion with Inktomi and if it is no longer then their contracts have been terminated before their terms end. Thanks for any information on this. -Beth
bethabernathy is offline   Reply With Quote
Old 06-13-2004   #2
David Wallace
 
David Wallace's Avatar
 
Join Date: Jun 2004
Location: Phoenix, Arizona
Posts: 887
David Wallace is a splendid one to beholdDavid Wallace is a splendid one to beholdDavid Wallace is a splendid one to beholdDavid Wallace is a splendid one to beholdDavid Wallace is a splendid one to beholdDavid Wallace is a splendid one to beholdDavid Wallace is a splendid one to behold
The last time I checked, MSN was still being powered by the Inktomi index. Of course, I wonder if the Inktomi index is really a version of the Yahoo index, just not as up to date.
David Wallace is offline   Reply With Quote
Old 06-13-2004   #3
bethabernathy
Looking forward to Summer
 
Join Date: Jun 2004
Location: Lake Tahoe, California
Posts: 419
bethabernathy has a spectacular aura aboutbethabernathy has a spectacular aura aboutbethabernathy has a spectacular aura about
Hotbot

I am not sure if Inktomi is gone, but at Hotbot its' logo is still listing at the bottom of the search results.
bethabernathy is offline   Reply With Quote
Old 06-13-2004   #4
Dodger
Honorary Member
 
Dodger's Avatar
 
Join Date: Jun 2004
Location: Central US
Posts: 349
Dodger has a spectacular aura aboutDodger has a spectacular aura aboutDodger has a spectacular aura about
All I know is that to some extent Inktomi still exists. The Inktomi spider is definitely still crawling, but it is also crawling non-Inktomi inclusions that were never submitted. Why that is, I cannot say. But if this is actually the Slurp spider, then Yahoo should at least rename it.
__________________
I am Ronnie
Dodger is offline   Reply With Quote
Old 06-13-2004   #5
bethabernathy
Looking forward to Summer
 
Join Date: Jun 2004
Location: Lake Tahoe, California
Posts: 419
bethabernathy has a spectacular aura aboutbethabernathy has a spectacular aura aboutbethabernathy has a spectacular aura about
Shouldn't MSN disclose who provides their results?
bethabernathy is offline   Reply With Quote
Old 06-13-2004   #6
Dodger
Honorary Member
 
Dodger's Avatar
 
Join Date: Jun 2004
Location: Central US
Posts: 349
Dodger has a spectacular aura aboutDodger has a spectacular aura aboutDodger has a spectacular aura about
Quote:
Originally Posted by bethabernathy
Shouldn't MSN disclose who provides their results?
Disclose to who? Us or the searchers?

You can go to Bruce Clay's Website and see who is supplying who with his Search Engine Relationship Chart.

I do not think it makes a difference to the searchers, as long as the Overture results are clearly marked at MSN as being paid results.
__________________
I am Ronnie
Dodger is offline   Reply With Quote
Old 06-13-2004   #7
MakeMeTop
Complete SEO Dunce!
 
Join Date: Jun 2004
Location: Northern Ireland
Posts: 28
MakeMeTop will become famous soon enough
>I several clients who paid for Inclusion with Inktomi and if it is no longer then their contracts have been terminated before their terms end.

Inktomi (as an index supplier) still remains - and will do until people who have paid for Inktomi PFI have had their contracts expire. These "Inktomi" results will still show in the results where the previous Inktomi data was used. Currently this includes Hotbot and MSN - though either may change. Confusingly, it does not include Yahoo. However, Yahoo Search! results also show in previous Inktomi partners.
MakeMeTop is offline   Reply With Quote
Old 06-14-2004   #8
dannysullivan
Editor, SearchEngineLand.com (Info, Great Columns & Daily Recap Of Search News!)
 
Join Date: May 2004
Location: Search Engine Land
Posts: 2,085
dannysullivan has much to be proud ofdannysullivan has much to be proud ofdannysullivan has much to be proud ofdannysullivan has much to be proud ofdannysullivan has much to be proud ofdannysullivan has much to be proud ofdannysullivan has much to be proud ofdannysullivan has much to be proud of
The official word from Yahoo is that there is no separate Inktomi index. There's only the Yahoo Search Technology index, and that's used by Yahoo and partners that Yahoo support directly through Yahoo, though the Yahoo-owned Overture subsidiary or the Yahoo-owned Inktomi operation.

That other thread talked about how despite their being one index, results at Yahoo may be much different than results at Yahoo's partners. This, according to Yahoo, is due to those partners perhaps using different ranking criteria for their indexes.

In reality, it looks like Yahoo is pushing out one version of a ranking algorithm combined with hitting only a subset of its total index to partners, while the full index and a different ranking algorithm is used by Yahoo itself.

You could call that smaller index "Inktomi," but I think that it's better to think of it as Yahoo and understand that Yahoo results may not always be the same as at Yahoo. That's important, because it suggests that you need to be in both Yahoo and Inktomi. There's no way to submit to just "Inktomi" any longer that I know of, not even through paid inclusion, because Inktomi no longer is supposed to exist.

It is odd there's still a spider out there calling itself Inktomi. I'd guess an oversight by Yahoo's part. As for why you see the Inktomi logo in places, there's still an "Inktomi" network being supported by various contractual obligations. And as Makemetop says correctly, those with paid inclusion contracts to be in "Inktomi" that still have time to run get put into these places and only these places if they've not bought Yahoo/Overture paid inclusion separately.

Life would have been much easier if Yahoo has simply said that anyone who'd bought Inktomi previously would have been grandfathered into the new Yahoo paid inclusion program.

FYI, guide to who powers whom from SEW is here, recently updated.
dannysullivan is offline   Reply With Quote
Old 06-14-2004   #9
seobook
I'm blogging this
 
Join Date: Jun 2004
Location: we are Penn State!
Posts: 1,943
seobook is a name known to allseobook is a name known to allseobook is a name known to allseobook is a name known to allseobook is a name known to allseobook is a name known to all
I wonder why Lycos Insite is still selling Inktomi inclusion?
http://insite.lycos.com/inclusion/se...ginesubmit.asp
__________________
The SEO Book
seobook is offline   Reply With Quote
Old 06-14-2004   #10
dannysullivan
Editor, SearchEngineLand.com (Info, Great Columns & Daily Recap Of Search News!)
 
Join Date: May 2004
Location: Search Engine Land
Posts: 2,085
dannysullivan has much to be proud ofdannysullivan has much to be proud ofdannysullivan has much to be proud ofdannysullivan has much to be proud ofdannysullivan has much to be proud ofdannysullivan has much to be proud ofdannysullivan has much to be proud ofdannysullivan has much to be proud of
Quote:
I wonder why Lycos Insite is still selling Inktomi inclusion?
I'll see if I can get Lycos to comment -- my guesses:

1) They simply haven't changed the page (happens more than you'd realize)

2) They don't have a contract to sell inclusion on Yahoo, so selling Inktomi means what Yahoo euphemistically calls "the historic Inktomi network."

In particular, notice there's no per charge fee included with Inktomi -- another reason why this probably doesn't get you into Yahoo itself.
dannysullivan is offline   Reply With Quote
Old 06-14-2004   #11
Dodger
Honorary Member
 
Dodger's Avatar
 
Join Date: Jun 2004
Location: Central US
Posts: 349
Dodger has a spectacular aura aboutDodger has a spectacular aura aboutDodger has a spectacular aura about
Quote:
Originally Posted by dannysullivan
It is odd there's still a spider out there calling itself Inktomi. I'd guess an oversight by Yahoo's part. As for why you see the Inktomi logo in places, there's still an "Inktomi" network being supported by various contractual obligations.
The WhoIs entries on those spiders have had the contact information changed to reply to Slurp-at-yahoo.com though. The answer section still reflects the Inktomi name. That is what is so confusing about it.

Quote:
Originally Posted by dannysullivan
I'll see if I can get Lycos to comment -- my guesses:

2) They don't have a contract to sell inclusion on Yahoo, so selling Inktomi means what Yahoo euphemistically calls "the historic Inktomi network."

In particular, notice there's no per charge fee included with Inktomi -- another reason why this probably doesn't get you into Yahoo itself.
That is probably more like it, it is paid inclusion into the Lycos search results would be my guess.

And you gotta love that "historic Inktomi" line. The one I liked is when the Fast Enterprise crawler was going gonzo about a month and a half ago -- Fast was not under contract to crawl the pages, nor was Yahoo owning up to it being part of AllTheWeb's part of Fast. Tim Mayer used that same message when asked at WebmasterWorld. Nobody was owning up to this little bugger that I could see.

Quote:
Originally Posted by USCity.Net
USCITY.NET logged over 22 gigabytes of data pulled by AlltheWeb's crawlers attempting to harvest USCITY.NET's member links. AlltheWeb's dysfunctional crawlers also generated over 12,000 errors on USCITY.NET servers scavenging for pages that don't exist. Repeated attempts to contact AlltheWeb and Overture about its questionable crawling yielded no response.
This all happened during the transition period when they were combining all of the Search products. My feeling is that something went unmonitored or they were unaware of the spider getting loose ... hehehe. But there has been no open acknowledgement of that.
__________________
I am Ronnie
Dodger is offline   Reply With Quote
Old 06-14-2004   #12
bethabernathy
Looking forward to Summer
 
Join Date: Jun 2004
Location: Lake Tahoe, California
Posts: 419
bethabernathy has a spectacular aura aboutbethabernathy has a spectacular aura aboutbethabernathy has a spectacular aura about
I ran the log files for a client and the spider is still named Inktomi:

Domain Name Hits Users Time of Last Access
inktomisearch.com 2 1 April, 06 2004 10:20:18 AM

Although that was in April. I'll check some newer logs.

The thing that was awful about the Yahoo paid inclusion was that I was lead to believe, some way or another, that the pages I included prior to the switch the pages would be grandfathered into the new Yahoo! index as well as into Alltheweb & MSN. I got the news just after the Google Florida and my clients were desparate to get some referrals, short of adwords, so we paid and got all their pages included. Then I was told about a month later that the pages would not be included in the Yahoo! index just MSN & Hotbot. Luckily I have very nice clients, but I went and created site maps for most of them and submitted the sitemaps to Yahoo!s new search engine and pleasantly to my surprise, most all the pages and more have been included in the Yahoo! index. It was a pretty frightening time. I think I have been complaining about this for a bit too long. -Beth
bethabernathy is offline   Reply With Quote
Old 06-14-2004   #13
bethabernathy
Looking forward to Summer
 
Join Date: Jun 2004
Location: Lake Tahoe, California
Posts: 419
bethabernathy has a spectacular aura aboutbethabernathy has a spectacular aura aboutbethabernathy has a spectacular aura about
Talking

I checked some log files for June 2004 and it still is calling the spider Inktomi search. This is the record:

Domain Name | Hits | Users | Bandwidth
inktomisearch.com | 90 | 46 | 5.93 megabytes
bethabernathy is offline   Reply With Quote
Old 06-15-2004   #14
TheotherTim
Official Y! Rep
 
Join Date: Jun 2004
Location: Palo Alto, CA
Posts: 27
TheotherTim will become famous soon enoughTheotherTim will become famous soon enough
Slurp and FASTCrawler

I think you will find that the user agent of the spider has changed to Yahoo Slurp but the domain has remained Inktomisearch. This was not overlooked and has been on our list of things to migrate over the coming months.
The former FAST Crawler is still out there crawling in a limited capacity. It has not been crawling for web results for any of our production sites for several months. I think you will notice the user agent says "former web crawler" which identifies that fact. This is a fact that the uscity.net people failed to notice or realize before issuing their "press release".
TheotherTim is offline   Reply With Quote
Old 06-15-2004   #15
seobook
I'm blogging this
 
Join Date: Jun 2004
Location: we are Penn State!
Posts: 1,943
seobook is a name known to allseobook is a name known to allseobook is a name known to allseobook is a name known to allseobook is a name known to allseobook is a name known to all
Hi Tim
could you tell us what the Lycos Inktomi purchase gets us
__________________
The SEO Book
seobook is offline   Reply With Quote
Old 06-15-2004   #16
TheotherTim
Official Y! Rep
 
Join Date: Jun 2004
Location: Palo Alto, CA
Posts: 27
TheotherTim will become famous soon enoughTheotherTim will become famous soon enough
The Lycos Inktomi purchase is a way to buy distribution into the legacy Inktomi network. This excludes Yahoo. You may want to ask Lycos directly for more info.
TheotherTim is offline   Reply With Quote
Old 06-15-2004   #17
Dodger
Honorary Member
 
Dodger's Avatar
 
Join Date: Jun 2004
Location: Central US
Posts: 349
Dodger has a spectacular aura aboutDodger has a spectacular aura aboutDodger has a spectacular aura about
Thanks Tim. I kind of knew the Inktomi domain was Slurp -- today anyway.

Back during the USCity event their spider was probably the same one that I was seeing as well. This was one of them:
User-Agent: Yahoo-Verticalcrawler-FormerWebcrawler/3.9 crawler at trd dot overture dot com; http://www.alltheweb.com/help/webmaster/crawler
IP: 68.237.225.195 (pool-68-237-225-195.ny325.east.verizon.net)
This was brought up at WebmasterWorld at the time in which you stated:

"This crawler has nothing to do with atW anymore other than it was the crawling technology that formerly discovered URLs for the FAST/atW Engine. This crawler could be crawling for a variety of search products or projects at Yahoo."

I tried contacting FAST and got no response from them. But while I was there, it was interesting to note (at that time) that they were still fulfilling outstanding obligatory contracts per their homepage. That could mean just about anything, I know -- but it made things even more confusing.

It was also at that time that I noticed that they dropped AllTheWeb as one of their Express partners (as they call them), but added Overture to the list. This confused me even more. (I have not gone back to see if they are still on the list).

As you can see there was a spider out there doing work for somebody or someone...and it was really going gonzo too. Two days I methodically watched this thing start at the top and work it's way down. It was actually pretty good at weaving thru a forum's links. I was actually pretty impressed. But then it started back at the top again, and that is when we pulled the plug.

It was following the FAST rules of engagement, i.e. one page request per minute. It was definitely a FAST spider, but it was also a spider for hire too.
__________________
I am Ronnie

Last edited by Dodger : 06-15-2004 at 02:26 AM.
Dodger is offline   Reply With Quote
Old 06-16-2004   #18
trellian
Trellian
 
Join Date: Jun 2004
Location: Melbourne, Australia
Posts: 37
trellian will become famous soon enoughtrellian will become famous soon enough
Quote:
Originally Posted by MakeMeTop
>I several clients who paid for Inclusion with Inktomi and if it is no longer then their contracts have been terminated before their terms end.
Inktomi (as an index supplier) still remains - and will do until people who have paid for Inktomi PFI have had their contracts expire. These "Inktomi" results will still show in the results where the previous Inktomi data was used. Currently this includes Hotbot and MSN - though either may change. Confusingly, it does not include Yahoo. However, Yahoo Search! results also show in previous Inktomi partners.
Hello Barry, as you mayd be aware, the terms of Inktomi services were never tied in specifically to any search engine. If MSN drops Inktomi/Yahoo today, the Inktomi service that you have paid for will still be honored by Inktomi and the remaining portal partners. Inktomi have always made sure that the service offered, was for Inktomi inclusion, not MSN or any other specific portal inclusion program, hence as long as one portal displays Inktomi results for the full term, the service terms will be fulfilled.

Cheers
David
trellian is offline   Reply With Quote
Old 07-11-2004   #19
Winklerb
 
Posts: n/a
MSN search engine makeover

One of my client's was completely dropped from MSN's search engine (after 4 years of reasonably steady traffic growth), apparently with the new "googleesque" makeover.
Is there any recourse other than the pricey Overture listing service to reestablish the MSN hits?
  Reply With Quote
Old 07-11-2004   #20
trellian
Trellian
 
Join Date: Jun 2004
Location: Melbourne, Australia
Posts: 37
trellian will become famous soon enoughtrellian will become famous soon enough
After speaking with someone at MSN afew days back, the news is: "we are considering all our options". As soon as we know what PFI program (if any) they choose to go with I am sure it will be one that looks at relevancy above all other factors.

Cheers
David
trellian is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off