USA vs. Canada - Different Results on MSN

[eitemiller]

If you were to do a search for Prescott, Arizona in the Canadian version of MSN, our site has the #1 spot out of 128109 results ). However, if you do the same search in the American version, we are not in the top 250 spots out of 106466.

Along the same lines, if you do a search for the Grand Canyon, our site comes in at #22 out of 679868 . I won't even bother trying that on the American version, because it would take me way too long to find it.

SO I pose this question...

Are there two (or more) seperate bots for each country? If that is the case, can I assume that the Canadian side has indexed our site quicker (we are a very new site - online for three months) and that when the American bot comes to visit, we will rank just as well, or fairly close? Or am I just completely way off base here?

Any help on this interesting find would be greatly appreciated. Thanks again!

[eitemiller]

Has to be two seperate bots - it is the only thing that makes any sense. But why so high in one, and not the other? Any ideas here?

[dannysullivan]

For a search on prescott, arizona, I get 128,181 matches in the US version and 128106 in the Canadian one. For grand canyon, it's 667728 versus 663846.

Those counts are close enough to make me think that the same core database is being tapped. I think the differences are probably due instead because of ranking criteria.

For the grand canyon search, there definitely looks like a skew toward favoring Canadian sites. That's an adjustment that's easy for MSN to make.

This is less easy to see for the prescott, arizona search. A bunch of .ca sites don't as readily leap out. But I did a traceroute back on the top listed eatstayplay.com site, which I assume is your site. I can see it is based with a hosting firm in Edmonton, Canada. So the IP is probably telling Yahoo (which crawls for MSN) that it is Canadian-based. That's likely giving you a boost.

If your target audience is Canadians, I'd leave things along. If you're after both people in the US and Canada, moving to a US hosting site might help a little (and likely may cause you to drop in the Canadian results).

Further complicating everything is the fact that likely by the end of the year, Yahoo will completely be replaced by MSN's own crawler. So anything you do for MSN currently will likely change. On the flipside, the changes may help you with Yahoo.com itself. You didn't mention that, but I don't see you ranked over there.

[NFFC]

If you play about with the region filters here http://www.hotbot.co.uk/?command=adv all will be clear.

[eitemiller]

Wow - that's some amazing stuff. Yes, eatstayplay is our site. Unfortunatly, we are currently only targeting the US market. As far as Y, using those keywords I suggested earlier, we do not show up in the top 500. The hits we get from Y are for much smaller keywords, such as the campground "Katherine's Landing". Same goes with G.

Now I am wondering this - if MSN can favor sites "based" out of Canada - think Y and/or G would give us a slightly negative view since we appear to be based out of Canada? We are located in sunny Arizona, and now I am wondering if we should go ahead and make the switch to a local host.

Thanks for you help, NFFC - this one had me stumped all day yesterday. Take it easy.

[dannysullivan]

I'm in the UK, and I often see Yahoo results that appear different than those in the US -- even if I use Yahoo.com. Part of that is due to paid inclusion -- you can target countries that way.

I've seen this occasionally with Google, too -- and they've said this is something that wouldn't surprise them. That's Google-speak for "Yes, we may be customizing results by country even if you don't ask, but we don't want to say so."

Related thread: Should Search Engines adopt different results outside the U.S.?. First post also links to two other threads on the topic.

My advice. If you're targeting the US, then be as much like someone in the US as you can. That means US English, a US .com and hosted in the US. None of that's a guarantee you're going to rank tops -- but it will contribute to any success.

[eitemiller]

Thanks for the help, dannysullivan, and showing me the other threads related to this topic. Guess I better get a US web hosting service - I just didn't see this one coming. There sure is a lot to learn about SEO. Thanks to both of you two, take care.

On a side note, our site got Joe-jobbed last night. Never heard of that one either until now. I love the internet business

[mcanerin]

Search Engine Geolocation is an area of special interest to me.

Fact: MSN.ca gives an automatic ranking *boost* to Canadian sites - you can't even turn it off.

Fact: MSN currently uses TLD+linking analyisis to determine the location of a website (NOT IP like Google). So does Yahoo and Teoma/Ask Jeeves. Mainly because it's still using Yahoo (Ink) technology.

Assumption: The last time I talked to Mike Sharma (Manager of Network Programming, MSN.ca) at SES Toronto, I was a fellow speaker and had some every specific questions regarding geolocation. He indicated to me at that time that MSN would almost certainly keep the TLD+Linking method when they switched to their own crawler. Although they can change their mind anytime they want, I'm assuming they won't.

Recommended proceedure: The TLD helps with all crawlers - a .ca gets a boost on MSN.ca, and I've heard rumblings this will eventually apply to many of the other crawlers, as well (but I have no proof, and when I do find it, it sometimes changes).

Worst case - it won't hurt. The next thing to do is get LOTS of links from known Canadian directories - this gets you seen as Canadian, and then you get the boost.

But switching hosts (ie changing your IP) won't help you a bit for geolocation anywhere but Google. Only Google looks at IP.

Yahoo's experience with asia, specifically, has led them to believe that IP geolocation is a bad idea, and they are not likely to change their mind, afaik.

The perfect scenario is a .ca, on a Canadian IP, with lots of Canadian links. Common sense (and reality) says that will be a very good indication for geolocation across the board.

Yours,

Ian

[NFFC]

>Fact: MSN.ca gives an automatic ranking *boost* to Canadian sites - you can't even turn it off.

Fiction: See http://search.sympatico.msn.ca/advanced.aspx

>Fact: MSN currently uses TLD+linking analyisis to determine the location of a website (NOT IP like Google)

Fiction: TLD yes, linking analyisis is just plain bogus. We are talking about a US company, targetting US keywords with a .com domain. Are you saying that they get flagged as Canadian because of links and not just the fact they host in canada?

>Assumption: The last time I talked to Mike Sharma

I've stopped talkng calls from Mike.

>But switching hosts (ie changing your IP) won't help you a bit for geolocation anywhere but Google. Only Google looks at IP.

False [you don't offer Canadian based hosting by any chance do you?]

>The perfect scenario is a .ca, on a Canadian IP, with lots of Canadian links. Common sense (and reality) says that will be a very good indication for geolocation across the board.

Its a US company, targetting US searchers.

Yours,

NFFC

[eitemiller]

It definatly is not the end of the world if Canadian users type in "Grand Canyon" and we have the #22 spot. Shocking, yes, but not bad. What I need to know is - if MSNCanada is able to favor us because our web hosting service is from Canada (which I am certain that is the case), does that mean MSNAmerica is penalizing us because we are hosted in Canada? We are a US based company with a .com name and targeting US users (be nice to expand into Canada - but first things first).

Should I change hosting services? How much of this will matter anyway come the end of the year with the new Yahoo/MSN changes that are scheduled?

Thanks for all the discussions - much appreciated.

[eitemiller]

If moving web hosting to a USA based company means that I can come up #22 for "Grand Canyon" - then I guess we are moving. But the Grand Canyon page itself, IMHO, doesn't look like it would/should. Give me your opinions - I appreciate them.

[NFFC]

>Give me your opinions - I appreciate them

I think you should take the information already given and do your own work from here.

You the webmaster, you make the call.

[mcanerin]

NFFC, you have some good posts around here but you obviously didn't do you homework on this one.

>Fiction: See http://search.sympatico.msn.ca/advanced.aspx

WRONG. This will allow you to restrict the searches even more, but there is no way to get out of the default group of listing. Try it.

>Fiction: TLD yes, linking analyisis is just plain bogus.

WRONG. Give me some credit for doing my homework, ok? I located a test set and actually did research, instead of making unwarranted assumptions. Then I followed up by actually asking reps of the SE's. I know they don't always give up the secret sauce (or even know the answer) but there is no reason for them to mislead on this question - it's not a "how do I spam more effectively" type of question, which are the ones they normally aren't very helpful on.

TEST

Grouping: a search on "canadian website designer"
Results tested: www.apluswork.com and www.alouwebdesign.ca
Selection Criteria: both Canadian companies, both show up in top 10 on all major search engines, both are HOSTED IN THE US and have US IP Addresses. One has a .ca TLD and one has a .com TLD.

As Reference, I did this test in May 2004 (for my presentation), and these were the results:

Note:Google.ca search done with the "Canadian Sites Only" choice on.

Search Engine          www.apluswork.com           www.alouwebdesign.ca

Google.com                         1                                 2
Google.ca                         GONE                               1
MSN.com                            1                                 2
sympatico.MSN.ca                   1                                 2
search.yahoo.com                   3                                 2
ca.search.yahoo.com                3                                 2

Which clearly showed that IP had no effect on any SE but Google.

I just did this test again (Aug 2004) and the results are a little different:

Search Engine          www.apluswork.com        www.alouwebdesign.ca

Google.com                       1                                   3
Google.ca                      GONE                                 2,3
MSN.com                          3                                   7
sympatico.MSN.ca               GONE                                 2,7
search.yahoo.com                 4                                   6
ca.search.yahoo.com              9                                   2
Yahoo (Canadian sites only)    GONE                                  2

This is new behaviour, so I checked to see why, and it turns out that their position in ODP has moved from the Canadian section to the General area. Which of course supports link analysis. Many of their backlinks are DMOZ clones, if you check. FYI, www.apluswork.com is NOT in Yahoo Directory that I could find.

In short, even thought the tests were different, both support link analysis, especially in view of the catagory move and resulting assumption that they are not Canadian.

>False [you don't offer Canadian based hosting by any chance do you?]

WRONG. We've already established the IP thing, and athough I have a few sites hosted on my server, I have enough grey hair as it is to go into the hosting business, thanks

>Its a US company, targetting US searchers.

TRUE. I wasn't paying attention to that part in view of the other (wrong) info that was being bandied about. Naturally they are unlikely to care about their Canadian status.

But I will point out that being a Canadian, UK, AU (or any other non-US) site gives you an advantage in local search engines and absolutely NO disadvantage that I can see in the US (.com) ones, so in effect it's a free boost to rankings, at least in some areas. Personally, as an SEO I consider that to be a good thing.

>I think you should take the information already given and do your own work from here.

Agreed.

Yours,

Ian

PS wow, it's really hard to make get an easy to use chart in these posts...

[eitemiller]

Originally Posted by mcanerin

But I will point out that being a Canadian, UK, AU (or any other non-US) site gives you an advantage in local search engines and absolutely NO disadvantage that I can see in the US (.com) ones, so in effect it's a free boost to rankings, at least in some areas. Personally, as an SEO I consider that to be a good thing.

That's what I was looking for - I can effectivly make my decision, now that I have the facts that I was looking for.

Thanks, mcanerin - looks like you did your homework here

[NFFC]

>you have some good posts around here

I have a lot of bad ones too

>WRONG. This will allow you to restrict the searches even more,

I think we are splitting hairs a bit with that, I'll concede.

>TEST

Honest, SES speaker or not, if you are doing a test use more than two sites.

>I have enough grey hair as it is to go into the hosting business, thanks

Accepted and withdrawn. Strangly warming to you though.

>But I will point out that being a Canadian, UK, AU (or any other non-US) site gives you an advantage in local search engines and absolutely NO disadvantage that I can see in the US (.com) ones

Thats false too, but for another day.


>Agreed.

See, thats all I ask?

Yours,

NFFC

PS Nice posts, raised the level, thanks.

[mcanerin]

<grin>

Cheers NFFC! I like a good debate, too

FWIW, I did a lot more tests, but these two were so clear that I used them in the slideshow back in May, and I no longer have the original tests to show the trend, but I agree you can't come to a conclusion with such a small sample by itself.

BTW, Here is a fun thing: go to search.Yahoo.com

and look for prescott, arizona - site is gone, basically.

THEN go to ca.search.yahoo.com and check off the "Canadian Sites Only" and the site shows up as #3

Interesting, though obviously expected.

I went looking through the backlinks that MSN knows about on the site in question and found a bunch of FFA and Guest Book entries, and some of the guestbook entries (with reasonable rankings themselves) were Canadian according to Yahoo - I imagine that's what set it off, in eitemiller's case.

I still don't know the number of links needed to set it off - ie what if I had 10 canadian directories and 10 german ones? What would the site be? I'm still looking into this, though I expect I won't really start looking until MSN actually comes out with it's own engine - I'd hate to have to do the tests all over again.

And eitemiller - good luck, glad to have been helpful.

Yours,

Ian

[AussieWebmaster]

Have to admit that I have been to busy with other things to do any empirical testing,but we do have the .ca, .au, .jp, .uk etc. of most of our sites and they do get better placement in most search engines even though they have less inbound links than our US .com site on the various local engines.

The power of the location of the links seems to be geographically considered in Asia at least by the region and maybe not the country - especially on the asian engines like sina, baidu and sohu.

[Marcia]

I'm bumping this because someone was asking about MSN and geo-location. I've not had a regional issue to deal with, and have thought that hosting in the US gives equal footing everywhere because so much is hosted here. But it seems I'm not correct.

[NFFC]

Well eitemiller I'm interested to know...did you change to a US host or stay with the Canadian host, etc etc?

[eitemiller]

Originally Posted by NFFC

Well eitemiller I'm interested to know...did you change to a US host or stay with the Canadian host, etc etc?

Negative. I was doing in-house SEO for that comany on a limited basis. I showed my bosses the thread, and they made the decision to stay where they are. Personally, I would have moved it - at least just to see what would happen. Curiosity and all that...