We have refined our site to be search friendly in the past month or so. Google is loving what we have done so far. The googlebot is consistently hitting our site on a daily basis and staying for quite some time. Many of our search terms are coming up on page one. The question remains as to why Yahoo has not been into spider our site once since this project launched. Ideas as to why we would see this happening?

It does not appear that our site has been black listed from Yahoo. Our site is www.shopathomeselect.com.

Please help provide some input.

Thanks!
Michelle

Yahoo has more than 2000 pages of your site,


http://search.yahoo.com/search?p=site%3Awww.shopathomeselect.com&toggle=1&ei=UTF-8&fr=FP-tab-web-t&dups=1

Google has more than 3000 pages, so what more you need from the yahoo crawler,

Yes yahoo slurp(yahoo crawler) is not as frequent as googlebot but it will and is indexing a lot of sites, Just be patient and keep adding quality inbound links,

Thank you for the comments. It seems that the pages Yahoo has indexed were from the site before we updated it because our new pages are not indexed. I will continue to be patient and keep watching to see if they come back in.

Thanks!
Michelle

Michelle, the code seems a little complicated and there seems to be something about frames. But just about every single link, within a click or two, leads to a log-in page where you have to become a member.

Crawlers can't deal with that. I haven't checked it all out because the code is so heavy, but it seems the links are going to a dead end. Crawlers can't sign in, so they're stopped short. And I don't know if the navigation all connects up; it doesn't seem like it does.

Google may be indexing, but look how these URLs appear in a search for part of the URL

http://www.google.com/search?q=allinurl:selectframe.asp%3F&num=100&hl=en&lr=&c2coff=1&safe=off&filter=0

They're not fully indexed; I haven't checked thoroughly but so far it looks to me lilke there's a crawling problem with the site.

Thank you for the feedback. The links that you pointed out from Google are part of the login issue. We are working to move the login to a new directory and add robots.txt. Would this solve the issue?

Also, while Google does show many of these links, they have also indexed many other pages that are not effected by the login and we are seeing visits from organic search on google, but not Yahoo. Google has done a much better job moving past the login page where Yahoo has just continued to keep hitting it and not moving on.

I am hoping that moving the login page to another directory and adding robots.txt will help. Thoughts?

Michelle

"we are seeing visits from organic search on google, but not Yahoo"

Just to be clear - the visits you get from the engines have nothing to do with the amount of pages you have indexed. You can have 10 pages in Google and 3000 in Yahoo and still end up with more traffic from google or vice versa.

Y and G have two different algos. Because a site does well in one does not mean it is automatically going to do well in another. You will need to learn more about the Yahoo algo to rank better in Yahoo if the traffic is a concern.

Again, this was aside from the crawling issue. I just wasn't sure from your last message if your concern was also that you were getting more traffic from Google and not enough from Yahoo.

I realize that Yahoo's algo is different from Google. Which is part of my question.. being, how do I figure out what will work well for Yahoo? Google has indexed 3xs the amount of pages that Yahoo has and I am trying to figure out what I need to do to get Yahoo to start indexing. ie. what can I do to modify my site to make it more search friendly for Yahoo?

Second point that you brought up... I am pleased with the traffic I am getting from Google, but I see no visits from Yahoo so I feel like I have only accomplished half of my goal. I know that this is a constant process of refinement and trial and error and I will keep trying until I hit the mark but I am hoping for a little insight as to where I should go first. Will adding Robots.txt to the login page help Yahoo index my site?

Do you have any suggested resources to look at to learn more about Yahoo's algo?