Yahoo, Redirection and the Problem of Disappearing Sites

[Marcia]

OK... I know this other discussion is not the Yahoo forum but Yahoo was mentioned with regard to the redirect issue

http://forums.searchenginewatch.com/showthread.php?t=2979

The disappearance of sites due to being redirected to is a matter of growing concern to webmasters, as indicated in this discussion back in October

Company URL Vanishes from Index (http://forums.searchenginewatch.com/showthread.php?t=2041)

It's been rumored that Yahoo is dealing with the issue of 301's, 302's and hijacking of domains. It would be nice to have something official or some clarification on what Yahoo is doing to remedy the situation.

[TheotherTim]

Well I can say we recently revamped the way we handle redirects in Yahoo search. We have documented the behaviors of 301s, 302s and meta refresh redirects in the presentation that I gave at Pubcon in Vegas. I will go through this again at the meet the crawlers session in Chicago next month. The presentation is available for download on the Yahoo Search Blog ( http://www.ysearchblog.com/ ) and is called "Search Engines and Webmasters". The changes have been rolled out and will take place incrementally over the coming weeks.
Tim

[Marcia]

The changes have been rolled out and will take place incrementally over the coming weeks.
Thanks Tim, and especially for making the information accessible and publicly available to concerned webmasters.

It's a serious issue and just isn't fair! We'll look forward to hearing more on it as the changes roll out and hope to hear about missing sites being restored.

[Nacho]

I'm doing the Yahooooooo Dance Now! Thank you so much Tim! My prayers have been answered. Drinks are on me in Chicago. :)

[jen]

There are many interesting comments by Tim Converse in the interview. On the problems with redirects he more or less blames webmasters for sloppy maintenance of their sites saying: "The underlying problem is that people out there haven't changed their links and search engines do pay attention to links."

On their perspective on search Converse reckons they have a new mantra: "The big things for us are "relevance," "comprehensiveness," "freshness," and "presentation." That's "RCFP" and it's kind of our mantra...

The way we're approaching search itself has changed a lot.

For instance, comprehensiveness is a much bigger deal these days. In the Inktomi days we wanted just one copy of anything that was good because serving documents costs so much. Now we'd really like to have everything.

So then the challenge is ranking everything appropriately. You really want to put everything out there but then... the challenge is identifying and appropriately ranking it all."

The head to head with Google about index size has been well documented but does the drive by Yahoo! for super sizing their index mean they will be crawling more of the web or lowering their quality criteria so as to increase the number of pages they can shout about?

[fatbill]

Well I can say we recently revamped the way we handle redirects in Yahoo search. We have documented the behaviors of 301s, 302s and meta refresh redirects in the presentation that I gave at Pubcon in Vegas. I will go through this again at the meet the crawlers session in Chicago next month. The presentation is available for download on the Yahoo Search Blog ( http://www.ysearchblog.com/ ) and is called "Search Engines and Webmasters". The changes have been rolled out and will take place incrementally over the coming weeks.
Tim

Thank you for the openness of the information Tim.... any idea when this will be going into affect?

Terms I target have rampant re-directs... IE: 6 of the top 10 are straight meta re-directs....

[TheotherTim]

Bill,
We are changing the way we treat redirects. If you are seeing spam that is using redirects you can report this at: http://add.yahoo.com/fast/help/us/ysearch/cgi_reportsearchspam
If you have examples of ways in which we are treating redirects inconsistently with how we have outlined in the presentation. Please send me forum mail and we will look into it.
Tim

[fatbill]

Tim, thanks for the reply.. it really is nice to have a responsive engine! ;-)

that url goes 404, though...

[Marcia]

Bill, let's see if this works then

To report dodgy URL or redirect problem to Yahoo (http://add.yahoo.com/fast/help/us/ysearch/cgi_reportsearchspam)

[Marcia]

Yahoo redirect handling information - link to slide here, message #2:

SES Redirection Recap Thread (http://forums.searchenginewatch.com/showthread.php?t=3338)

It's very clear how they're being handled now.