My 301 Redirect/Indexing Story

[ephricon]

(also posted in HighRankings and DP forums, thought I'd share the news though as I'm quite excited and up until this morning was dying to hear about an expected timeline for this sort of thing...)

Hi All,

I just wanted to share some news in case anyone out there is just implementing some 301s and wondering how long it will take to kick in.

My site is a dynamic e-comm site with several thousand pages. We redesigned the site but kept in on the same URL, although all the pages changed URLs as we had been using ASP and wished to changed. We put 301 redirects for all pages and also implemented a mod-rewrite equivalent to make the URLs a little more search-friendly by removing the parameters. The site is a PR 5 on the homepage and previously had roughly 2,000 pages indexed on average in each of the big 3.

- March 19 - launched new site w/ redirects
- March 23 (about) - MSN and Yahoo show some of the new pages - MSN shows 11 new pages and Yahoo shows about 30. Old pages are still in the indexes. Google shows that it has re-indexed the homepage only.
- March 26 (about) - MSN still showing 11 new pages and all the old ones, Yahoo now showing 95 new pages and all the old ones. Google still only 1 (homepage) and all the old ones.
- March 31 - Google still only showing the new homepage and all the old ones. I'm getting worried. Yahoo shows 110 new pages and all the old ones. MSN shows still 11 new pages and all the old ones.
- April 1 - Google now "reporting" about 4,000 pages and will actually let me see 1,000 of them (they stop after that apparently). Hooray! Old pages appear to still be there too. MSN still showing 11 new and all the old, this hasn't changed for a while. I've not seen their bot in my logs for a while. In checking the logs I did see Google make a real deep crawl just yesterday - about 10,000 hits on about 100 visits from it. Yahoo has been crawling much more frequently but not as deep. Yahoo now suddenly drops down to only showing 70 new pages.

So today, just shy of 2 weeks to the day I've gone from 1 (homepage) to 4,000 + pages all now indexed in Google. Hooray! Let's hope this isn't an April Fool's joke on me!!!

[dizzle]

Nice, congrats on the conversion.

I am trying to do something similar ( http://forums.searchenginewatch.com/showthread.php?t=4988 ) but I am worried that they will index both pages and give me duplicate penalties. Was this a problem for you? My site is also a PR5 and about the same amount of pages. I have a couple of questions if you wouldn't mind...
Were you using ISAPI_Rewrite (IIS) or mod_write (Apache) or something else?
How did you do the 301 redirects? The same way you did the rewrites, or did you use ASP/PHP ?

[krisval]

I am about to do this with some of my older sites. I have stayed very clear of any redirects for a year and a half after I made the mistake of using a temporary redirect.

What is the safest possible way to do a permanent redirect? Does anyone have detailed instructions for the .htaccess file so that I ensure that I do not get a penalty. I know the techincal procedure, but if anyone has a list of things to do or not do in the .htaccess, please point me in the right direction.

Using Apache.

[ephricon]

Not sure I can give you the technical details regarding the 301s, as I work in a corporate environment and thus I said "we need to do 301 redirects and preferably rewrite the URLs" and the developers said "okay". :)

With regard to duplicate penalties, as I understand it 301s tell the search engines that its a permanent redirect, and thus they should index the new page and remove the old one. I expect it to take a little while to complete, but in theory I've told them the old pages should be removed, and even so they are no longer on the server - just the new pages.

Another point here is that we didn't just change the server technology, we did a full redesign - new graphics, new table structure, new copy, new navigation, new copy, new title tags, new new new. The only thing that is the same is the data in the DB that feeds the product detail pages. As such, this is maybe 10-20% of each page, and in fact its even being displayed differently in conjunction with the new design. Thus, for this reason I'm not worried about duplicate pages since they are very different, plus in theory the 301s should cover my back anyways.

I'm not sure how the exact rankings will shake out as in my experience its best to wait a couple of days following any major shift to really do any testing and draw any initial conclusions. Thus, I'm not going to even check any rankings until next week.