tonerman
12-06-2006, 09:15 PM
After high rankings for many years for most of my site's pages we have hit the skids big. I still have a homepage pagerank of 4, most other top level pages indexed with a low page rank (but they are pretty general in content like contact data, etc.), but none of our important product related ecommerce store pages are in Google's index.
My site was originally indexed as a www. site in 2000. Earlier this year we did a 301 redirect to all non-www urls and specified non-www as the way we wanted Google to index us. That was a big mistake, although our home page regained its original pagerank of 4. Highest pagerank we ever had was a 5. Some of our lower level pages held a 4 until recently.
During the course of this year we have had some self-inflicted wounds like the redirect to non www., and some dings from other people's mistakes. The worst ding from other people's hands came from a site in Europe that pointed their domain to our domain!
We didn't detect this until we saw some funny stuff in Yahoo's backlinks to our site. When we clicked on the first link it was our homepage! So all of our pages and their content got indexed on Yahoo and about 50 of them in Google. The urls read theothersite.com/then our directory structure and content. Because some of our pages create sessionid data and the guilty site had nothing to block the spiders a lot of duplicate pages were created in Google and Yahoo's index except for sessionids in the urls.
The third ding was by our own hands. We made a server move to a new server with a different directory structure and forgot to modify our robots text file for the new directory structure. Without having the proper disallow statements, Googlebot spidered our dynamic pages and created roughly 2200 to 2700 pages of duplicate pages in Google's index. The only difference in these pages was the sessionid data in the URLS.
Then in frustration I stupidly used the Google automatic urls removal tool with our robots.txt file (now correct) to remove the duplicate content. This backfired big! It not only removed (temporarily) the duplicate content, it also took all our old www. pages completely out of Google's index. It also blew away about 1/2 of our non-www pages that were in google's index even though the robots.txt file allowed them.
The automatic URL removal tool also blew away our verified site owner status in Google Webmaster Tools and all the site stats. We're still trying to reverify but gettng no where fast.
Summary? A total mess. We're ok on Yahoo and MSN. We never had a lot of inbound links and getting them now in our current state is going to be very difficult. We set up a blog recently, but getting the new blog in the Google index is going to be hard now.
My plan is to:
1. Keep trying to verify our site ownership in Google Webmaster Tools until it gets verified. Request reinclusion on my hands and knees. If it isn't granted live with the lost pages until they reappear in six months. Hope they reappear sooner.
2. File DMCA violation notices with Google and Yahoo regarding our content indexed under the european site's dot.com name and ask them to remove it.
3. Create as much good new content as possible, hope the new content and blog gets picked up by Yahoo and MSN, and try to get some inbound links from the blog effort.
4. Block the duplicate dynamic content in Google's index with the disallow /*? statement in my robots.txt file and hope the server with the duplicate dynamic content urls (all supplemental results) dies or gets tired of being blocked.
Until my stupid move with the url removal tool I was planning on trying to go back to our www. urls with another redirect. I think at this point that might make things worse.
What I am asking for is any help and suggestions on how to clean this mess up. Feel free to tell me I am an idiot, but please follow ranking on me with concrete doable suggestions. I would appreciate any help and advice the members of this forum can provide. Thank you.
Edit: after posting this thread I got some good news. Our site ownership was finally reverified in Google webmaster tools.
"Some people's lives serve merely as a warning to others." :)
My site was originally indexed as a www. site in 2000. Earlier this year we did a 301 redirect to all non-www urls and specified non-www as the way we wanted Google to index us. That was a big mistake, although our home page regained its original pagerank of 4. Highest pagerank we ever had was a 5. Some of our lower level pages held a 4 until recently.
During the course of this year we have had some self-inflicted wounds like the redirect to non www., and some dings from other people's mistakes. The worst ding from other people's hands came from a site in Europe that pointed their domain to our domain!
We didn't detect this until we saw some funny stuff in Yahoo's backlinks to our site. When we clicked on the first link it was our homepage! So all of our pages and their content got indexed on Yahoo and about 50 of them in Google. The urls read theothersite.com/then our directory structure and content. Because some of our pages create sessionid data and the guilty site had nothing to block the spiders a lot of duplicate pages were created in Google and Yahoo's index except for sessionids in the urls.
The third ding was by our own hands. We made a server move to a new server with a different directory structure and forgot to modify our robots text file for the new directory structure. Without having the proper disallow statements, Googlebot spidered our dynamic pages and created roughly 2200 to 2700 pages of duplicate pages in Google's index. The only difference in these pages was the sessionid data in the URLS.
Then in frustration I stupidly used the Google automatic urls removal tool with our robots.txt file (now correct) to remove the duplicate content. This backfired big! It not only removed (temporarily) the duplicate content, it also took all our old www. pages completely out of Google's index. It also blew away about 1/2 of our non-www pages that were in google's index even though the robots.txt file allowed them.
The automatic URL removal tool also blew away our verified site owner status in Google Webmaster Tools and all the site stats. We're still trying to reverify but gettng no where fast.
Summary? A total mess. We're ok on Yahoo and MSN. We never had a lot of inbound links and getting them now in our current state is going to be very difficult. We set up a blog recently, but getting the new blog in the Google index is going to be hard now.
My plan is to:
1. Keep trying to verify our site ownership in Google Webmaster Tools until it gets verified. Request reinclusion on my hands and knees. If it isn't granted live with the lost pages until they reappear in six months. Hope they reappear sooner.
2. File DMCA violation notices with Google and Yahoo regarding our content indexed under the european site's dot.com name and ask them to remove it.
3. Create as much good new content as possible, hope the new content and blog gets picked up by Yahoo and MSN, and try to get some inbound links from the blog effort.
4. Block the duplicate dynamic content in Google's index with the disallow /*? statement in my robots.txt file and hope the server with the duplicate dynamic content urls (all supplemental results) dies or gets tired of being blocked.
Until my stupid move with the url removal tool I was planning on trying to go back to our www. urls with another redirect. I think at this point that might make things worse.
What I am asking for is any help and suggestions on how to clean this mess up. Feel free to tell me I am an idiot, but please follow ranking on me with concrete doable suggestions. I would appreciate any help and advice the members of this forum can provide. Thank you.
Edit: after posting this thread I got some good news. Our site ownership was finally reverified in Google webmaster tools.
"Some people's lives serve merely as a warning to others." :)