Info about duplicated content

[onesearch01]

I have read a post who wrote Vanessa Fox in google webmaster central blog (http://googlewebmastercentral.blogspot.com/2007/06/duplicate-content-summit-at-smx.html) about duplicated content in a Portal. Vanessa didn´t comment anything about if this question could be banned or not.
My questios is if a very big portal which use rewrites for url´s could be banned because some rewrite isn´t be correctly and there are some pages with different url´s. So, we have the same content (the same page, it isn´t copy content) in different url´s, it depends to the navigation of the user (from where arrive the user to that page).

Is important to try avoid this content duplicated because we´ll see shared the page rank and outlinks to our page, or we could be banned?

thanks a lot for replies!

[mcanerin]

"Banned" isn't really the right word here. I'm normally not too picky about semantics but I think it's important to outline search engine responses, because it also helps you understand how a search engine approaches things and what your response should be.

Search Engine Responses (in order of seriousness from a webmasters viewpoint):

1. Lack of trust -the page will rank lower than would ordinarily be expected. This can be for many reasons, including internally duplicated pages, lack of links, low numbers of keywords, etc. The search engine simply isn't sure this is the page it should be showing as a result, so it moves it down the list. It still shows it because it thinks it may be relevant to the user. This can be recovered from fairly quickly by fixing the issue. It's page specific, though if the reason is because of a bad URL structure, it may affect every page in the site. One variation of this is sites in the supplemental results. In many cases, the page simply hasn't been SEO'd.

2. Filter - The page in question is removed from a particular SERP, but is still a candidate for showing up in other SERPS. In short it's keyword based. This is common for duplicate content. If you had 2 pages that were almost identical, but one was focused on Red Widgets and the other on Blue Widgets, then only one of the pages would show up for "Widgets". However, a search for either "Red Widgets" or "Blue Widgets" would still show the appropriate page.

3. Penalty - The page ranks much lower than it should because the search engine has serious trust issues with it - ie it thinks you may be spamming. In this case, the penalty is for a period of time or until the search engine decides to change it. This time-based penalty prevents spammers from constantly doing something, then fixing it, then doing something else to test different spamming tactics. It is also often applied to the whole domain, rather than just the page in question.

4. Banning - The entire domain is removed from the index and the spider no longer indexes it. This is the most serious response, and is usually permanent unless a search engine manually adds the site back in, for example, due to a successful reinclusion request.


To answer your question then, you would be looking at either potentially a lack of trust or filter issue. I would suspect a lack of trust, since it's the same page, just different URL's. The fix, then, is to change your URL structure. One method would be to use a redirect script (with a 301) from one version of the URL to the preferred version, so you only have one URL per page.

Ian

[rememberramu]

"Banned" isn't really the right word here. I'm normally not too picky about semantics but I think it's important to outline search engine responses, because it also helps you understand how a search engine approaches things and what your response should be.

Search Engine Responses (in order of seriousness from a webmasters viewpoint):

1. Lack of trust -the page will rank lower than would ordinarily be expected. This can be for many reasons, including internally duplicated pages, lack of links, low numbers of keywords, etc. The search engine simply isn't sure this is the page it should be showing as a result, so it moves it down the list. It still shows it because it thinks it may be relevant to the user. This can be recovered from fairly quickly by fixing the issue. It's page specific, though if the reason is because of a bad URL structure, it may affect every page in the site. One variation of this is sites in the supplemental results. In many cases, the page simply hasn't been SEO'd.

2. Filter - The page in question is removed from a particular SERP, but is still a candidate for showing up in other SERPS. In short it's keyword based. This is common for duplicate content. If you had 2 pages that were almost identical, but one was focused on Red Widgets and the other on Blue Widgets, then only one of the pages would show up for "Widgets". However, a search for either "Red Widgets" or "Blue Widgets" would still show the appropriate page.

3. Penalty - The page ranks much lower than it should because the search engine has serious trust issues with it - ie it thinks you may be spamming. In this case, the penalty is for a period of time or until the search engine decides to change it. This time-based penalty prevents spammers from constantly doing something, then fixing it, then doing something else to test different spamming tactics. It is also often applied to the whole domain, rather than just the page in question.

4. Banning - The entire domain is removed from the index and the spider no longer indexes it. This is the most serious response, and is usually permanent unless a search engine manually adds the site back in, for example, due to a successful reinclusion request.


To answer your question then, you would be looking at either potentially a lack of trust or filter issue. I would suspect a lack of trust, since it's the same page, just different URL's. The fix, then, is to change your URL structure. One method would be to use a redirect script (with a 301) from one version of the URL to the preferred version, so you only have one URL per page.

Ian
Very helpful suggestion ..
Thanks for your suggestion

[bestimtools]

Yes, a great one indeed

[JPRuss]

If I understand correctly you have 2 urls that point to the same site. So is one url considered a duplicate of the other?

You could use a robots file to have the search engine not index one version of the url, but okay to index the other. That way the search engine doesn't bother with urls that point to the same info.

Just a suggestion