E-Commerce Website optimization

[Deelip]

Hi,

I am working on an e-Commerce website. I need some tips from Search Engine Optimization point of view. I am facing problem with URL's. Suppose a reebok shoes is present in our website.
There are different situations comes here:
1. That particual shoes is present under brand category (URL is based on brand here)
2. That particular shoes is present in some hot deals (Here URL is based on hot deals)

That means if we see our breadcrumb navigation its looks like
Parent Category -> Sub Category -> Brand Name -> Product Name
Parent Category -> Sub Category -> Special offers -> Product Name

In the above scenario we got 2 different URL's with same product and that page has same description. In case of a large website of ecommerce, even we cannot generate different sets of content for the same product.

Please share your valuable ideas and experiences here.

Thanks

[beu]

When it comes to e-commerce there is really no way around having duplicate content in some form. For that reason, you shouldn't be penalized. At the same time you could be dividing "link juice" and/or "PageRank" between multiple URLs.

Keep duplicates to as few as possible and don't use this as an excuse to go crazy with your categories.

Concentrate on the user and things will fall into place for the most part!

See bullet point 2 at the link below for more on duplicate content:
http://www.google.com/support/webmas...y?answer=66359

best of luck!

[Marcia]

Originally Posted by Deelip

Hi,

I am working on an e-Commerce website. I need some tips from Search Engine Optimization point of view....

...if we see our breadcrumb navigation its looks like
Parent Category -> Sub Category -> Brand Name -> Product Name
Parent Category -> Sub Category -> Special offers -> Product Name

In the above scenario we got 2 different URL's with same product and that page has same description. In case of a large website of ecommerce, even we cannot generate different sets of content for the same product.

Please share your valuable ideas and experiences here.

Deelip, I have no "valuable experience" or ideas, I'm just pondering this. Why does the product need to have two different URLs? Couldn't there be some way that Product Name could be served to user agents at the same URL regardless of which place (page or category) that it's linked to from?

Added:
Incidentally, will the site be hosted on IIS or on Apache? That could possibly make a difference. I'm just thinking this through, since if it'll be on Apache there might be a way to rewrite (remap) URLs so that Product Name always gets delivered to user agents with the same URL, using regular expressions.

Just wondering.

[AussieWebmaster]

you can have one group in a set directory and use your robots text file to block spiders

[Jazajay]

"you can have one group in a set directory and use your robots text file to block spiders "

True and definatly worth a try, however depending on the amount of products this could be quite tedious and ending up in quite a large robots.txt file. Bare in mind you dont want to exceed 5000 characters. Also this wont stop you getting penalized if any one links to them. Robots.txt just stops the SE's from visting them directly. They will still visit them and index them if the SE find them via a secondary source ie- a direct link.

There is a solution though -
Place this at the top of every page that you want excluded

<meta name="robots" content="noindex" />

What this says is basically do not add this page to the index but follow all links from it.

This has several benifits.
1. If you just block a page via robots.txt you will still get penalized if it is directly linked to.
2. It passes all link equity from and links to that page to other internal pages.
3. Removes link equity from that page meaning all your other pages get stronger equity and a slightly better ranking.
4. No duplicate content penalization.

To answer Marcia I do the same thing useabiltiy. Sometimes items just fit into mutliply categories ie discount/sale but you still want it in the main product page as people might not click on discount and might go straight to the product page. If they cant find it there the chances are they will just assume you don't have it. So to double your chances of a sale you include it in both. 50% of people will leave after only 2.5 clicks satisically therfore you need to maximize information to click ratio which this does.

[beu]

Originally Posted by Jazajay

1. If you just block a page via robots.txt you will still get penalized if it is directly linked to.

Store items shown or linked via multiple distinct URLs are considered non-malicious duplicate content. As a result there would not be a penalty.

To answer Marcia I do the same thing useabiltiy. Sometimes items just fit into mutliply categories ie discount/sale but you still want it in the main product page as people might not click on discount and might go straight to the product page.

Maybe I'm missing something here but, if you noindex a page that is duplicated by default the duplicates are also noindexed. The code in the page including the noindex is duplicated at various URLs.

[jimbeetle]

There's really no point in talking about robots.txt or noindex for this situation; they would be merely Band-Aids that don't patch the underlying problem. For that we have to go back to Marcia:

Couldn't there be some way that Product Name could be served to user agents at the same URL regardless of which place (page or category) that it's linked to from?

From there it depends on DB structure, server config, rewrite capabilities, etc.

[Jazajay]

Store items shown or linked via multiple distinct URLs are considered non-malicious duplicate content. As a result there would not be a penalty.

Cool can you provide me with the URL of an example/test done or of a quote from Matt, or any other of the gods that control our lives, that backs this up?

Maybe I'm missing something here but, if you noindex a page that is duplicated by default the duplicates are also noindexed. The code in the page including the noindex is duplicated at various URLs.

It depends on how you do it. I write dynamic code that places the meta on certain variables. Also if you have clear categories it's relatively simple just to block one url and let the main one through. But as the page is not being added to the index it's content is "invisible" to SE thus removing the duplicate content issue.

There's really no point in talking about robots.txt or noindex for this situation; they would be merely Band-Aids that don't patch the underlying problem. For that we have to go back to Marcia:

I disagree, Marcia's ways good if you know what you are doing however I prefer to implant my "band aid" solution for several reasons -

1. Robots.txt and meta are easy to implement. Every ones not a programmer like us.
2. It works and is relatively easy to set up. The robots text blocks the file from being indexed and if it does get linked to you it doesn't get indexed thus removing the problem of penalization. You can skip the robots.txt if you want to save even more time. But also it allows the equity to pass to all the other links.
3. What if the sale item is only for a day this would be quicker to implement and less chance of you making a mistake and getting penalized.
4

Couldn't there be some way that Product Name could be served to user agents at the same URL regardless of which place (page or category) that it's linked to from?

If that is done wrong surly that could be seen as cloaking - I'm not saying that is what you are suggesting. Also it brings me back to point 1. not everyone knows how to do it where as the robots.txt and meta noindex are perfect for this situation.

Hay there are always multiply was to accomplish the same thing, especially with server code.

Also this is the way I have been taught with dealing with SEO and duplicate content issues. - Professional SEO with PHP - brill book for the programmers a recommended buy (only if you know PHP code though).

The end of the day it's how I do it and I don't have any problems. I disagree that it is "a band aid solution" as it implies it is a stop gap solution and may not work in the future, to me. I think it is effective and not only requires less knowledge to be able to implement, which means there is less of a chance of you making a mistake, but it also uses the html tags and robots.txt how they were meant to be used. Which I always prefer.

Also one if statement on the page brings in the meta. This is also a way to achieve Macia's solution. You apply the meta tag if the refferer isn't what you specify this is easily done if you no what you are doing.

Also using meta noindex was how Matt suggested to get rid of duplicate content about 4 months ago. Sorry I don't have the URL but it is referenced 2ice already on this forum and I gave the link to Beu, I think if it wasn't you I apologize but it was defiantly one of the regulars, in another post on this forum.

Check it out, if you can find it, his answers changed how I deal with duplicate pages.

[Jazajay]

Heres what Matt said on October 8 2007

...So, robots.txt was one of the most long standing standards. Whereas for Google, NoIndex means we won't even show it in our search results.

So, with robots.txt for good reasons we've shown the reference even if we can't crawl it, whereas if we crawl a page and find a Meta tag that says NoIndex, we won't even return that page. For better or for worse that's the decision that we've made. I believe Yahoo and Microsoft might handle NoIndex slightly differently which is little unfortunate, but everybody gets to choose how they want to handle different tags.

Eric Enge: Can a NoIndex page accumulate PageRank?

Matt Cutts: A NoIndex page can accumulate PageRank, because the links are still followed outwards from a NoIndex page.

Eric Enge: So, it can accumulate and pass PageRank.

Matt Cutts: Right, and it will still accumulate PageRank, but it won't be showing in our Index. So, I wouldn't make a NoIndex page that itself is a dead end. You can make a NoIndex page that has links to lots of other pages.

For example you might want to have a master Sitemap page and for whatever reason NoIndex that, but then have links to all your sub Sitemaps.

Heres the link and it was Beu. The quote is taken from that page. The end of the day if that is what Matt suggesting than that is probably the best solution for a white hat.

Also this is from the same interview and is probably the most relevant to this discussion

Eric Enge: Another example is if you have pages on a site with content that from a user point of view you recognize that it's valuable to have the page, but you feel that is too duplicative of content on another page on the site

That page might still get links, but you don't want it in the Index and you want the crawler to follow the paths into the rest of the site.

Matt Cutts: That's right. Another good example is, maybe you have a login page, and everybody ends up linking to that login page. That provides very little content value, so you could NoIndex that page, but then the outgoing links would still have PageRank.

Now, if you want to you can also add a NoFollow metatag, and that will say don't show this page at all in Google's Index, and don't follow any outgoing links, and no PageRank flows from that page. We really think of these things as trying to provide as many opportunities as possible to sculpt where you want your PageRank to flow, or where you want Googlebot to spend more time and attention.

I would suggest reading the whole article it is very informative.

[Marcia]

The most efficient way is to use mod_rewrite to remap the URLs, so that when the "good" URL is requested by User Agents, the server fetches the dynamic URL to serve up. That way, the page only ever *exists* with one URL, but the site's links should then link only to the good URLs.

That's on Apache; for IIS it's ISAPI rewrite, I understand. If neither is an option, a bandaid is the option that has to be used.

[Jazajay]

Yeah I'm not disputing what your are saying I just prefer to go with what Matt says.

[beu]

Originally Posted by Jazajay

Cool can you provide me with the URL of an example/test done or of a quote from Matt, or any other of the gods that control our lives, that backs this up?

"Duplicate content generally refers to substantive blocks of content within or across domains that either completely match other content or are appreciably similar. Mostly, this is not deceptive in origin. Examples of non-malicious duplicate content could include:

* Discussion forums that can generate both regular and stripped-down pages targeted at mobile devices
* Store items shown or linked via multiple distinct URLs
* Printer-only versions of web pages"

- http://www.google.com/support/webmas...y?answer=66359


Because Matt and Eric are talking about duplicate content in that interview, I didn't take it as being applicable in cases where store items are involved because of what they say in my quote above. Also, don't forget that ecommerce sites want to rank and be indexed for the names of products they sell.

Like Marcia said, I'd think a rewrite would be the best idea but I'm not sure what shopping cart issues could come up with all the different carts on the market (as in would it process the order and other). Most carts I've seen make a duplicate by default when a product is added to a category (unique product ID parameter & category parameter). It seems to me that forums would face the same kind of issue as I'll illustrate below.

Playing devil's advocate here but would you consider the following URLs duplicate content, is it a problem here and if so how would you fix the problem?


1 http://forums.searchenginewatch.com/...unt=1&p=119325

2 http://forums.searchenginewatch.com/...25&postcount=1

3 http://forums.searchenginewatch.com/...t.php?p=119325

[Jazajay]

Healthly debate I like it.
Ok I'm not saying Marcia's idea is a bad one I'm just pointing out several thraws with implamenting it for everyone and an unternative solution. you have to bare in mind whats best for you, and me, might not be best for someone who is new to rewriting or doesn't know how to do it.

To answer your question through I probably would class those 3 pages as duplicate content. From what I have read duplicate content exists when 70-80% of the content on 2 pages match. I reckon this is true due to the fact that if it wasn't you would only have 1,2 or 3 ecomerce product pages from differnet sites in the SERPS as they all contain same to similar content most of the time. Manufactuers product description, price etc.. however the pages differ by navigation, page display etc... meaning that pages are relativly different.

So as all 3 pages you provided are 100% similar I class that as duplicate content.

How would I get around that?
That is a difficult one.
Not knowing how they are called in and from what pages makes it even more difficult to answer.

If there was a noticable pattern as in off the CPanel or off the thread number and they are all just pop-ups I would block the popup they are in via robots.txt and use meta noindexed in the head of that page/popups.

I have a how to use this item section on my site which is simlar. However I don't want it in every page due to the fact that it is quite large and gets brought in via every item that is in that category.

How I deal with this is I use an object tag, as iframe wont validate, for all but 1 item and block the object.
So something similar to this

PHP Code:

if($thisPage!="$itemname"){
object .....
}
else
{
display how to information directly in the page.
}


the object tag is blocked via robots.txt and has meta noindex in the head. What I can then do is simply change the $itemname varaible which is set at the top of the page to another item. The previous item then gets the noindex object tag. This is only done if I want to improve the rank of another page in that category by giving it more detailed information and pushing it up in the SERPS. The previous item and the item that have just got the "new data" go straight on my home page as featured items so they can be reindexed quickly. Just incase there is a delay in spidering and I get penalized for duplicate content for a short period of time. In this situation rewrites aren't a great idea, in my opinion.

However there is no reason why I wouldn't use it on the pages you gave. I would have to look into how they are brought in in more detail. But I cant see why it wouldn't work.

Again I'm not saying Marcia's idea is a bad one I'm just saying it's not the only one and with server code there are always numerous ways to acheive the same thing. Just depends on what you prefer and your level of expertise. I can do both but I prefer my way as I'm using the tags how they are ment to be used and is A way Matt suggests to remove duplicate content.

I don't always take what Matt says to heart but that interview did change how I deal with similar pages/duplicate content. You also have to bare in mind Google isn't perfect it wasn't so long ago that thier blog or something simlar was de-indexed/penalized for apparently using bad techniques when it wasn't.

So for that reason I use this technique on pages that have similar urls - Googles not perfect. Bare in mind I choose one of the simlar pages, on my site, to be "the main page" and that page does not get the robots.txt or meta applied to it. So the product item still shows in the SERPS.

I hope this is balanced as both techniques, in my opinion, are a vailid way of acheiving the same thing, less crap on the net

Dellip I bet you didn't think you were going to get a nice debate hay lol