Do keywords in URL influence your rank? Research by Web CEO Team.

[Web CEO Team]

Moderator note:

Moved to Search Technology and Relevancy area from Beta Test area
__________________________________________________ _______________

Hello guys!

My name is Serge Bond and I am a search engine analyst working for www.webceo.com. I decided to find out the real importance of keyword presence in URL. There are so many empty-minded topics with lots of opinions and no research preceding them. That's why I performed the following research:

I made a list consisting of both very competitive and uncompetitive keyphrases. The level of competition was estimated by number of results (in Google and Yahoo) and by the number of daily searches (taken from WebCEO).
Overall, I have taken 24 keyphrases with almost 30000 URLs. Here’s the list:



By each keyword I have extracted the URLs of top-700 pages. This was done, again, with the help of Web CEO. Even though our tool lets you extract up to 1000 URLs, Google rarely gives really more than 700 (even less, as a rule). You can see the number of URLs for each kwd in the table above.

After this, I have created a spreadsheet summarizing the number of keywords from the keyphrase in these URLs. For example, let’s consider the keyphrase “search engine optimization”. In my spreadsheet, next to the URL search-engine-bla.com would stand “2” (because in the URL are present “search” (1) and “engine” (2)). Accordingly, in front of search-engine-optimization.com would stand “3”. For example:



You can see all other tables at my blog http://seometry.blogspot.com

Then, I have made charts for each keyword. At the horizontal axis we have site rank. At the vertical axis we have the number of kwds from keyphrase in the URL corresponding to each rank. Here you can see the most typical patterns:

Competitive keyphrases:


For competitive keywords we can see the following picture:

• In Yahoo pages with many keywords in URL are saturated at the top of the list.
  
  In Google the picture is almost the opposite. We can see more pages with keywords in URL at the bottom of the list.

Uncompetitive keyphrases:


For uncompetitive keywords the situation is different

• In Yahoo the general picture remains the same.
  
  In Google the distribution becomes similar to Yahoo: pages with many keywords in URL are situated at the top of the list.

My conclusions:

For Yahoo presence of keywords in URL is a considerable ranking factor.

In case of competitive keywords, Google takes presence of keywords in URL as overoptimization and penalizes a page.

In case of uncompetitive keywords, Google uses this factor in its ranking process.

[creativecraig]

Looking through your competitive keyphrases they dont seem to be that competitive. Its very difficult to say a phrase is competitive just by looking at how many results are returned!

I would say one of the most competitive industries to optimise in would be the financial sector. Because the returns are so how high for good placement everyone wants a good ranking - yet I dont see you mentioning that in your research.

[Web CEO Team]

Hi Craig,
Thank you for your advice. Please tell me several highly competitive finance keyphrases and we'll perform this research on them. It will take two hours or so, then we'll immediately post it in this thread

[ppg]

Agree with craig regarding the 'high' competition phrases. The only really competitive phrase I see in there is search engine optimization.

Try a search on G for the secong one in your list in quotes. I really wouldn't call that competitive.

As far as the conclusions go, agree with number one, however:

In case of competitive keywords, Google takes presence of keywords in URL as overoptimization and penalizes a page.

hmmm. Try some searches on popular 1 word pharmacy terms. Keyword in URL all over the first page, I can't see any penalties there.

Its good to be posting the results of research for everyone's benefit, but imho both the method and conclusions are a little shaky.

[Web CEO Team]

Originally Posted by ppg

hmmm. Try some searches on popular 1 word pharmacy terms. Keyword in URL all over the first page, I can't see any penalties there.

Tell me several and we'll do this. It would be better if you tell me the keywords or keyphrases (I don't want to be biased in any way)

Originally Posted by ppg

imho both the method and conclusions are a little shaky.

Yes, I agree with you. They are a little shaky. If you have a better method or better conclusions (no sarcasm here, I sincerely would be very glad to see any new idea), please share them with me.

Best,
Serge

[creativecraig]

Mortgage, remortgage, loans, credit card... the list is endless - I think that most people would agree that these are competitive.

[ppg]

If you have a better method or better conclusions (no sarcasm here, I sincerely would be very glad to see any new idea), please share them with me.

Well, it would be better to be using some more competitive keyphrases for your high competition terms. People will rarely post their pet terms on a public forum though :-)

Other things need to be taken into account to judge how competitive a phrase is:

how many links the top sites have
how many of those are internal
how many are from distinct domains
how sharp their on page optimisation is
how effective their internal linking is
how much 'agressive' seo is on the first page of results and how well its done
etc

I'd also like to see a bigger sample size.

Of course all that is difficult to automate, but would produce more meaningful results I think.

I'm not the most dilligent bloke around when it comes to research and testing, so you might want some advice from people who spend more time on that than me, but the above would help I think if you could find a way to factor it in.

[Anthony Parsons]

My conclusions:

For Yahoo presence of keywords in URL is a considerable ranking factor.

In case of competitive keywords, Google takes presence of keywords in URL as overoptimization and penalizes a page.

In case of uncompetitive keywords, Google uses this factor in its ranking process.

I don't believe these results are actually very factual at all. Google actually does not factor bias against a particular keyword, ie. keywords in URL's help non-competitive but not competitive. I think you need to get something more factual before presenting results here.

[projectphp]

Problems with your methodolgy:
1. There are no labels on either the X or Y axis. I assume the X axis is the rank, and Y is number of keywords in the URL, but would have been nice to have been told.
2. Small sample size. Is this enough results to utter the dreaded "therefore"? I don't think so.
3. Other factors ill considered. PageRank, anchor text, on page KWD etc are not mentione at all. These result could all be caused by something else, e.g. Yahoo likes on page text, Google prefers off page factors. keyword-hyphen-domains tend to be well SEO-ed on page, but short on natural links. This would make such sites more likely to rank well in Yahoo than Google.
4. No isolation of factors. This was not a test run on 4 domains and a made up word in which you controlled all variables, including links in and out and all anchor text. Therefore (that dreaded word) can you really make any positive statement that positively excludes other factors?

Not a bad start, but unfortunately the evidence is far from conclusive, the mathematics and assumptions made are not spelled out clearly and concisely, and there is far to much room for doubt. Without control and reporting accuracy, there is very little one can do.

I applaud your attempt at making SEO accountable, but think that perhaps there are better, more conclusive tests, that could have been performed. Next time, perhaps start by posting your methodology, then we can critique that and save you a bunch of time compiling results. If that started out right, you wouldn't have spent time heading in the wrong direction.

[Web CEO Team]

2CreativeCraig:

Mortgage, remortgage, loans, credit card... the list is endless - I think that most people would agree that these are competitive.

OK, I'll check these keywords and post results here. Tomorrow. But, I should note, that checking keyphrases is more important, because optimizing for just one keyword is not a good idea considering fierce SEO competition.

Its very difficult to say a phrase is competitive just by looking at how many results are returned!

I also took into attention the number of daily searches. Next time, I'll show these parameters in the table.

2PPG:

Other things need to be taken into account to judge how competitive a phrase is:
...

We perfectly know which factors influence a page's position in SERPs. In our post we mentioned that this factor is not decisive. We were simply interested to find out how strong can presence of kwd in URL influence page's position is serps. The charts show that pages with keywords in URLs are placed higher in Yahoo than in Google. The next step will be studying other factors. This way, we'll be able to determine which factors are deciive and which are minor.

I'd also like to see a bigger sample size.

Do you mean more keywords or more pages by each keyword?

2Anthony Parsons:

Google actually does not factor bias against a particular keyword, ie. keywords in URL's help non-competitive but not competitive. Absolute rubbish.

Anthony, do you have any factual proof that Google does not factor any bias against particular keywords? We suppose that Google's ranking algorythm consists of several levels. It is widely known that PageRank is the main ranking factor. For pages that have close PageRank values, on-the-page factors come into play, including keyword presence in URLs. There is no bias against some special keywords, however non-competitive ones are weaker optimized, they do not have lots of high PR pages linking to them, that's why off-the-page factors can be better seen.

2projectphp:[indent]

1. There are no labels on either the X or Y axis. I assume the X axis is the rank, and Y is number of keywords in the URL, but would have been nice to have been told.

Fixed

2. Small sample size. Is this enough results to utter the dreaded "therefore"? I don't think so.

What do you mean? More keywords or more URLs for each keyword?

3. Other factors ill considered. PageRank, anchor text, on page KWD etc are not mentioned at all. These results could all be caused by something else, e.g. Yahoo likes on page text, Google prefers off page factors. keyword-hyphen-domains tend to be well SEO-ed on page, but short on natural links. This would make such sites more likely to rank well in Yahoo than Google.

These results are caused by "something else". However different SEs react to the researched factor differently. This can be seen in the pictures.

4. No isolation of factors. This was not a test run on 4 domains and a made up word in which you controlled all variables, including links in and out and all anchor text. Therefore (that dreaded word) can you really make any positive statement that positively excludes other factors?

I agree I can't make any 100% true conclusions. What about the expirement you are talking about, I'll do this. What methodology of testing would you suggest?
Let's do this right here in this thread. Everything: from methodology to math to assumprtions and results. Your help will be very appreciated.

[Marcia]

Hi WCT, and welcome to SEW forums.

One thing I'd love to see some stats on is the effect of keyword repetitions in URLs. Just to illustrate:

www.food-site.com/healthy-food/healthy-breakfast-food.html

See? How are multiple occurrences of the same word in the URL doing in the SERPs, and is there any type of effect with any of the major engines, positive or negative?

[Anthony Parsons]

Anthony, do you have any factual proof that Google does not factor any bias against particular keywords?

I have lots of facts about lots of things, being the founder of SEO Testing and all. I have tested the use of keywords in domain names, folders and filenames, and currently doing some more testing for archive purposes. Many of the people here commenting have also done their own tests, thus not found what you are commenting.

We suppose that Google's ranking algorythm consists of several levels. It is widely known that PageRank is the main ranking factor.

PageRank itself has very little to do with ranking a website actually. PageRank is a unique algorithm that is combined with the main algo's to rank pages. PageRank is "one" of hundreds / thousands or techniques scrutinized to rank a page.

For pages that have close PageRank values, on-the-page factors come into play, including keyword presence in URLs. There is no bias against some special keywords, however non-competitive ones are weaker optimized, they do not have lots of high PR pages linking to them, that's why off-the-page factors can be better seen.

Having high PR pages pointing to a keyword has nothing to do with their competitiveness either. Competitiveness is a measure of how many people are attempting to capture that term, nothing more, nothing less. The impact from people linking to pages attempting to capture that term has no real relevance for the test you are trying to perform.

I would take projects advice, place up what you are looking at testing first, then let people give you some advice on which way to structure your test so you get it right the first time. Keyword presence, hypenated URL's, etc, have nothing to do with PageRank, so you can keep that out of your test to begin with. As projectphp said, good start, but not quite there to an accurate standard. As soon as you have multiple links pointing to a simple test like this, the results diminish rapidly because of all the factors within the link itself. You need to test in a completely controlled and untainted environment.

[ppg]

We perfectly know which factors influence a page's position in SERPs.

I was talking about factors relating to how competitive a keyphrase is, not what factors make a page rank well. I thought that was pretty clear.

It is widely known that PageRank is the main ranking factor. For pages that have close PageRank values, on-the-page factors come into play, including keyword presence in URLs.

Really? I want to come and optimise in your world :-)
PR is important yes, but no more so than other factors. Number, quality and theme of links will get you further than high PR. If what you say is true then results would be ordered first by PR then within that by on page factors. Is this really what you see in the serps?

I also took into attention the number of daily searches.

How does this relate to how competitive a term is?

There is no bias against some special keywords,

In case of competitive keywords, Google takes presence of keywords in URL as overoptimization and penalizes a page.

To be honest, it looks to me like you've started with the assumption that keywords in url does influence ranking and then set out to prove it.

[weby]

Hello Serge,

I think your test is very good for a start and no similar or better tests are publicly available.

Originally Posted by Anthony Parsons

PageRank itself has very little to do with ranking a website actually. PageRank is a unique algorithm that is combined with the main algo's to rank pages. PageRank is "one" of hundreds / thousands or techniques scrutinized to rank a page.

Look at:
http://www.prsearch.net/index.php?Qu...BSearch=Search

If PageRank is a so makes so little difference, how come all listings have a PageRank higher than 5. And how come the only PR 8 site on that page is the first?

[projectphp]

What about the expirement you are talking about, I'll do this. What methodology of testing would you suggest?

Ok. To know that this one issue causes the problem this one factor needs to be the only variance. So, you need several different domains all competing for a common nonsensical word, and then to see which site "wins".

If Google has 100 factors it considers, then a result like you displayed may be caused by any of the other 99 factors. How can we know for sure? It is for this reason that any sort of conclusion is iffy at best.

Some examples of unanswered questiosn:
1. Did where the keywords appear have an effect? Is keywords in the domain different to in the rest of the URL?
2. Was the biggest influence in the results something specific, i.e. PageRank on Google.
3. Do pages on keyword URLs have commmonality, i.e. are they well on page SEOed? What does this indicate? Sometimjes, we test one thing, learn soimething else (taht is how fingerprints being unique were discovered).
4. Are the results on Google slanted towards Brand name non-keyword domains?
5. Have keywords in the URL been isolated enough to have a conclusion?

IMHO, SEO testing often can't prove much of anything.

[Anthony Parsons]

IMHO, SEO testing often can't prove much of anything.

And this is very true, from my experience. The only thing that can be found is whether it influences a ranking or not. How much is always unknown, and I know this from running SEO Testing. If your testing keywords within a domain, folder or filename, the first thing is to ensure those words do not appear in the title or page itself, to ensure no influence is given from that end. No anchors, no nothing in regard to the terms used in what you are testing. That is the only way in which you can achieve a close result, though still not 100% conclusive.

Another thing I have found, is that as soon as you combine factors, then everything can go out the door as one affects another, positive and negative.

There is nothing wrong with your test idea Serge, just that its not near as conclusive as it could be. I think you know that now from the feedback already.

[Marcia]

There are those who believe that the value of keywords in URLs, at least in the domain itself, lies only in the fact that there will be links containing the keywords in anchor text.

That created a bit of stir, when some concluded that there were filters being applied for repetitions of anchor text that exceeded an acceptable threshhold, and that it was possibly put into place for things such as Googlebombing.

Another rumor had it that another engine was seriously frowning on repetitions of the keywords in the filepath to given pages.

I'm not overly "scientific" - I pretty much believe what I see consistently enough - so I'm wondering how things like that can be tested, for example, whether it's the keywords in the URL or anchor text actually giving a boost.

Also wondering how it could be isolated that in fact multiple repetitions in the filepath were causing a problem, or whether a drop in rankings could rather be due to other factors.

[Chris_D]

I decided to find out the real importance of keyword presence in URL

The only way to do that - imho - is to compare two otherwise identical pages - with and without keyword presence in the url.

the sites to test would require identical inbound anchor text links, identical content. Identical. Just isolate the ONE issue you are trying to test - and test it.

Call me simple - but isn't that the definitive test?

Have a look at these SERPS - there's more ways to skin a cat than putting 'skin-a-cat' in the URL:

http://www.google.com/search?hl=en&q=miserable+failure
http://www.google.com/search?q=Weapo...ss+destruction
http://www.google.com/search?hl=en&l...&q=flash+games

[Anthony Parsons]

Exactly Chris. The results are not conclusive due to the nature of the test. Test keywords in domain. Easy.

Make page, put keywords in domain, put keyword in page or title once.

Make identical page, change domain and record ranking.

Same pages, different domains. Measured at unique times obviously. Which one went higher? As I know very well, this is not conclusive because you have rankings all round the page changing.

You could isolate it fairly well by choosing a good keyword that is not so impartial to other page affects, ie. something that is very unique, but has atleast one or two other pages competing against it.

If you have the two domains indexed though, the results will take affect on a near weekly basis though in Google, so you could get a good judgement to any affect through changing them around and measuring each time.

Marcia, I am not hypeing about PageRank or PageRank bashing, it is more that the original test goes on about how PageRank is used in the test to measure keywords. What does pagerank have to do with measuring keywords in a URL. This is from the original post.

[dannysullivan]

I've made a number of edits and deletions to put this thread back on track. I'll ask everyone to please stay on the topic of this thread -- look at the data presented and offer criticisms and comments on it, as was requested. That's useful. Going off on tangents about personal motives and some of the other stuff I've had to delete isn't useful.

By the way, here are some past threads where we've dicussed the issue of keywords in URLs influencing rank:

• Hyphenated URLs
• SEO and file names Feel free to cross-link to other similar discussions you may have seen. I'm keeping a list of these, as it's a popular topic.
  
• Change To Link Bomb Sign Of New Link Analysis Shift?, July 2004
• repeating terms in URL , Sept. 2004
• URL Spaces & Alt Tag Naming Conventions, Sept. 2004