What technique Works Best for Dynamic URLs

[! !]

What Technique has worked the Best for You for getting High Ranking SERPs on Dynamic URLs / Web sites

:) Server-side URL changes (substituting "/" for "?" for example)


:) Using XML feeds


:) Saving the URLs of the dynamic Pages That are the most popular - and including their HYPERLINKS on a frequently spidered page

[Anthony Parsons]

Why not just keep it simple.

#1 - Keep the query string very very short
#2 - Include keywords in the query if possible
#3 - Stay away from session ID's if possible (use *Num* instead of *ID*)
#4 - Build a static sitemap and link all site pages to it / them (plural - sitemaps)
#5 - Have good content, pages will rank if spider can find them as above
#6 - Normal SEO, deliver good code to browser, content rich, internal - external - inbound links

[Mikkel deMib Svendsen]

> #3 - Stay away from session ID's if possible (use *Num* instead of *ID*)


The issue with session IDs is that it can create an infinite number of unique URLs to the same pages. You do not solve that by calling your session ID Num instead of ID.

Some way or another you HAVE to remove session IDs for search engines if you want to minimize the risk of duplicate content being indexed :)

[Anthony Parsons]

Sorry, I meant more along the lines of when you are not using actual Id's, then do not use the ID within in URL, instead replace it with NUM. Session ID's require it, sorry, that should correct the statement as was meant.

[Mikkel deMib Svendsen]

Ok, got your point :)

[seomike]

I say if your going to do it do it right.

Yes search engine spiders can index a dynamic page IF it's not an extensive query. On the other hand I think the SE's know that a dynamic page is just 1 page repeated with different info over and over.

I don't think the solution is to build a dynamic site then link to the dynamic pages from static pages or a site map page.

If you can change your code on page to make it appear static. Then you've solved 2 problems. 1. You won't ever have to worry about your site not being index due to dynamic strings 2. You won't have to build doorway pages into each dynamic page.

A mod rewrite is usually about 5-10 lines of code and a hand full of on page tweeks to make the url look static VS. Making page after page after page of link/ doorways to get your dynamic pages indexed.

Any one can see that it is more economical to write 10 lines of code and change 20 links.

I've tried both and hands down mod rewrites works better. They look static you wouldn't be able to tell the difference if you coded the page static from hand. You can SEO them very easily by using keywords in the rewritten file names or folder names etc.

Here is a directory I build using a dynamic script. I patterned the folders and cross linking after Yahoo's directory. Yahoo is done by hand. Mine is dynamic. Could you tell the diference? www.agent-pro.com

If you want a better description about it you can read it here. http://forums.spider-food.net/index.php?showtopic=1863

[Nick W]

>>Some way or another you HAVE to remove session IDs for search engines

Absolutely, then you get into the whole fun debate about 'is this cloaking..' LoL!

For a really good example of server side implementation (1st choice in poll) checkout the drupal.org CMS (http://drupal.org). Just browse around the site and check the urls, it's luvverly ;-)

Nick

[Anthony Parsons]

I don't think the solution is to build a dynamic site then link to the dynamic pages from static pages or a site map page.

Absolutely Mike. Nice points. I must say though a sitemap is quite essential within any large site. The SE's like to travel, though the better you point them the less problems you have getting them deep into the site. A sitemap from the homepage, whether linking all pages or at least linking the main pages and then some others a little further in on each structure, will give you a higher probability of all engines travelling further through your site.

Everything doesn't work like Google, thus must be accounted for in the equation of probability. A sitemap is essential in any large site IMHO.

[seomike]

I totally agree about site maps :) they are essential. But doing a site map >> static pointer page >> dynamic page was what I was not in agreement with.

Using a mod would allow you to just do site map >> static page. Along with good crosslinking your site will be easily indexed.

btw this is a really good topic of discussion keep the ideas coming!!
:)

[Anthony Parsons]

Aarrrrrr...on the same wavelength now. Totally agree....

[seomike]

right on :D

[Mikkel deMib Svendsen]

Nick, as you are well aware (I know :)) cloaking is a term that not everyone agree on how to interpret, but serving up a different URL-format to spiders (e.g. removing session ID in URLs) is definately not something any of the current major engines will consider cloaking - or spam. It's actually one of the solutions Google directly recomend - on site, and at conferences.

[Nick W]

Spot on. Amazing how it gets some people all hot and bothered though isn't it? LoL!

Nick

[DaveN]

Mod rewrite it or buy the Google Search Appliance Gb-8008 it worked for Amazon

[seomike]

Cloaking is a great method if it's IP delivered and server side based rather than those freakin' piece of crap java redirect cloaks ;)

I'd still stick with a mod but if the site was banned or had penalties in the past. I'd have no problem cloaking a 3rd party domain to point to the product pages ;)

[Jeff Martin]

I have to agree that a URL rewrite handled on the server is the best approach.

Would you guys buy an add on, like ISAPI Rewrite (IIS), or buld your own like using the 404 error page to server redirect to dynamic pages using a static displayed URL (article here: http://www.webmasterworld.com/forum47/1383.htm)?

While the coded 404 error page way is easy I'm thinking with an add-on that runs in memory with IIS and that is built in C++ that it would be easier on the server and provide an MMC console from which to administer.

Thoughts?

[Mikkel deMib Svendsen]

I would chose URL-rewrite over 404 error trapping any time! (if possible)

[Dodger]

Another option is to create static files with the actual query string in them as an include. You can cross-reference the filename in the database and it can be keyword rich too.

Further, on the directory containing the static files with the one-line include in them, you can give them an extension of .html if you want. Just instruct the server to process everything in that directory as PHP, Perl, ASP (whatever). This would be less burden on a server than a mod_rewrite would be and you can get fancy and strip out stop words from the filenames.

For instance, this thread here currently is:

http://forums.searchenginewatch.com/forum/showthread.php?t=45

This could easily be filed under the forum as a sub-directory, then the topic name as the filename:

http://forums.searchenginewatch.com/forum/dynamic-websites/techniques-for-dynamic-urls.html

The file techniques-for-dynamic-urls.html will simply contain:

<?php
include '../../showthread.php?t=45';
?>

[seomike]

Another option is to create static files with the actual query string in them as an include. You can cross-reference the filename in the database and it can be keyword rich too.

When time is money that really isn't very economical.

A mod rewrite would take an hour, two hours tops.

[Dodger]

Maybe I should have explained it in more detail then.

Your CMS/Forum/Ecommerce software writes the new files on the fly as the topics get created. This is done in unison. It does the rewrite progmatically without a mod_rewrite file. This will only take a couple of hours to write as well, then it is done.

The benefits are that you get a keyword rich Url to boot. Instead of a bunch of numbers and a Url that looks like it is 5 or 6 directory levels deep, depending on the complexity of it.

Another advantage to the pseudo-static files is that you can cache some of the output inside of them if you wish. Or archiving (pruning) of the database can be accomplished by dumping it into the file in a raw pure content format (standard html file) and without breaking any links (backward compatiblity), nor have the need for a dynamic link.

A couple of examples would be WebMasterWorld (obvious one) and Craven de Kere's Able2Know forums and portals.

[! !]

These intelligent replies to this Topic are worthy of it's own Newsletter.

If this forum had been more established - this Topic would probably be in the Top 20 on Google and Yahoo for the related important keyword queries - based on experiences with Seochat and Digitalpoint posts and their immediate SERPs appearance.

It is interesting to compare these responses to a post made about a year ago on an equally popular SEO forum.

Sometimes - it there is a belief that a topic is ongoing in it's importance - it pays to get alternate perspectives.

Interesting to compare these replies to those on this forum.

Both were informative - but look at the difference.

http://www.highrankings.com/forum/index.php?showtopic=434
;)

[! !]

After comparing Amazon's Google Results for about a Year - through-out various "GOOGLE Updates" it seem they have found an extremely productive solution for having Dynamic URLs in SERPs for competative Keywords :eek:

Here are just a handful of examples - using a free Online Google PageRank / SERPs
Results Finder: (included are the POSITION AND PAGERANK AND SEARCH TERMS AND LINKS

(notice the "ZERO" PageRanks among nearby 6 - 7 and 8 )




72 http://www.amazon.com/exec/obidos/tg/detail/-/1561712035?v=glance 0
73 http://www.amazon.com/exec/obidos/tg/detail/-/0375507523?v=glance 1

http://seo-guy.com/seo-tools/google-pr.php?act=go&keywords=george+bush&pdepth=10&rlimit=0

George Bush

__________________________________________________ _______________

18 http://www.amazon.com/exec/obidos/tg/detail/-/0895264080?v=glance 0
19 http://www.amazon.com/exec/obidos/tg/detail/-/0375414576?v=glance 0



http://seo-guy.com/seo-tools/google-pr.php?act=go&keywords=Bill+Clinton&pdepth=10&rlimit=0

Bill Clinton


__________________________________________________ _________


56 http://www.amazon.com/exec/obidos/tg/detail/-/0060523999?v=glance 0
57 http://www.amazon.com/exec/obidos/tg/detail/-/0394555082?v=glance 0


http://seo-guy.com/seo-tools/google-pr.php?act=go&keywords=Ronald+Reagan&pdepth=10&rlimit=0

Ronald Reagan
(there are NOW many more examples for Reagan)
__________________________________________________ ______________


10 http://www.amazon.com/exec/obidos/tg/detail/-/B00004ZAV3?v=glance 0
11 http://www.amazon.com/exec/obidos/tg/detail/-/0811826848?v=glance 0

http://seo-guy.com/seo-tools/google-pr.php?act=go&keywords=beatles&pdepth=10&rlimit=0

Beatles

(overture keyword suggestion tool)

Searches done in May 2004
Count Search Term
408438 beatles


__________________________________________________ _______________


66 http://www.amazon.com/exec/obidos/tg/detail/-/0596004478?v=glance 1


http://seo-guy.com/seo-tools/google-pr.php?act=go&keywords=Google&pdepth=10&rlimit=0


Google

(overture keyword suggestion tool)

Searches done in May 2004
Count Search Term
21003312 google





__________________________________________________ __




http://msdn.microsoft.com/msdnmag/issues/02/08/HTTPFilters/

Here is an interesting Feature/ Tutorial on Microsoft's Web site for the non-apache group -

Could this be - "algorithmically catagorized" as cloaking by "some search engine's programming"?

[Mikkel deMib Svendsen]

Indexing and ranking are not the same :)
One thing is to get your database driven pages indexed, the next thing is to rank well accross your keyword scope.

You are right that Amazon have found a way to get good indexing. Part of the reason for that, i believe, is site-architecture but I also believe that a lot of it, in their case, has to do with the millions of other websites that link to them. Great linkpopularity tend to give better indexing. The huge amount of inbound links is also likely to be a very important factor in the many good rankings they obtain.

So, I think it's at least fair to say that running a dynamic website did not limit Amazon in their SEO-goals :rolleyes:

[Webmaster T]

Building dynamic sites which appear static to SE is not that hard to do but it is best to plan it out before starting. IMO, Mod Rewrite or a component are good workarounds for fixing an existing site where the options are limited due to the legacy application. The strategy adopted should fit the solution required. I don't think there is 1 strategy that covers all. IMO, if you only have a hammer everything looks like a nail. ;)

I'm not a big believer in the sitemap solution because it doesn't scale well for large sites with 1000's of products and pages and rarely does it address the real problem which is the dynamic url. If you fix the url you don't need a sitemap which puts where the pages are found in the link architecture 1 level deeper therefore likely affecting PR/webrank (Yahoo equivalent of PR)negatively.

I use embedded parameters/variables in static pages on new or re-designed sites. The static pages can be generated by hand or programatically using the underlying database and a system of included files and templates. For SEO it enables title, descriptions and copy to be edited across any number of pages with an adjustment made to 1 file and keeps the url short, free of parameters and topical/keyword rich since page names and directories are generated by the database. I may add a single parameter to a url to get a subset of the recordset for the page. For instance getting all the products from 1 manufacturer in a category or sub-category. This isn't for indexing or placement but is a good way for users to navigate a large recordset.

IMO, rewriting URLs often doesn't provide the users an easier URL to remember or better navigation since parameters appear to look like directories and urls can get pretty long depending on the number of parameters.

[Mikkel deMib Svendsen]

Very good points, WebmasterT! Good post

I do not think sitemaps are golden bullets either but for many sites I think they do create an additional layer of links that seems to be helpfull in getting a high percentage of pages indexed. But you are right, they do not scale well and as such are not always a realistic method.

When it comes to how rewritten URLs look I understand what you are saying and I do agree that most rewritten URls does not help the users. But it dosen't have to be so. A good example is a commerce site that I recently did some work on. Very ugly .NET URLs with session IDs and 5-7 parameters. With the rewrite we managed to get a clear and readable directory structure - even products end on logical names - like this: www.site.com/children/jeans/cowboy_5436.html You know how those .NET URLs look (when they are worst) so I think this trade was very good - for users too. Changing the entire backend structure was just not an option with this client.

But I think the most important thing you said was: "I don't think there is 1 strategy that covers all." - you are so right! Thats why I am not too crazy about voting on such a thing - at least not without having this discussion too so new users see that there is no ONE best way - but ONE best way for each situation :)

[sidyadav]

I try to use mod-rewriting whenever I can, but if I get stuck - XML's the thing.

I generate an XML file from the stored data in the MySQL database, and then parse it to a seperate file.

This usually takes a decent amount of time, but works well.

Sid

[! !]

News FEEDS, be they RSS, Atom or others can be VERY helpful in publicizing Web sites or Immediate Impact Issues.

Especially when Google Yahoo MSN spiders and places them on their NEWS sites. :)

Are there any Strategies or Techniques for gettting them spidered.

Google - seems to not publicize it's standards for accepting feeds

The selection and placement of stories on this page were determined automatically by a computer program.



The headlines on the Google News homepage are selected entirely by a computer algorithm, based on many factors including how often and on what sites a story appears elsewhere on the web. This is very much in the tradition of Google's web search, which relies heavily on the collective judgment of web publishers to determine which sites offer the most valuable and relevant information. Google News relies in a similar fashion on the editorial judgment of online news organizations to determine which stories are most deserving of inclusion and prominence on the Google News page.

http://news.google.com/intl/en_us/about_google_news.html


source-suggestions@google.com

[Dodger]

Are there any Strategies or Techniques for gettting them spidered.


Outside of the fact that this has nothing to do with Dynamic Urls, I can answer this question easily enough.

The only thing you need to get them spidered is to have a syndication link to the XML file. It is usually that orange colored button labled with a variety of feed types (RSS, XML, etc.) or the Atom button or the Coffee Cup whatever button.

As far as I know -- googlebot was doing some random stabs at the more common RSS files whether they existed or not a while back. Yahoo Slurp is indexing them. MSNbot, not sure. With that said though, what they do with them is another story (although it is obvious that Yahoo is working with them).

[sanchezjr43]

Hello All,

Im new to this.....but anyways, I'm trying to find out how to get better indexing in search engines with dynamic URL's. My website is www.idcardsavings.com and it is database driven. I use monstercommerce as my web server.

If anyone is familiar with monstercommerce, please let me know how to get better rankings in search engines.

Thank you.

[Dodger]

You Urls do not appear to be that bad, as far as dynamic Urls go. Two parameters at most.

What you will need to do is work on more backlinks to your site. The site is only two months old too, at the most. Be patient.

[sanchezjr43]

Thanks for your input Dodger....I'm trying my hardest to be patient, but when we don't get any sales calls my boss looks over my way. You know what I mean?

But I will take your advice and be patient. Thank You.

[! !]

http://www.ddj.com/documents/s=9209/ddj040723asp/ddj040723asp.html

[glenn]

Each method listed will work, but each situation is usually different and the problem is multi-dimensional. Operating system, applications, database, system environment, policies and procedure all play into this equation.

[Nick W]

.......and?

Nick

[glenn]

Just had a hard time voting on the issue and the "other" option ask to explain why...

[searchengine]

SERPs Dynamic URLs appear to be cached in Google and MSN reqularly now.



Google
Results 1 - 10 of about 93,500 from inventory.overture.com for

http://www.google.com/search?hl=en&lr=&q=site%3Ainventory.overture.com&btnG=Search

MSN

http://search.msn.com/results.aspx?FORM=MSNH&srch_type=0&q=inventory.overture.com