Dynamic to Static indexing Problems

[kutklose]

I recently mod_rewrote my dynamic content to static. MSN Search has managed to index the static pages. Google has dropped all the old dynamic pages but have not yet indexed the new static urls. Same for Yahoo. My question, will Google and Yahoo index the static urls or is ther a trick to bypass this problem. I'm aware of that people have posted about the same problem but I havent been able to find a good answer.

Thanks in advance.

[kassad]

same for me

[seomike]

Take a look at tip 1 (301 redirect old dynamic urls to new static urls)

http://forums.searchenginewatch.com/showthread.php?t=3925

Mod's aren't a definite gaurantee to get reindexed. If you have a new url structure with tons of subdomains like site.com/this/is/my/new/mod/rewrite/1 then you'll need to rethink your structure because you'll need some hefty page scores to push a spider that deep.

If you've just bearly kicked your mod rewrite then just wait a bit the new urls can take a while to get picked up.

[kassad]

ah ok thx a lot

[kutklose]

Thanks for the answer.
The old dynamic links look for instance like
www.domain.com/menues.php?head=27
and the equivalent static is
http://www.domain.com/category/index.html

For instance take a look at the following; that is how business.com has it.

http://www.business.com/directory/finance/

As for me, I have it like http://www.business.com/directory/finance/index.html
and same goes for all the other categories. I run a similar service like business.com. Somebody suggested that I get read of the index.html. How does the 301 work in this case?

[PhilC]

It makes no difference to a 301 as long as it redirects to the right version. You have to remember that ....../ and ....../index.html are 2 different URLs and are rightly seen as 2 different URLs by the engines. Whichever version you choose, stick to it for all links, including the 301s.

[kutklose]

It makes no difference to a 301 as long as it redirects to the right version. You have to remember that ....../ and ....../index.html are 2 different URLs and are rightly seen as 2 different URLs by the engines. Whichever version you choose, stick to it for all links, including the 301s.

The old dynamic links www.domain.com/menues.php?head=13 is redirected to the mod_rewriten static http://www.domain.com/category/
since the "category" can have many under pages, I have choose solution like http://www.domain.com/category/page1.html and son on, should I have those links like http://www.domain.com/category/page1/ or??

[PhilC]

There is no need to make a directory URL for every page. If a category has sub-pages, put them in the category directory:- ...../category/page1.html

[kutklose]

Is it enough to redirect the old dynamic links with 301 to the new static link.? and how long can it take before they are crawled?

Thanks

[Mikkel deMib Svendsen]

how long can it take before they are crawled?

From a few days to weeks or months depending on how many pages you have, how popular your website is, what kind of other (even minor) indexing barriers there might be and not the least how well your internal and external linking is to those new pages. In other words, there are so many variables that it's very hard to guess the exact time it will take for your site specifically :)

[kutklose]

Thanks alot for your time and advice.

I noticed that Yahoo has started indexing the new static links. However, it has indexed them like: www.domain.com/cat but he desitination links that are mod_rewritten look like www.domain.com/cat/. In other words, when you click on www.domain.com/cat, it does not take you to www.domain.com/cat/. But msn.com has indexed them like www.domain.com/cat and they are linked to www.domain.com/cat/.

If i take www.business.com and an example, yahoo has indexed www.business.com/cat and when you click on the indexed link, it takes you to the right link which would be in this case, www.business.com/cat/ which is probally is rewritten also.

Any ideas on how to overcome that porblem with yahoo?

[PhilC]

It's normal for Yahoo! to leave the trailing slash off even though it's wrong. What normally happens with Apache servers is that the server gets the browser to request again with the trailing slash included. But you are intercepting the requests so you need to accommodate it in your request handling - not just for Yahoo!, but for every time a request is made without the trailing slash.

[Andy1969]

Ok I've been trying out quite successfully an asp mod rewrite for my product pages but want to know why my dynamic pages are not being indexed as I don't think the mod rewrite is necessary for the products I want indexing.

e.g. dynamic link
www.mydomain.com/item/product.asp?ProdNo.=1234

mod rewrite

www.mydomain.com/modrw/cool/1234_coolitem.html

Now the actual mod rewrite has been very successful and Google has been lapping up these pages, however I still want to know why the other dynamic pages are not getting indexed (the ones I havent re-written yet). Some of the dynamic pages are linked to from pages with a PR5.

The dynamic and the html pages are exactly the same when it comes to tags, copy, code (i.e. view source), so anyone have any ideas for me please?...it's sending me mad!!

[PhilC]

Unless the dynamic URLs have too many name/value pairs, and as long as the links to them are crawlable, there is no reason why they shouldn't be spidered.

[Andy1969]

Sorry Phil I don't understand 'name/value pairs'?

Thanks

Andy

[PhilC]

Name/value pairs are item=value. E.g:-

page.asp?item1=value1&item2=value2&item3=value3

That has 3 name/value pairs. People think it is advisable to keep them down to 3 pairs or less in a URL, although it is not a hard and fast rule.