Site map for big sites

[Dj Morri]

Hello, I work for a company with more than 500 thousand pages but only Google index the majority of the pages.

There is any type of technology or any type of sofware where I can create an automated site map for all those pages?

Thank you

[dyn4mik3]

You may want to check out one of these solutions:
http://code.google.com/sm_thirdparty.html

If that doesn't work for you, then you could write a simple crawler that could create a sitemap for you.

[ssjothun]

Even though I've only used it on pages with less than 100K pages, I really like GSite crawler.

As an example, one site I was working on had about 2500 pages in the Google index. The day after submitting the sitemap, he had 12 500 pages indexed, and about 15K now, so it certainly helps a lot.

I would recommend making a plain HTML site map as well, however, your site is really huge, so it would be a pretty comprehensive job.

[v9designbuild]

The Google Sitemap indexes all your pages. The place to auto-produce the code for this is http://www.sitemaps.blocklayer.com/Default.aspx. It will take a while to put dates and priority ratings on them, but worth it!

[ssjothun]

it says "Max entries in sitemap = 1000" which is about 500 times less than what this guy has...

Go for GSite Crawler...it's awesome and free.

[v9designbuild]

You are indeed right. Will play with it when I have time. Thanks