Is there a way to search select web pages only?

[DenisS]

Hi,

Is there a way to search select web pages only? I.e., you have a list of web pages (URLs) and you need to identify which of them contain the information you need. The list can be rather big (dozens or hundreds of items).

Are there search engines or tools that can help?

Thanks

[dannysullivan]

Gigablast does something exactly like this, brand new, a site search (http://sitesearch.gigablast.com/sitesearch.html) tool.

[seobook]

additionally GigaBlast allows you to search a group of sites with their topic search (http://sitesearch.gigablast.com/cts.html)

[dannysullivan]

Oops! Thanks seobook -- it was the topic search link I meant :)

[DenisS]

Thanks,

I didn't know I could use page URLs instead of site URLs in Gigablast's topic search. Impressed!

Unfortunately, there are some drawbacks:

1. Some pages I need are not indexed by GigaBlast.
2. Some of them were indexed several months ago and are not relevant any more.
3. I couldn't make it work with URLs with parameters. Example: www.ridgeequipment.com/inven.php3?tab=main&mirror=all

Are there any other tools that will search arbitrary sets of web pages?

[seobook]

3. I couldn't make it work with URLs with parameters.
Are there any other tools that will search arbitrary sets of web pages?
if you set it to the site level Gigablast will search any page on that site. additionally you need not specify the full file name, just the site or the path of the group of pages you are interested in.

freefind and atomz are two site level search products you may want to look at.

most major search engines also have a site: search function

[DenisS]

seobook,

Thanks, but I'm not interested in site search.
The idea is to search the select pages only. Site search may return results from many irrelevant pages.

[seobook]

seobook,

Thanks, but I'm not interested in site search.
The idea is to search the select pages only. Site search may return results from many irrelevant pages.

Right but I was altering you to the fact that you could search filepaths and not just entire sites.

[seobook]

I believe there are also bookmarking programs which allow you to search specific pages... something like furl or spurl may help...

[DenisS]

I believe there are also bookmarking programs which allow you to search specific pages... something like furl or spurl may help...

I'll take a look at these programs...

[DenisS]

spurl and furl look like what I need. The only concern is how they manage frequently updated pages? Will I search the real pages or the cached content?

[seobook]

spurl and furl look like what I need. The only concern is how they manage frequently updated pages? Will I search the real pages or the cached content?
I do not think the system would be too scalable or useful if it had to cache unique dated copies of all the pages over and over again on rapidly changing pages that were frequently bookmarked.

I have not used the systems much yet though...just know of their existance.

another social bookmarking program I did play with for a few days is
del.icio.us

you might be able to find cached copies of some of the dated info at archive.org

[DenisS]

I asked spurl and the answer was: Real-time search is not currently available (although it is technically possible). Spurl stores the pages as they were at the time you spurled them.

Are there any real-time web search tools?

[seobook]

Are there any real-time web search tools?
I cant believe they keep all those old pages as they were then... wonder if there are copyright problems associated with that.

some of the blog tracking systems such as technorati get ping updates