PDA

View Full Version : How do Google and other search engines work?


Lad
07-09-2008, 11:54 AM
As far as I know search engines crawl my website and then index it. But if my website has say 1000 pages and all were crawled does it mean that Google has stored text from all those 1000 pages on their hard disk?(If so it would imply that Google has to store "all" Internet on their hard disks).
Or Google has, for each web, reserved say 100 pages and they are modified and choosen from the list of those 1000 in a way Google wants?
Thank you for reply
la.

AussieWebmaster
07-09-2008, 09:17 PM
yes Google caches and then breaks into data information all pages of all sites they index... well some are not cached if told not to

Misscj
07-11-2008, 10:59 AM
Read "Modern information retrieval" by Baeza-Yates and Ribeiro-Neto, it's a good introduction if you want some specifics.

If you really like that, then Jurafsky's "Speech and Language Processing: An Introduction to Natural Language Processing, Speech Recognition, and Computational Linguistics." will be useful.