To my surprise, I notice on Yahoo link results the caching of ALL pages on the site, not just the interlinked ones. Peeking where they probably really shouldn't.

Does G follow the same practice and should all non-linked pages be removed from the server?

Yes all search engines can do do this. Pages outside the navigation structure can get indexed when they are found in some unplanned way. If you don't want them indexed you can block them from indexing with a robots meta tag or with robots.txt.