SEO Thoughts on Google

[Dave Hawley]

Nope. Plenty predate Google on this. Infoseek, Excite, Lycos -- all had "deep crawls" in their time back in 1995 completely for free.

Not quite true. Back in 1995 spiders were not developed enough to grab all the 'stuff' that was out there. Google, even now, is going places that it couldn't before. However, it is true that there was not anywhere near as much 'stuff' out there as there is today.

[Mikkel deMib Svendsen]

However, it is true that there was not anywhere near as much 'stuff' out there as there is today.

Yes, and I would even question that Google today index a larger percentage of the web than the early engines did. We don't really know how much is out there but the past studies I've read about it looks like they are still only covering a very small percentage of the entire web.

Chris Sherman recently quoted a Google engineer for saying that to crawl the entire web (including what Chris often refer to as "the hidden web") would probably take them another 50 years of development! Anybody that have been working with large scale web crawling, I think, will believe him. However, we don't have to get to a 100% to be pretty good, in my mind

[Dave Hawley]

I would even question that Google today index a larger percentage of the web than the early engines did

That is an interesting question. I would say yes, but have no proof. Is there any unbias info out there on this?

[Mikkel deMib Svendsen]

Is there any unbias info out there on this?

I actually don't know but I doubt it. It was just a wild guess - and an open question. Maybe Chris Sherman knows it ...

My point was just that Google was far from the first crawler engine that created a WOW effect in the industry

[dannysullivan]

Back in 1995 spiders were not developed enough to grab all the 'stuff' that was out there.

And as Mikkel pointed out, they still aren't. Google Scholar, that program they announced recently? That the spider getting into some content it wouldn't ordinarily access -- but that also mean there's lots of non-scholarly info locked in databases and password-protection systems not getting out. Yahoo's CAP program works similarly -- pulls content by working with non-profits or vendors who pay, but there's stuff missed.

Google's a great search engine, but it's a disservice to the shoulders of those it stands on for anyone to suggest that we didn't find things before it arrived or that only Google has opened up the web more to locating information. AltaVista was a huge, huge deal when it arrived. Go check out how the competitors had to react: Search Memories. That's why it was the Google of its day; or really, why Google is the AltaVista of its day.

Google's great advance was to combine the comprehensiveness that AltaVista spurred on with the relevancy that Yahoo's then human-compiled database offered.

Google, even now, is going places that it couldn't before. However, it is true that there was not anywhere near as much 'stuff' out there as there is today.

Sure, and so do Yahoo, Ask Jeeves and even the new MSN crawler. All they are really doing is being braver about handling dynamic URLs and getting smarter on avoiding traps.

That is an interesting question. I would say yes, but have no proof. Is there any unbias info out there on this?

We haven't had a good estimate on the size of the web for ages. My Search Engine Sizes page covers past estimates. The major search engines have long exceeded estimates of the "surface" web. Last estimate of the complete surface and invisible web was 500 billion pages. Google claims 8 billion pages indexed -- meaning there's a ton left out. But then again, it's never been about having everything -- it's been about having a good sample and great relevancy.

So, what I am trying to saying is that 99% of the people who do SEO are because they want to get listed in this particular SE.

I'm totally with Andy here. I know in my writings for SEW, and producing content for the SES show, I've always strongly resisted trying to focus people just on Google. That's because the dominant position it has held was obviously going to change. It also has changed, but many search marketers are still waking up to this.

Veterans will remember back in say 95-00 how a Yahoo listing, a directory listing at Yahoo, was crucial. You could live or die by 25 words. Backdoors, tips on finding editors, it was all about that -- and in fact, in 99, most of the results on the web were human powered. The focus on SEO -- getting good crawler listings -- seemed diminished.

Yahoo doesn't depend on its directory any longer and hasn't for some time, as the primary results it presents. Anyone who focused solely on that was in trouble. The same was true for a number of firms that dealt solely in poor quality doorway pages, when Google started its rise. Its use of links was very effective initially in combatting that.

The marketplace has already changed. Google is still dominant, but Yahoo has its own results, and MSN will split from Yahoo soon. Anyone who has focused only on Google is being short-sighted. Though having said this, most ordinary site owners people will find that the "google work" they might do still with help with others.

The real, real change people aren't largely understanding is the shift away web results to specialty results. This is the invisible tabs concept I've talked about.

I did a Search Detours panel on this at SES San Jose, and the audience seemed largely confused. We talked about how search engines were shifting slowly away from being so dependant on web results. People didn't get it -- so what were the tips to follow, someone asked.

My response was that they needed to understand that down the line, in some cases, a search will bring back shopping results by default, or news results, or local results, or image results and so on. Web search results will be "backup," the stuff that kicks in only if other things aren't found.

So to come all the way back up to the start of this thread, it's not a problem to me that people might focus just on Google. It's a problem that they assume Google = Google Web Search and they may neglect the many specialized search engines Google and others offer. Those are the ones on the rise, in my view.

[Chris_D]

Google is the most popular SE ever

Dave - haven't you already been told 3 or 4 million times not to exagerate?



I even remember when Dogpile - as one of the early metasearch engines - was touted to be the next rival to AV!

In fact in 1999-2000, Yahoo alone accounted for between 55-60% of all search engine referrals. The second place saw a tug of war between the other search engines namely Altavista, Excite and Go.com. Yahoo's reach was so great that at most times, the second-most popular engine would not account for over 10% of total search traffic.

http://www.promotiondata.com/article.php?sid=320

May-June 2000 .....Yahoo! selects Google as its default search results provider to complement Yahoo!'s web directory and navigational guide......

http://www.google.com/corporate/timeline.html

[neuron]

It's difficult to see how much traffic you get from each search engine if you do not have similar rankings in each engine. I have seen a few sites that were designed just to rank in Google, on the idea that if they can rank well in Google then they are confident they will rank well in the other search engines (I don't care about the other #&*#^& search engines, just get me ranked in GOOGLE!!!).

In these cases, in a site designed to rank in Google, the site op might further disregard the amount of traffic from the lesser search engines because they are bringing significantly less traffic than Google, the search engine the site was designed to rank for, and significantly less traffic than it could bring if the site had attained similar rankings with the lesser search engines. For instance if Google got 65% of all web search, yahoo got 20% and MSN got 15% (just a rough idea), and you designed your site for google, then you might end up with 90% of your traffic coming from Google and only 6% coming from Yahoo and 4% coming from MSN. In which case you say, "what? optimize for MSN or Yahoo? are you nuts? 90% of my traffic comes from Google!!!"

My point here is that the tide has already turned. You may already be losing a lot more traffic than you have anticipated. If you do not have equivelent rankings on all major search engines (watch for contenders) then you cannot appropriately judge how much traffic one engine may give you more than another.

The amount of money and effort you spend on optimization for different seach engines should be in proportion to the amount of traffic you would get from those search engines if you had equivelent rankings.

Also, while I wouldn't bet against google, there are a lot of lessons to be learned in SEO right now if you will simply watch your SERPs in MSN beta dance around, and see who moves up and who moves down in areas you are familiar with.

[sebastian]

...shucks, ...and all this time i have been optimizing for InfoSeek not knowing google was the only *real* SE.

folks, reading all the responses, you can tell who does SEO as business and thus answers questions as a subtle promo for their companies. then you've got your rogue monkeys like myself who try to read and experiement with everything trying to apply logic while filtering through hype. ...but, the best comment in the post is the person who pointed the finger at quality design and development.

it still stands to reason that you will be better than your competition if:

- you follow the DOM in design and layout pages built to xhtml standard utilizing CSS and all the right headers.

- utilize quality copywriting and keyword rich content

- update content consistenly

- build a decent 'inbound' ...and outbound link bank

- pump out some press releases with your url in the first paragraph and link to them on your own site

- destroy any pages called "links.htm" or anything similar

and keep on posting to directories.

thus, it seems quality page development, a little marketing attention and some common sense is the best strategy.

leaves more time for learning and less time bickering over whether or not that meta inclusion will help you on XYZ seach engine or not. ...everything changes at search engines except for the 'weight' of quality development.

christian "sebastian"

[idahoguy]

Just a little pointless blather on this...

In 95 it was pretty hard to get a grasp on who, where, and why people were using various search engines, not to mention that the reporting from places such as webproworld, SEW, and others...was not in place yet. The forums nowdays are great, and weren't even half of what they are today... back then.

Now as mentioned in the previous page, the circle is coming back, more and more engines popping up, more learning to be done, and more money/time to be spent.

On one hand, I can't stand the fact that Google has such a large "monopoly" on the search world... BUT, on my other hand, I love the fact that over the last 3 years, Google has averaged 86% of our referrals each year, making SEO simple and clear..... PLEASE GOOGLE and they please you.

Just like it simple, Google's dominance has made it simple for SEO firms like ours. I do not see them losing any ground in the next few years to MSN and Yahoo, nor any newcommers. Against my forecasts (google loosing referral percentage this year), Google has had a steady 2% rise in referrals to our site over the last few years.

Watching it snow in Idahoooooo!

[St0n3y]

I think this thread is treading close to a thread I started a while back. If we do end up with several dominant search engines, all pulling about equally, and we find it nearly impossible to get keywords ranked in the top 5 on all of them in most situations, how does an SEO go about showing a measure of success with the campaign?

Lets assume that no spamming, doorway pages, multiple domains, etc. are not being utilized, just one domain, with good optimization practices. I think there does have to be some sort of measure that balances marketshare, search performance, conversion rates, positions, etc. to produce a "score", if you will, and that score must give some sort of measure of the campaigns overall success considering the above factors.

As long as Google is the ONE engine, this measure isn't so necessary, but more engines, pulling more marketshare, each with vastly different algorithms, some kind of score system would be beneficial to the industry overall.

[Kal]

Originally Posted by Dave Hawley

At present, SEO firms would mostly be optimizing pages for Google. Stands to reason with Google being so popular.

Any SEO worth their salt would be optimizing pages in a way that makes them more compatible with a range of popular search engines and directories, not limiting themselves to a single engine. Those companies are already prepared for major shifts in the market share because that is the norm in this industry.

[AussieWebmaster]

Originally Posted by Kal

Any SEO worth their salt would be optimizing pages in a way that makes them more compatible with a range of popular search engines and directories, not limiting themselves to a single engine. Those companies are already prepared for major shifts in the market share because that is the norm in this industry.

It can be achieved - strong placement for numerous engines, but it is done by combining efforts. Inbound links -anchor text - for Google (from relevant sites as well as your own), keyword density for Yahoo and MSN - directories for all, strong basics covered for all, and fresh content, large page numbers, the list is extensive but attainable.

[Dave Hawley]

IMO, any SEO worth their salt would be optimizing mainly (not only) for Google. Being on page 1 in Google will, in most cases, deliver about 40% more traffic than Yahoo or MSN. If by, doing so, you also get page 1 in Yahoo or MSN etc great! However, if by making page 1 in Yahoo or MSN etc means page 2 in Google...very bad.

[Mel]

Originally Posted by AussieWebmaster

It can be achieved - strong placement for numerous engines, but it is done by combining efforts. Inbound links -anchor text - for Google (from relevant sites as well as your own), keyword density for Yahoo and MSN - directories for all, strong basics covered for all, and fresh content, large page numbers, the list is extensive but attainable.

Not really all that hard IMO Aussie. A fifty page site without much fresh content can rank well for competitive terms on the three major engines in my experience.

[Chris Boggs]

Just to get the record straight, we can and should optimize for all SE's. I believe that if you follow strict methodology to focus on the searcher while keeping Google in your peripheral vision, the other rankings will also occur.

[Chris Boggs]

Originally Posted by dannysullivan

The real, real change people aren't largely understanding is the shift away web results to specialty results. This is the invisible tabs concept I've talked about.

I did a Search Detours panel on this at SES San Jose, and the audience seemed largely confused. We talked about how search engines were shifting slowly away from being so dependant on web results. People didn't get it -- so what were the tips to follow, someone asked.

My response was that they needed to understand that down the line, in some cases, a search will bring back shopping results by default, or news results, or local results, or image results and so on. Web search results will be "backup," the stuff that kicks in only if other things aren't found.

So to come all the way back up to the start of this thread, it's not a problem to me that people might focus just on Google. It's a problem that they assume Google = Google Web Search and they may neglect the many specialized search engines Google and others offer. Those are the ones on the rise, in my view.

Will Google and the other big boys just buy these out and use them within "invisible tabs?"

I read the “invisible tabs” article and it is very relevant to this discussion. Funny how a whole year later Ask Jeeves still provides a better (IMO) result w/images rather than simply suggesting the “Try Google Images” for a search of “picture of dna” at each. You would think that this would automatically be the next step in order for Google to essential “keep up.”

Already Google provides a variety of “invisible tabs” built into the use of a command and colon within the box - from something as elementary as using the “define:” for a tough word or to check the spelling, to as specialized as using the “inanchor:keyword” search. Question is, since all the SE’s don’t seem to be using many visible tabs on their search home pages, are they going to skip the visible ones altogether?

Will this affect us? I don’t think so, because inclusion will still be attainable within all search engines as long as the content, popularity, and linking is there (certainly not necessarily in that order). Spots will of course exist with a “for sale” sign attached, but that is for a PPC discussion.

[Receptional]

I think that serious SEO firms - and the industry in general - would have died WITHOUT a new player coming in. To me, going back from one major player to three is a breath of fresh air and I am sure those of us that are so long in the tooth that we remember Inktomi being on Yahoo and Black Monday on Alta Vista will all agree.

Certainly, Google is a relatively recent phenomenon (as is pay per click by the way) and there were some serious technologies out there were deep crawling way before Larry and Sergei got a temporary monopoly.

But that is just it, isn't it? Temporary. Live by Google, Die by Google. Serious clients (and long term SEOs) can't risk that approach and would be foolish to do so. This has made for some careful maturing of SEO firms in the Google period. You can get to the top using black hat, or get to the top using white hat, but as time marches on, you can't do it white hat without brand, longevity and big bucks and (hopefully) actually BEING the best site (or one of the best) for the search phrase in question.

So we get back (let's hope) to content is king - or is at least a prince among thieves and judging a site's reputation based on link structures is what Google bought to the mix. But hey - who owns the Direct Hit technology these days? that is another way to measure reputation, as is technologies like Hitwise which could be brought into the algos. There is plenty left to do and competition is good for us as SEOs because it means the pace of change gives us a place within the scheme of things.

[Chris Boggs]

Wadabout optimizing for Accoona, the "SE with AI." This is certainly the use of invisible tabs...

probably going to move this post or delete it and suggest a new thread, but take a look at what I'm talking about if you haven't seen:

"Accoona plans to improve Web search with artificial intelligence and data on more than 10 million businesses. Its search engine will let people refine search queries with particular emphasis on certain words. Alternatively, people can look specifically for company information such as its address, phone number or revenue." article here

[idahoguy]

I just checked one of our top listings with all the major players. The KW phrase I checked for is EXTREMELY popular, and just last month we finally acheived the number one position on Yahoo, MSN, and yes, now Accoona.

I look in Google, not even in the top 100!!!

There are no tricks or black SEO ops getting our page to rank #1 in the other sites, just good content and good links. That is why I get annoyed with Google. Yes, our page has good info on the given subject, but not to many links pointing to it.

Thus, another Google failure to rank the page well because of the lack of links.... gotta luv their warped SERPs.

[St0n3y]

Originally Posted by Chris Boggs

Just to get the record straight, we can and should optimize for all SE's. I believe that if you follow strict methodology to focus on the searcher while keeping Google in your peripheral vision, the other rankings will also occur.

I don't know if its feesible to optimize for "all" search engines. Not to say you cannot perform well for many just by good optimization tactics. We made a decision to target and report only on the top 5 engines, or whatever covers 90% of the search marketshare.