View Full Version : What's The Legality Of Google Blocking Rank Checkers?
Google uses some pretty strict and to the point verbage on the idea of automated engines hitting there site and scraping the results:
"Do not use unauthorised computer programs to submit pages, check rankings, etc. Such programs consume computing resources and violate our Terms of Service. Google does not recommend the use of products such as WebPosition Gold™ that send automatic or programmatic queries to Google."
My question is this, how could this possibly be enforced? Google is a free public facing internet site. With that said, how can they say who can hit their site and who can't? Someone who uses a http request to send a search query to there site is logically no different to them than me going to their site and doing the same thing.
They can say whatever they want as far as violating their terms etc, just as I could wear a t-shirt that says if you look at me I will sue you. The real question is how enforcable is their policy? Any comments?
Does anyone know if existing tools that do this are facing legal problems with Google at this time, or are they just hoping Google will leave them alone? Has anyone been shut down by Googles legal dept with a cease-and-desist order?
Even further, why would they want to stop this behavior? I don't see how it promotes behavior that tries to "cheat" their system. All it does is report current page rankings for keyword searches. What's the big deal?
AussieWebmaster
03-06-2006, 11:53 AM
what ever the publishing platform copyright holds... if you walked into a news stand and started giving away the papers do you think you will not get arrested....
what is your domain I will copy all your stuff and add it to my high PR site so you get booted from the SERPs....
lines are lines and rules are rules....
ncpain
03-06-2006, 12:29 PM
AussieWebmaster,
Your post seems to be filled with anger. I don't think that TJ69 was trying to say anything was right or wrong but instead asking some interesting questions.
Based on TJ69's post (and I believe it to be acurate after reading the Google Terms of Use Policy myself) it seems that the software that is available to track your domain's ranking in Google and other search engines flys in the face of Google's very plain english Tersm of Use Policy (TOU).
There is a short bit in the TOU that says something about contacting Google prior to developing a meta search tool. Does anyone know if any of these tools have done this?
I don't want to spend a bunch of time permoting my site only to use a tool that will somehow get me booted from the search engine all together. An equal concern is to purchase such a tool (some as much as $500) only to have the producer of the tool go out of business because Google shuts them down.
AussieWebmaster
03-06-2006, 12:35 PM
AussieWebmaster,
Your post seems to be filled with anger. I don't think that TJ69 was trying to say anything was right or wrong but instead asking some interesting questions.
Based on TJ69's post (and I believe it to be acurate after reading the Google Terms of Use Policy myself) it seems that the software that is available to track your domain's ranking in Google and other search engines flys in the face of Google's very plain english Tersm of Use Policy (TOU).
There is a short bit in the TOU that says something about contacting Google prior to developing a meta search tool. Does anyone know if any of these tools have done this?
I don't want to spend a bunch of time permoting my site only to use a tool that will somehow get me booted from the search engine all together. An equal concern is to purchase such a tool (some as much as $500) only to have the producer of the tool go out of business because Google shuts them down.
I really am not angry was using the same style of reply as was used here:
They can say whatever they want as far as violating their terms etc, just as I could wear a t-shirt that says if you look at me I will sue you. The real question is how enforcable is their policy? Any comments?
That's a good start, but doesn't really answer my question. You saying "whatever the publishing platform copyright holds" is a pretty ambiguous statement. What do you base this opinion on? What is the legal presidence on this? There are many gray areas in copyright law, and i'm sure this issue does not have a tremendous amount of case law behind it for guidance. So again, what are you basing your response on besides your own personal opinion?
Granted, there is an ethical angle which there is to consider. However, I would go out on a limb and say that scraping search engine results, parsing them, and displaying the results in another application so webmasters such as yourself can see how there efforts are paying off hardly seems unethical or deceptive. It's not like i'm talking about creating a search site which behind the scenes scrapes googles results and posts them on my site as if they were my results. All i'm taling about is being able to tell a webmaster, hey your hardwork on content, and your link exchange has gotten your keyword ranking on Google from 12th to 10th for whatever page.
As far as your comment about you copying content from my domain and putting it on your **high** PR site so bla bla bla... I was mearly asking a question, to see what other thought and had an opinion on. You know, the whole reason behind forums...after all, you are a "WEBMASTER"
AussieWebmaster
03-06-2006, 12:41 PM
That's a good start, but doesn't really answer my question. You saying "whatever the publishing platform copyright holds" is a pretty ambiguous statement. What do you base this opinion on? What is the legal presidence on this? There are many gray areas in copyright law, and i'm sure this issue does not have a tremendous amount of case law behind it for guidance. So again, what are you basing your response on besides your own personal opinion?
Granted, there is an ethical angle which there is to consider. However, I would go out on a limb and say that scraping search engine results, parsing them, and displaying the results in another application so webmasters such as yourself can see how there efforts are paying off hardly seems unethical or deceptive. It's not like i'm talking about creating a search site which behind the scenes scrapes googles results and posts them on my site as if they were my results. All i'm taling about is being able to tell a webmaster, hey your hardwork on content, and your link exchange has gotten your keyword ranking on Google from 12th to 10th for whatever page.
As far as your comment about you copying content from my domain and putting it on your **high** PR site so bla bla bla... I was mearly asking a question, to see what other thought and had an opinion on. You know, the whole reason behind forums...after all, you are a "WEBMASTER"
I guess I came off a little testy today... the example of copying your stuff was intended to ask- how would you react if someone was doing it to your site?
If I wrote an article and somebody plagerized it without consent or credit I would without a doubt not be happy. I guess I just don't feel like someone would be plagerizing Google by scraping their site, and showing some statistics on the results to help people out with seo campaign. However, being on the side of the person wanting to do this, there is no doubt I have a biased opinion. That's why I posted this question. I wanted to find out if others thought it should be okay, not okay, or somewhere in between. I also wanted to find out if there was any precedence on this issue. I guess nobody knows or cares to answer since I haven't gotten the answers I was looking for.
dannysullivan
03-06-2006, 03:03 PM
Legally, they might use a trespass argument. eBay successfully did that against Bidder's Edge, saying they effectively trespassed and cost them money by gathering up results.
Realistically, what Google's done to date is to shut down the IP addresses from sites they think are hitting them in ways that go against their terms.
http://news.com.com/2100-1023-883558.html from News.com back in 2002 was a good public example of this, of 100 Comcast subscribers finding Google access blocked because one of them was hitting Google too hard.
ncpain
03-06-2006, 03:05 PM
I guess I came off a little testy today... the example of copying your stuff was intended to ask- how would you react if someone was doing it to your site?
That is a great question Aussie. I personal would be very upset and want what belongs to me removed. However, I say this under the context that you have taken my content and made it your own. If you gave me credit for it and used it as a bases for your research or commented on it then I might not react the same way and even if I did I might not be able to do anything about it.
Back to the tools thing... are the tools that are out there sliding by because they they are quantifing the results as data owned by Google and they are just quoting Google (for lack of a better term).?
ncpain
03-07-2006, 03:49 PM
An interesting point of view that I hadn't thought about until today was Googles.
There are obvious reasons why they wouldn't want you using automated tools.
#1 - If you can gain a high organic ranking why would you ever buy an add? This cuts into there bottom line... ad words.
#2 - Resources could run thin if many people were quering Google using automated queries. This kind of activity would be similar to a DOS attack. Please keep in mind I know that Google has more bandwidth than God... I am only saying that it would be simmilar a DOS attack.
From the article posted above I got another interesting point of view. A business has an inherent right to know how Google is listing the business in it's search results. If businesses started to turn off Google and not allow them to spider there sites because Google wouldn't allow a business to look at how they are being listed how much value would others find in the search results?
Google browses sites with automated tools, right? And then they tell the owners of those sites that they can't use automated tools to look at Google, right?
That doesn't really seem fair to me.
Marcia
03-07-2006, 07:46 PM
Google browses sites with automated tools, right? And then they tell the owners of those sites that they can't use automated tools to look at Google, right? You don't have to let Googlebot browse your sites at all; you can disallow them in robots.txt
Y'all have the same free choice as to what you will allow on your respective sites as Google has as to what they'll allow on theirs. YES, they have banned websites for hitting them excessively with automated tools. And YES the site owners had a freedom of choice in the first place whether to do it or not - in other words, whether to observe Google's TOS or not.
PhilC
03-13-2006, 07:38 AM
It's not really a legal issue - it's a Google's rules issue, and, like any other website owner, they are allowed to make any rules they wish concerning their own site. As Danny said, they enforce the rule by blocking IP addresses.
The argument that Google automatically fetches pages from my site, so I can automatically fetch pages from their site, is fair. As long as both sides abide by the robots.txt protocol, the argument can't be faulted. Google's robots.txt file includes the line:-
Disallow: /search
That line disallows all organic search results, and is applied to all robots, including those that fetch search results - automated searches.
Incidentally, it isn't anything to do with ads, ncpain. They had and enforced the rule from the beginning - long before they went into advertising.
Blocking the ip of the offending search engine is a great idea in theory, however, it isn't hard to find ways around hiding your ip. There are numerous proxy seriveces out there which lend themselves very nicely to this practice, or writing one of your own wouldn't be all that difficult either.
I guess the reason I was looking into the "legal" side of things is this: Google says no search engines, but people will find a way to crawl their site one way or another. With that, I would be surprised to find this hasn't already happened....and was wondering how far Google would take it. Would they or have they already tried to use the courts to enforce their policy? I just don't see how they could do it themselves no matter how strong their wording, or blocking ip's or whatever, and obviously they haven't because there are known sites out there that are crawling Google as we speak.
Maybe it's just a simple cost vs convenience situation. As long as these sites are costing Google more than it would cost them to lititgate the situation they just let it go??