Does New Google Patent Validate Sandbox Theory?

[msgraph]

Must read

Probably one of the best bits of information released by them in a patent.

Large number of inventors listed on here, even Matt Cutts that guy who attends those SE conferences . Explains a bit about what is already known through experience as well as comments made by search engine representatives.

Example:

[0039] Consider the example of a document with an inception date of yesterday that is referenced by 10 back links. This document may be scored higher by search engine 125 than a document with an inception date of 10 years ago that is referenced by 100 back links because the rate of link growth for the former is relatively higher than the latter. While a spiky rate of growth in the number of back links may be a factor used by search engine 125 to score documents, it may also signal an attempt to spam search engine 125. Accordingly, in this situation, search engine 125 may actually lower the score of a document(s) to reduce the effect of spamming.

Information retrieval based on historical data

[I, Brian]

Bloody interesting read. Not only does it look like the Sandbox Uncovered, it's got Matt Cutt's name on it!!

[Phoenix]

Brilliant find, does explain a few things, but then again not everything. I wonder why we haven't found this sooner. Last sentence in the document is particularly interesting in how it explains the "boost" new sites normally receive after initially getting indexed. You do well for awhile, and then you crash later. There is for a "lag" on scoring for this it seems.

Accordingly, in this situation, search engine 125 may actually lower the score of a document(s) to reduce the effect of spamming.

[Phoenix]

"Also, "stale" documents (i.e., those documents that have not been updated for a period of time and, thus, contain stale data) may be ranked higher than "fresher" documents (i.e., those documents that have been more recently updated and, thus, contain more recent data). In some particular contexts, the higher ranking stale documents degrade the search results."

Here is a line to support the "freshness" of a site does better line of thinking. It appears older and "staler" is not always better.

[bhartzer]

I wonder why we haven't found this sooner.

We couldn't find it sooner. Notice the date on the patent, March 31, 2005.

I'm particularly interested in this part:

38. The method of claim 1, wherein the one or more types of history data includes domain-related information corresponding to domains associated with documents; and wherein the generating a score includes: analyzing domain-related information corresponding to a domain associated with the document over time, and scoring the document based, at least in part, on a result of the analyzing.

40. The method of claim 38, wherein the domain-related information is related to at least one of an expiration date of the domain, a domain name server record associated with the domain, and a name server associated with the domain.

It looks like I'll now be renewing my domains for additional years now. I've heard that you can now register your domain from NetSol for the next 100 years.

[SEjack]

But what does this all mean. its a patent and that doesent mean that this is how the SE works. Correct me if I am wrong but...this dosent mean anything.

[dannysullivan]

If they've taken the time to patent this, it strongly suggest they are using some of what's described. You're correct that it doesn't explain exactly how the search engine works completely or that any or all of what's described is being used. But part of what are described fit in with things some people have been scratching their heads over -- the new info seems to neatly explain some of what's been seen.

[SEjack]

I see what you mean Danny. Great points. I am so glad I joined this forum filled with great knowledgeble members.

[Mike Grehan]

For what it's worth, I wrote a long paper about network theory which covers some of this (Filthy Linking Rich).

I'm with my dear friend and supporter Dr. Edel Garcia. If we don't start and pay attention to information retrieval science, which Edel is an expert in, and even listen to my own musings on network theory - we're screwed!

If we talk about anecdotal nonsense over and over again, just to point clients in the direction of some substance. And then fail miserably over and over again...

It's just as I said in New York about "do we need to know how a search engine really works."

And this supports the article Edel wrote about keyword density analysis yesterday (I can't find where it is in the forum - help moderator!).

This industry is growing - we better learn to grow with it.

[rustybrick]

38. The method of claim 1, wherein the one or more types of history data includes domain-related information corresponding to domains associated with documents; and wherein the generating a score includes: analyzing domain-related information corresponding to a domain associated with the document over time, and scoring the document based, at least in part, on a result of the analyzing.

I am happy this came out as well.

Matt Cutts at SES NYC told me, we did not become a register to register domain names. He went on to explain exactly what is written above. Why didn't I say anything? Because we all knew it, but now its on paper.

[dazzlindonna]

The user behavior parts of the document also fits right in with the purchase of Urchin.

[I, Brian]

Check out 114-116 - toolbar data usage for ranking purposes.

[PhilC]

5. The method of claim 2, wherein the inception date corresponding to the document is based on at least one of a date when a search engine first discovers the document, a date when a search engine first discovers a link to the document, and a date when the document includes at least a predetermined number of pages.

Alright. What does that last bit mean? Up to then I'd assumed that a document *is* a page.

[Frank Kilkelly]

From the DETAILED DESCRIPTION section

[0020] A "document," as the term is used herein, is to be broadly interpreted to include any machine-readable and machine-storable work product. A document may include an e-mail, a web site, a file, a combination of files, one or more files with embedded links to other files, a news group posting, a blog, a web advertisement, etc. In the context of the Internet, a common document is a web page. Web pages often include textual information and may include embedded information (such as meta information, images, hyperlinks, etc.) and/or embedded instructions (such as Javascript, etc.). A page may correspond to a document or a portion of a document. Therefore, the words "page" and "document" may be used interchangeably in some cases. In other cases, a page may refer to a portion of a document, such as a sub-document. It may also be possible for a page to correspond to more than a single document.

[Relevancy]

So basically they are analyzing register information for how many years you are dedicating to the domain and because they want to see the interconnectivity of all your domains?

Then they are looking at fresh index types of data such as staleness of a page and how many links point to it. Plus if those links stay pointing to it or not?

[Phoenix]

17. The method of claim 1, wherein the one or more types of history data includes information relating to search terms that increasingly appear in search queries over time; and wherein the generating a score includes: determining whether the document is associated with the search terms, and scoring the document based, at least in part, on whether the document is associated with the search terms.

The above might be of interest to Dr. Garcia, and validates the importance of considering linearization in optimization he mentioned, as well as the importance of looking how tokenization and especially filtering applies to your documents. You can then use that information obtained from the study so that you don't end up on the wrong side of what is mentioned in the quote above: a document not associated with the search terms used.

There is also mention on something I have studied more closely about how in the patent it tries to detemine if the domain has changed owners. It says "[0128] a significant change over time in the set of topics assoicated with a document may indicate that the document has changed owners and previous document indicators, such as score, anchor text, etc..are no longer reliable."

I know a few people that have avoided the "sandbox" so to so, by "rehabing" older domains in order to set up new content or pages on the domain. For the most part this works if done without extending the domain to much as much as I have discovered. As it says "Similarly, a spike in the number of topics could indicate spam...that the document has been taken over as a "doorway" document". I don't think this was something before you would consider when rehabing a domain, but it sure will be know. At least for me.

Google is also acknowledging the following while describing how it gathers "User Maintained/Generated Data" in its patent filings

"According to an implementation consistent with the principles of the invention, user maintained or generated data may be used to generate (or alter) a score associated with a document. For example, search engine 125 may monitor data maintained or generated by a user, such as "bookmarks," "favorites," or other types of data that may provide some indication of documents favored by, or of interest to, the user. Search engine 125 may obtain this data either directly (e.g., via a browser assistant) or indirectly (e.g., via a browser). Search engine 125 may then analyze over time a number of bookmarks/favorites to which a document is associated to determine the importance of the document.

[0115] Search engine 125 may also analyze upward and downward trends to add or remove the document (or more specifically, a path to the document) from the bookmarks/favorites lists, the rate at which the document is added to or removed from the bookmarks/favorites lists, and/or whether the document is added to, deleted from, or accessed through the bookmarks/favorites lists. If a number of users are adding a particular document to their bookmarks/favorites lists or often accessing the document through such lists over time, this may be considered an indication that the document is relatively important. On the other hand, if a number of users are decreasingly accessing a document indicated in their bookmarks/favorites list or are increasingly deleting/replacing the path to such document from their lists, this may be taken as an indication that the document is outdated, unpopular, etc. Search engine 125 may then score the documents accordingly. "

I don't know about you, but reading this particular part definitely gave me chills. After posting this article I guess I will let the Google toolbar go.

[Scoreboard]

I wouldn't worry too much about anything in this patent. Pragmatically, it conflicts it's own aims, and if I were a betting man, Google floated that purely to lay claim to any conceptual-based arguments to be made by competitors going forward.

It really sounds like a bunch of engineers got high on the Tahoe ski trip and whiteboarded their thoughts.

[Relevancy]

I agree, it does sound like a lot of clutter to mask a few key ideas. Most of it is "ifs" and "in some ways". The basic idea is they seem to be analyzing register information and then they go into how the value fresh vs stale pages.

[Mel]

It gives me the chills too Whistleman but there are a couple of questions I have regarding this patent, the first of which is why is it not in Googles name or at least assigned to Google?

Secondly the idea of rankings based at least partially on how often a page is accessed from bookmarks reminds me of Alexa, just too easy to manipulate.